IMHO these are basically Paperweights since there is no OpenWRT Support available for them right now and neither there will be in the Future

.
Only issue is I assigned them hostnames via dnsmasq but i cannot login unless i use the ip.
That makes zero Sense

.
Do you mean you assigned hostnames via
dnsmasq on your upstream router (e.g. OpenWRT) ?
Are you sure you are maybe not having a Web Browser such as Librewolf / Brave (possibly others) that is trying to force to use DoH (DNS over HTTPS) / DoT (DNS over TLS) using some custom DNS Servers, instead of your System DNS Servers ?
Are you sure you don't have some DNSSEC Issues ? On Ubuntu sometimes I was having Issues with DNSSEC for some Internet Domains, so I could see it in the Logs with
journalctl -xeu systemd-resolved or just
resolvectl query hostname.MYDOMAIN.TLD.
Are you sure you have configured DHCP Static Mapping correctly ?
Check that the hostname is NOT resolving using IPv6 to another Device, which probably doesn't exist / doesn't Reply.
Try
curl -L -vvv http://hostname.MYDOMAIN.TLD,
ping -4n hostname.MYDOMAIN.TLD as well as
traceroute hostname.MYDOMAIN.TLD
By definition DNS resolves a hostname to an IP Address, so they should be one and the same.
Recently (last 4 Years ???) though, Web Browsers sometimes force the use of their own DNS Servers, thus some unexpected behaviours could arise.