aiodns does not work with mDNS and is the default resolver

[gsmecher]

Describe the bug

With aiohttp v3.10.10, mDNS hostnames do not resolve when aiodns is installed.

When I force the use of a ThreadedResolver, or when I uninstall aiodns, mDNS hostnames resolve fine.

I see (#8522) that aiodns was made the default resolver again, recently, which explains the regression. The underlying c-ares library has a long-time open request for mDNS support (c-ares/c-ares#171) and no movement over the past few years.

To Reproduce

You will need a mDNS-resolvable service for this to work. (If you have avahi-daemon installed, your hostname with a .local suffix should suffice.)

>>> import aiohttp
>>> import asyncio
>>> async def main():
...    async with aiohttp.ClientSession() as session:
...        async with session.get('http://some_mdns_hostname.local') as resp:
...            print(resp.status)
...            print(await resp.text())
>>>asyncio.run(main())
ClientConnectorDNSError: Cannot connect to host rfmux0033.local:80 ssl:default [Domain name not found]

Expected behavior

Correct resolution of the hostname via mDNS.

Logs/tracebacks

ClientConnectorDNSError: Cannot connect to host some_mdns_hostname.local:80 ssl:default [Domain name not found]

Python Version

$ python --version
Python 3.12.8

aiohttp Version

aiohttp.__version__
'3.10.10'

multidict Version

multidict.__version__
'6.1.0'

propcache Version

Not installed.

yarl Version

yarl.__version__
'1.13.1'

OS

Debian Linux (testing)

Related component

Client

Additional context

No response

Code of Conduct

• I agree to follow the aio-libs Code of Conduct

[Dreamsorcerer]

Perhaps we can bypass aiodns for .local or something, in order to improve the situation?

[bdraco]

At least for Home Assistant, .local domains get resolved by the local DNS server, so there's no need to fall back to ThreadedResolver for these.

While .local domains are rare enough that it probably wouldn't matter that much for performance (unless all of the domains the user is resolving are .local) if we did force the ThreadedResolver for .local when using AsyncResolver, mDNS resolution is usually implemented via some type of glibc plug-in (sometimes https://github.com/avahi/nss-mdns) or similar so it wouldn't cover all the cases. If you are using musl it might be tied into the DNS server like Home Assistant is doing. Adding fallback for .local feels like a bit of a slippery slope to start trying to guess what the behavior the user wants so I'm -0 on adding it to AsyncResolver.

If your target system is making use of such plug-ins, it won't be obvious to aiohttp, and I would recommend creating the threaded resolver manually instead. The downside is the performance hit can be substantial as the threading cost has to be paid for all domains and not only .local domains, this is especially relavant if you aren't accessing the same domains over and over as with that case resolution is mostly cached.

If fallback for .local is something we are really want to do I'd be more comfortable if we added a new hybrid resolver class so the fallback behavior wasn't forced for AsyncResolver. We could consider making the new resolver class the default for 3.12, but considering AsyncResolver has been the default for 3.10+, and we are already shipping 3.11 now, I wouldn't want to change it in a patch release.

[gsmecher]

I have added a workaround for our codebase (gsmecher/tuberd#54), so we're OK for now.

Our aiohttp code runs client-side on a PC, not in an embedded appliance - so we don't control the execution environment enough to insist on any local infrastructure (DNS cache, DNS server, or resolver modifications). In fact, this lack of control over client networks is why we went with mDNS in the first place. It's a shame to see support moving backwards here.

[bdraco]

so we don't control the execution environment enough to insist on any local infrastructure (DNS cache, DNS server, or resolver modifications)

On a side note, since you don't control the environment, using zeroconf to resolve the hosts might be a better fit for your use case as it's not going to dependent on the libc stack or nss configuration as even with ThreadedResolver mDNS is likely not to work on Alpine or any Linux using musl without some local dns server configuration.

[bdraco]

I think making something like aiohttp-asyncmdnsresolver would be a good solution here, as we could have zeroconf as a dependency, and it should work on all platforms.

I wouldn't want to add that in base aiohttp, as adding zeroconf as a dependency to aiohttp directly seems like a nonstarter given how narrow the use case and potential user base overlap is too small to carry such a dependency

[gsmecher]

I understand where you're going with this, and the reason why an async DNS resolver is desirable for this stack. However, for our use case, giving up mDNS resolution via NSS is a bridge too far - we'd lose the ability to ping, ssh, and curl from these .local domains. (We also host a HTTP interface - so browser-based .local resolution is user facing.) This means two things:

• Any linux that doesn't do this (musl/Alpine) has dropped enough of our network stack that its users are in a "you broke it, you bought it" situation. We typically deploy on Debian or Ubuntu, and occasionally bump into people running OS X or Windows. IIRC there's a broad minimum support for .local across all these platforms.
• From my perspective, dropping mDNS support for the sake of an async resolver seems like an odd prioritization of form over function. (When the same thing happened with IPv6 breakage, the code got rolled back.) Surely mDNS is not so rare that we're the only ones in this bucket.

Again, I understand the what and why. I'm also looking a gift horse in the mouth - thanks for the excellent work you do. We will carry forward our patch enforcing the use of NSS-based resolution for now. I'm just registering a small frown.

[bdraco]

I understand where you're going with this, and the reason why an async DNS resolver is desirable for this stack. However, for our use case, giving up mDNS resolution via NSS is a bridge too far - we'd lose the ability to ping, ssh, and curl from these .local domains. (We also host a HTTP interface - so browser-based .local resolution is user facing.)

For nearly all systems (there is always some corner case), there isn't any reason your dns plugins and zeroconf can't both co-exist on the same system. We see this type of configuration all the time for Home Assistant users. You would still need to be careful not to create hundreds of instances of zeroconf but its also designed to be able to share the instances between all apps running in the event loop.

[bdraco]

I'm going to try to work on a fully async aiodns+zeroconf based resolver this weekend if I don't run out of time.

[bdraco]

I've create an initial version of https://github.com/aio-libs/aiohttp-asyncmdnsresolver

I still have some more docs to write for it but the basic usage is

https://github.com/aio-libs/aiohttp-asyncmdnsresolver/blob/main/examples/run.py

[bdraco]

https://github.com/aio-libs/aiohttp-asyncmdnsresolver 0.0.1 is available with docs and example