3

Environment

  • Ubuntu 22.04 LTS
  • Netplan configuration

Configuration

Multiple DNS servers configured:

network:
  version: 2
  renderer: networkd
  ethernets:
    eth0:
      dhcp4: true
      nameservers:
        addresses: [192.168.1.1, 8.8.8.8, 1.1.1.1]

Questions

  1. What algorithm does systemd-resolved use to choose which DNS server to query first? Does order matter?

  2. If DNS resolution succeeds but the HTTP request fails (403, etc.), will systemd-resolved retry with a different DNS server?

  3. Can I enforce strict priority ordering so servers are tried in exact specified order?

  4. After failover to a secondary DNS server, will it try the primary again automatically? Or requires netplan apply?

  5. Can systemd-resolved rotate through all configured DNS servers for load balancing?

Found contradictory information - some say order matters, others claim systemd-resolved uses complex logic ignoring specified order. Need clarification on actual behavior.

Ilgar
  • 41

3 Answers3

5

How does systemd-resolved prioritize DNS servers in Ubuntu?

It can get complicated and it depends.

Systems resolvd allows per link DNS servers and provides an API for applications and daemons like for example VPN clients to dynamically register DNS servers and/or search domains when the VPN gets activated.

resolvectl status should give you an idea of the current effective systemd-resolved configuration though.

The most simple scenario is to only have one or more name servers in the classic /etc/resolv.conf similar to what you seem to have have in your net plan.

That is the same as having only DNS= entries and no FallbackDNS= the resolved.conf configuration file. Those name servers will get queried in parallel.

What algorithm does systemd-resolved use to choose which DNS server to query first?

The applicable DNS servers all get queried in parallel.

Does order matter?

No. That was how the classic glibc resolver would work (each query there would always use the first name server in /etc/resolv.conf and only when it encountered an error from that first nameserver would the next ones be tried in order) but systemd queries all equivalent DNS servers in parallel, and although there is the FallbackDNS= setting that is not quite the same behavior.

If DNS resolution succeeds but the HTTP request fails (403, etc.), will systemd-resolved retry with a different DNS server?

No. For Systemd-resolved and DNS resolution there is no failure.

In a custom application you can do whatever makes sense for you but for the typical web client there was a host found, a web request was made and the server returned a clear response. Unfortunately an error code response but that is not a DNS resolution failure or something that a resolver needs to fix.

Can I enforce strict priority ordering so servers are tried in exact specified order?

Not as far as I know (and order shouldn’t matter, each equivalent DNS server should get you a result)

AFAIK You would need to not use systemd-resolved and let the classic glibc resolver do its thing. That did query (max 3) nameservers in the order they are defined in /etc/resolv.conf where always the first one gets used. (Unless tuned with for example options rotate) Only when that first doesn't doesn’t respond at all or responds with certain errors will the second nameserver be used as fallback or the third when both previous ones fail.

After failover to a secondary DNS server, will it try the primary again automatically? Or requires netplan apply?

What failover?

What secondary?

All configured DNS servers get queried in parallel.

When they all fail the FallbackDNS= servers get queried for that failed query. AFAIK When a subsequent query is made it again first goes to the DNS= servers and in case all of those still don’t respond the fallback happens for that query too.

Can systemd-resolved rotate through all configured DNS servers for load balancing?

Not that I know. It uses all of them in parallel rather than one after the other.

Load balancing is normally not done by a client anyway but rather by the provider of the service.

HBruijn
  • 84,206
  • 24
  • 145
  • 224
2

Can I enforce strict priority ordering so servers are tried in exact specified order?

When someone asks for an exact specified order, most of the time it's because their servers give different answers: the 192.168.1.1 knows about LAN hosts while the other servers do not.

An exact specified order would not help with that, even if it were an option. All resolvers distinguish between two types of "failures": complete lack of response (timeout or total server failure) triggers fallback to a different server, but a negative response (domain not known) does not – even a "no such domain" answer is still considered to be the final answer to the query and the resolver won't look any further than that.

So you wanted this because you have servers that have LAN-only names, you should use exclusively those servers.

(If you have multiple sets, e.g. two separate LANs with different internal subdomains, systemd-resolved has per-interface DNS lookup routing.)

grawity
  • 17,092
2

Your questions and your configuration suggest that you have one internal DNS server 192.168.1.1 and also you seem to want to use two public resolvers, the well known Google's 8.8.8.8 public resolver and CloudFlare's 1.1.1.1.

Background

Typically an internal name server can answer DNS queries for your internal hosts and subdomains like printer.lan.example.com , intranet.example.com , maybe you have assigned a subdomain like *.ad.example.com to your Active Directory domain etc.
Most internal resolvers can also DNS queries for internet hosts.
Usually the internal name server can also answer reverse DNS lookups for host in your internal network, i.e. it will be authoritative for the RFC 1918 private IPv4 address ranges used in your internal network and have records like
1.1.168.192.in-addr.arpa. IN PTR router-gateway-ns1.lan.example.com.

Public resolvers only resolve internet hosts. They don't provide reverse DNS for RFC 1918 private use internal IP-address ranges and they don't revolve hosts that only exist in your internal DNS. ( Technically it is possible to publish DNS resource records for internal hosts in your public internet DNS zone, but convention and security best practices advise against that.)

The problem

systemd-resolved expects that all DNS servers that are configured at the same level, like you do with nameservers: addresses: [192.168.1.1, 8.8.8.8, 1.1.1.1] to behave the same and will use them concurrently.

By mixing an internal name server with two public resolvers, you have some that don't behave the same and you may get seemingly inconsistent and erroneous behaviour where sometimes requests for internal host will fail (because the responses from public resolvers takes precedence) and sometimes they will succeed (because the response from your internal name server takes precedence).

Solution

  1. Only configure internal name server(s) in your netplan config as AFAIK there is no support to define preferred and fallback name servers:
network:
  version: 2
  renderer: networkd
  ethernets:
    eth0:
      dhcp4: true
      nameservers:
        addresses: [192.168.1.1]
  1. When you still want to use public resolvers as fallback to your internal name server(s): AFAIK there is no netplan syntax, but that is supported in systemd-resolved ; You'll need to edit /etc/systemd/resolved.conf and/or create a drop-in /etc/systemd/resolved.conf.d/*.conf snippet that sets FallbackDNS=8.8.8.8 1.1.1.1
HBruijn
  • 84,206
  • 24
  • 145
  • 224