HAProxy internal DNS resolution flooding DNS with queries

luisramos

I've found the same issue on my appliance and applied successfully the solution.
But I noticed that it's flooding the DNS server with A and AAAA queries every second.
Where can I stop this?

HAProxy internal DNS resolution issue on backend servers

luisramos

Found the bug.

I configured on HAProxy 'Global DNS resolvers for haproxy' a separate DNS for internal private domains:
Pfsense doesn't have have this DNS server on the list
that was why I was getting an error 'unable to resolve <fqdn>' when clicking 'apply'

even if the HAProxy can resolve the name and the load balancing works, somehow it creates a loop resolution error with the PFsense DNS resolution and retries every second.

Workaround:

on PFsense 'system>general setup' configure that DNS server and thick 'Disable DNS Forwarder'
clear the Global DNS resolver for haproxy' list on the 'HAProxy > Settings'
you don't need to set anything in 'Global Advanced pass thru > Custom options'

PiBa

@luisramos
And now when the IP of your server changes haproxy wont see that..

What did you configure for the 'Interval' (hold valid) setting of haproxy for its DNS settings?

luisramos

@PiBa it's empty right now (default=10)
I already change the IP address of the server as a test. It updates the DNS (TTL=300) and then PFsense dns client will resolve for HAproxy

The interesting observation is that if I put a dns server (even non-existing) my DNS server will get with DNS queries every second from Pfsense.

PiBa

@luisramos
Haproxy does not look at the TTL of the dns record.. but should with the default 'hold valid' setting query only once every 10 seconds.. that is assuming it did get a 'valid' response.

Can you run on a ssh/console this command?:

/usr/local/pkg/haproxy/haproxy_socket.sh show resolvers

It will likely tell that it got invalid responses.?

luisramos

Since there no DNS is configured on HA proxy, the output is:
"No resolvers found"

Setting the DNS on HA proxy, I will start getting the queries every second on the DNS server, and the output shows:

show resolvers Resolvers section globalresolvers
 nameserver InternalDNS:
  sent:        112
  snd_error:   0
  valid:       56
  update:      0
  cname:       0
  cname_error: 0
  any_err:     56
  nx:          0
  timeout:     0
  refused:     0
  other:       0
  invalid:     0
  too_big:     0
  truncated:   0
  outdated:    0

PiBa

@luisramos
Ah found it.. haproxy webgui is writing 'hold valid' in the config which is no longer used to determine the interval for valid queries by the binary.. (it changed: http://git.haproxy.org/?p=haproxy.git;a=commit;h=f50e1ac )

If you replace in the file /usr/local/pkg/haproxy/haproxy.inc the 'hold valid' valid by 'timeout resolve' (around line 1571) and restart haproxy then you should start seeing queries only every 10seconds.. So it would then look like this:

			fwrite ($fd, "\ttimeout resolve {$tmp}\n");

Ill send this change also on github, to be included in a future haproxy-pfSense package..

luisramos

Problem solved!
Thanks for the quick support action ;)

Note: I had to disable libc resolution to get it working (in Global > custom options)

defaults
   default-server init-addr last,none