NTP set to time.google.com not working after reboot

rpsmith

I'm not using KEA and it works flawlessly when I restart the NTP service.

Last time I tried switching to KEA it stop renewing leases and I had to scramble to reset a bunch of firewalls to stop using it. I'm not impressed.

Roy...

johnpoz

@rpsmith kea would have zero to do with dns or ntp - ZERO!!!

See my edit above about setting ntp to prefer IPv4 dns, and also how to stop unbound using IPv6 or trying and or answering a client that asks for a AAAA

Kea doesn't like fqdn for ntp that you would hand to your clients.. Because per RFC setting a ntp server in dhcp is an IP only thing.. ISC and pfsense just resolves it before handing it out via a dhcp lease to some client. maybe kea in the furture or current iteration on pfsense does that now - but its bad to let clients people think you can put in fqdn for ntp server to hand to clients - because the dhcp that is not borked isn't going to do that.. The rfc clearly states IP for ntp servers. That pfsense ever allowed the option in the first place was a mistake if you ask me.

rpsmith

I'm not using unbound. I'm using forwarder to forward DNS lockups to my two pi-holes.

Roy...

johnpoz

@rpsmith unbound can forward.. Do you have pfsense asking loopback 127.0.0.1 or ::1 which would be service running on pfsense.

You can for sure do the same setting in pihole, because unbound can be used on pihole. You can prob look in pihole on how else to not return AAAA for clients that ask, but almost all clients will ask for both AAAA and A when looking up something.

But if you have no IPv6 and ntp/pfsense thinks it should talk to something via IPv6 since it got back an AAAA back - then yeah your prob not going to have a good day.

Patch

@johnpoz said in NTP set to time.google.com not working after reboot:

kea would have zero to do with dns or ntp - ZERO!!!

Release note suggest there is an interaction
https://docs.netgate.com/pfsense/en/latest/releases/2-8-0.html

DHCP (IPv4)
Fixed: Kea does not allow FQDNs for NTP servers but input validation does not prevent them from being added #14991

dennypage

@Patch said in NTP set to time.google.com not working after reboot:

@johnpoz said in NTP set to time.google.com not working after reboot:

kea would have zero to do with dns or ntp - ZERO!!!

Release note suggest there is an interaction
https://docs.netgate.com/pfsense/en/latest/releases/2-8-0.html

DHCP (IPv4)
Fixed: Kea does not allow FQDNs for NTP servers but input validation does not prevent them from being added #14991

The release note, and the associated diffs, indicate that the UI incorrectly allowed giving Kea a DNS name instead of an IP address. Kea, and the DHCP protocol, require an IP address. ISC apparently would take the DNS name and resolved it for you, but Kea will not.

Patch

The details are beyond me but this post suggests significant interaction occurs and will increase https://forum.netgate.com/post/1214629

dennypage

@Patch said in NTP set to time.google.com not working after reboot:

The details are beyond me but this post suggests significant interaction occurs and will increase https://forum.netgate.com/post/1214629

That’s about DHCP client registration and resolution. Not related to this NTP issue.

stephenw10

Try setting Prefer IPv4 over IPv6 in Sys > Adv > Networking.

johnpoz

The big question is here - will ntp do a A and AAAA query or not.. If you can stop it from asking for AAAA or stop what it asks for dns - in this case his pihole from answering AAAA..

Any of those options and the problem of ntp trying to talk to an address it clearly can not goes away.

This is why the whole clients asking for A and AAAA is problematic - yes in a perfect world everyone would have IPv6 and be using that, and the A stuff would only be there to support sites that do not have IPv6 yet..

My problem is with any sort of dns client even asking for AAAA (ipv6) address of something if has no viable gua IPv6 to talk to it worth - its pointless to ask for something you can not use - its added traffic, overhead and leads to weird stuff like this.

I am not exactly sure what that prefer IPv4 in ntp services actually mean - that might just mean hey only ask IPv4 dns - but it still asks for AAAA?

What I would hope it means is not ask for AAAA for ntp servers your trying to resolve. Problem solved.. If that is not the case than having your dns resolver or forwarder not answer AAAA back to the client, again problem solved.

I know dnsmasq has a filter-aaaa that should work, I know unbound has the options I showed above that will stop it. And know bind has an option of filtering AAAA as well..

So to me the best solution when you do not have a viable IPv6 address is to just stop the name server your asking from answering any AAAA query.. Now best option would to stop the client from asking - but that might not always be possible.

I will try and sniff with those options set in ntp and see if it stops AAAA query - I suspect maybe not. And just uses IPv4 to ask whatever nameserver the OS its running on is pointed too.

rpsmith

Thanks for all your help. I really appreciate it.

Also, all of my nine firewall have this same problem and only my home one uses pi-hole. The other eight also use the forwarder but are pointed to 1.1.1.2 and 9.9.9.11 and multiple ISP are involved so my gut feeling is this is a BSD or pfSense bug. BTW, I don't have any problem if I only use us.pool.ntp.org.

Roy...

johnpoz

@rpsmith I am not sure what your not getting.. It is not a bug, it is how IPv6 works.. If your device is going to try and use IPv6 for ntp - your not going to get an answer - doesn't matter what you use for dns, not what client, not what server you point to, etc.

Your not going to try and talk to something with IPv6 when what your asking for doesn't have any AAAA records - doesn't matter if you ask for them or not.. We went over this already. us.pool.ntp.org does not have AAAA..

Here are turned on dns query logging, and changed to auto in the dns protocol setting in ntp.. put in time.1.google.com

See there is asks for both A and AAAA

May 25 20:39:13 	unbound 	59639 	[59639:1] info: 127.0.0.1 time1.google.com. AAAA IN NOERROR 0.026392 0 62
May 25 20:39:13 	unbound 	59639 	[59639:1] info: 127.0.0.1 time1.google.com. AAAA IN
May 25 20:39:13 	unbound 	59639 	[59639:1] info: 127.0.0.1 time1.google.com. A IN NOERROR 0.062686 0 50
May 25 20:39:13 	unbound 	59639 	[59639:1] info: 127.0.0.1 time1.google.com. A IN

I then changed it to IPv4 only in the dns protocol setting I showed before - and nice to see it doesn't ask for AAAA just ipv4

May 25 20:41:16 	unbound 	59639 	[59639:1] info: 127.0.0.1 time2.google.com. A IN NOERROR 0.029949 0 50
May 25 20:41:16 	unbound 	59639 	[59639:1] info: 127.0.0.1 time2.google.com. A IN

if you made that setting I suggested before, and pfsense in general was set to ask itself (127.0.0.1), it wouldn't even ask for AAAA, doesn't matter if who you asks goes and ask elsewhere or resolves it just wouldn't even ask for AAAA.. I changed it to use 8.8.8.8 with IPv4 only setting in ntp.. And nope no query for AAAA

Set to auto - and yup it asks for both

Maybe there is a bug in that setting on the version of pfsense your using? I am on 24.11 - if I set ntp to use Ipv4 only, it doesn't ask for AAAA, if set to auto it asks for both..

Nice to see this setting actually works to stop query for AAAA in ntp

rpsmith

Seems to be working now after I changed DNS Resolution from Auto to IPv4. I could swear I tried that earlier today and it didn't resolve the problem but I must have missed a step. I'll try that on several of my other firewalls and see if it takes care of the problem.

Also, I don't see why that option would need to be changed to IPv4 if my WAN was already set to IPv6: None and Advance - Network Allow IPv6 box unchecked but I'm OK with making that small change if that fixes my problem.

Roy...

dennypage

@rpsmith said in NTP set to time.google.com not working after reboot:

I don't see why that option would need to be changed to IPv4 if my WAN was already set to IPv6: None and Advance - Network Allow IPv6 box unchecked

One refers to IP traffic, the other refers to DNS queries. They are discrete items/settings. I.E. just because the firewall is not forwarding IPv6 packets doesn't mean that clients are not allowed to ask "What is the IPv6 address of dns.google.com?"

rpsmith

Interesting! Thanks!

Roy...

johnpoz

@rpsmith said in NTP set to time.google.com not working after reboot:

I don't see why that option would need to be changed to IPv4 if my WAN was already set to IPv6: None and Advance

Which is what I was saying as well - why a OS or application would send out a AAAA query when it has no viable IPv6 gua or even ULA makes zero sense.. I went over why that is problematic.. This a prime example - pfsense has no IPv6 other than link-local, it makes no sense to send out AAAA queries.. The application should be smart enough to do that, you shouldn't have to toggle a setting like this.

But I am glad to see ntp has the option.

firefox is another example - my windows machine has no IPv6, not even a link local.. Yet firefox will send out queries for AAAA still. Unless I specifically tell firefox not too via the config:option setting. Just seems stupid.

rpsmith

It looks to me like dnsmasq has no awareness of setting outside of itself so it just blindly uses IPv6 regardless of other pfSense setting disabling IPv6.

Thanks again for all you help. I have been living with this problem for years now so I'm glad to have it resolved! :o)

Roy...

johnpoz

@rpsmith dnsmasq is only going to ask its upstream server for what a client asked for - if a client asks for AAAA its going to ask what you setup as a forwarder.

It is more than happy to ask for AAAA to some IPv4 address - and that will work.. Unless you tell dnsmasq not to return what it got back to the client with

--filter-AAAA
    Remove AAAA records from answers. No IPv6 addresses will be returned.

I think you're not understanding the difference of the protocol used to talk to some name server, and what a client can ask for.., You can ask for AAAA over IPv4, just like you can ask for A over IPv6.

But a resource A (ipv4 address) or AAAA (ipv6 address) of the resource record. has nothing to do with the protocol used to talk over the network.

If pfsense has zero IPv6 addresses - then its not possible for dnsmasq to use IPv6 to talk to the server you told it to forward queries too. But that doesn't mean it won't ask for a AAAA from the IPv4 server told it to ask if a client asks for it. And then hand that back to the client.

you either stop the client from asking dnsmasq for AAAA or you tell dnsmasq to not give any answers for AAAA if a client asks for it.

I agree if the OS the application (ntp in this case) is running on does not have IPv6 it is moronic to ask for AAAA it could never talk to because it has no ipv6 address. Bring that up with application, the actual OS doesn't really control what an application can ask for. But at least with ntp you can say hey don't ask for AAAA even if you have a IPv6 address.

rpsmith

@stephenw10 ~ That looks like a good setting for all my IPv4 firewalls! Thanks!

Roy...

johnpoz

@rpsmith that doesn't stop a application like ntp from asking for a AAAA..

At least you have multiple ways to circumvent the problem - but there are many things you have no control over.. Plex for example - asks for A and AAAA even though there is no IPv6 on the box it is running on, and there is no way to get it to stop doing that.

As you can see the host plex is running on - my nas, has NO ipv6, not even link-local

ovs_eth0  Link encap:Ethernet  HWaddr 00:11:32:7B:29:7D  
          inet addr:192.168.9.10  Bcast:192.168.9.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:16998457 errors:0 dropped:0 overruns:0 frame:0
          TX packets:32242358 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1 
          RX bytes:2962532801 (2.7 GiB)  TX bytes:131074968353 (122.0 GiB)

Yet it continues to ask for AAAA

I have many iot such devices that insist on asking for AAAA, again they have no IPv6 address, again no way stop it.

My alexas always ask for both A and AAAA - I agree its stupid.. But it is what it is

I can stop my Nameserver they are asking from returning an answer if there is one - but I can not stop the client from asking.