NTP server issues

bwanajag

I’ve configured the NTP server in pfsense, but my Netgear switch times out when trying to poll the NTP server. I don’t know how to resolve this issue. I get this is probably a Netgear config issue, but I am hoping someone will be able to assist. Within the Netgear SNTP setup, one has to select either Unicast or Broadcast. It defaults to Unicast, but since my NTP server is on the pfsense box, should I select Broadcast - or does it matter? I’ve selected IPv4 as the server type (as opposed to DNS), input my pfsense IP address (192.168.0.1) as the NTP server address, and selected Version 4 for NTP. The message the switch gives is that “Request Timed Out” and Address type “unknown”. Any suggestions?

stephenw10

Are other hosts able to sync against the pfSense ntp service?

Run a packet capture on the internal interface filtered by the switch IP and port 123. Do you see requests? Do you see pfSense respond?

Steve

gwaitsi

I've got a netgear GS110TPP and GS108v3 and neither of them are able to sync with pfsense. dd-wrt has no problems at all. Originally, i NAT'd 123 to the pfsense box and my PC's (windows/linux) and dd-wrt devices had no problems obtaining time (even with external servers in their configs).

The netgear devices refused to sync either time out or bad data.

To be sure, i allowed 123 through and configured the pfsense as the 1st preferred NTP server, and used the netgear services or various country .ntp.org services. The netgear devices were able to sync with their own servers of course, but for the ntp.org i found that some pools, or individual country services work intermittently but not consistently as the netgear servers. I am surprised no one else has found this issue, because mine (with latest firmware) appears same as described above

stephenw10

After they sync'd with some other server are they able to sync with pfSense? I could imagine the time being so far out they refused to sync at some point.

Steve

gwaitsi

@stephenw10 no. 100% failure rate and either gives "bad data encoded" or "request timeout"

johnpoz

@gwaitsi said in NTP server issues:

I am surprised no one else has found this issue,

Found what issue - that you have some issue in your network or your setup.

he message the switch gives is that “Request Timed Out” and Address type “unknown”. Any suggestions?

Show us what you have done, lets see the ntp request and answer via a sniff, etc..

Show us the status of ntp on pfsense, let see your rules allowing said traffic.. Lets see your ntp setup on your switch..

i NAT'd 123 to the pfsense box and my PC's

That makes no sense at all..

gwaitsi

@johnpoz I originally NAT'd 123 to 127.0.0.1 to force all NTP requests to pfsense. everything (win10, linux, dd-wrt) worked except the near gear. I turned this off and allow 123 to pass now. Everything still works, only this time netgear also works.

johnpoz

So if you forwarded all traffic to loopback on 123.. and your other clients worked - but only this switch didn't, that points to something on in the switch.. Not pfsense.. If it was pfsense - how would any clients work?

I have never seen a device hard code an IP for ntp... They always use either a vendor fqdn, pool, or other tqdn. So if you want your local clients to use your local ntp, and you can not set them, or they don't accept dhcp ntp.. Then see what they query for fqdn for the ntp and just create a host override..

Example: I have a couple of smart outlets, you know you can turn off power via app, etc. or have alexa turn lights on and off with them... They like to use uk.ntp.pool.org - which is just moronic to be honest.. I am not in the uk... I am in the US.. These are US devices.. why would they be set to use uk pool.. They should be using a vendor ntp pool fqdn, but anyway... So I just setup a host override to resolve that to my local ntp server.

C:\WINDOWS\system32>dig uk.pool.ntp.org

; <<>> DiG 9.16.1 <<>> uk.pool.ntp.org
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 14704
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;uk.pool.ntp.org.               IN      A

;; ANSWER SECTION:
uk.pool.ntp.org.        2665    IN      A       192.168.3.32

;; Query time: 1 msec
;; SERVER: 192.168.3.10#53(192.168.3.10)
;; WHEN: Sun Apr 12 05:17:17 Central Daylight Time 2020
;; MSG SIZE  rcvd: 60


C:\WINDOWS\system32>w32tm /stripchart /computer:uk.pool.ntp.org
Tracking uk.pool.ntp.org [192.168.3.32:123].
The current time is 4/12/2020 5:17:33 AM.
05:17:33, d:+00.0006882s o:-00.0005237s  [                           *                           ]
05:17:35, d:+00.0008071s o:-00.0005022s  [                           *                           ]
05:17:37, d:+00.0008540s o:-00.0005418s  [                           *                           ]
05:17:39, d:+00.0008378s o:-00.0005332s  [                           *                           ]
05:17:41, d:+00.0008126s o:-00.0005193s  [                           *                           ]
05:17:43, d:+00.0008444s o:-00.0005247s  [                           *                           ]
^C
C:\WINDOWS\system32>

You can see that resolves to my ntp server, and if query time via that name - get back time from local server..

gwaitsi

@johnpoz forget setting to local ntp, i have already said i am passing through 123. it is the only way to get the netgear devices to pickup time. I know the issue is to do with the netgear devices, hence the first guys post.

What i don't know is, is it something specific to pfsense or to debian in general. But i think netgear are pretty widely used, so i can well imagine someone is most likely using them with pfsense and may have solved the issue of picking up the time from the local pfsense box

p.s. what you describe is what i was doing before i got these netgear switches. i.e. i simply pointed all 123 request to the pfsense box. this has worked fine for a long time, but i had to stop doing that, because i can't get the netgear devices to latch the time from the pfsense ntp daemon

stephenw10

There is a difference between having a host resolve as a local IP and redirecting to the same local IP though.

Perhaps the switches are using some ntp authentication and will only sync to Netgear's pool? It would be the first time I've seen that though.

Run a packet capture and find out. Whatever they are using it's clearly on port 123 since redirecting that breaks it. Should be easy enough to capture.

Steve

johnpoz

pfsense is a standard ntp service - running the latest version 4.2.8p14.. Anything designed to use ntp would work...

There is nothing fancy about gs108v3 - pretty sure I have one of those on the shelf even..

Yup, sure do... Give me a sec to plug it in and sent its ntp up...

Ok - the GS108Ev3 doesn't even have time settings..

JKnott

@johnpoz said in NTP server issues:

I have never seen a device hard code an IP for ntp... They always use either a vendor fqdn, pool, or other tqdn.

My tablet does that. I have an Asus Zenpad 10 which uses 5.144.132.2.123 for ntp. The host command does not return a host name.

host 5.144.132.2.123
Host 5.144.132.2.123 not found: 3(NXDOMAIN)

There is no ntp setting on this tablet that I can find.

johnpoz

5-144-132-2.static.hostiran.name

That is IP in Iran...

Why would they hard code an IP in iran? That makes ZERO sense!!

its part of the pool for global, asia and iran..
https://www.ntppool.org/scores/5.144.132.2

So what is likely is you missed the query where it found that IP via dns via a query to ntp pool..

You would have to be a complete an utter moron to hard code an IP for some mass produced device to use.. When if that IP goes away, what if can not run ntp on it. What if you sell 10 million devices that all now point to a single IP, etc. etc..

They query a pool fqdn, and pick and IP.. or multiples from this pool..

But its clear why his switch gs108Ev3 can not sync time - is because it has no settings for that at all.. It has no log that I can see.. so what point of time would it have?

JKnott

@johnpoz said in NTP server issues:

So what is likely is you missed the query where it found that IP via dns via a query to ntp pool..

Actually, I did watch for DNS when I first noticed that and didn't see any. Also, if it queried pool.ntp.org, it should have got an address on my LAN, as I have a host override to point to my pfSense firewall.

It's not the first time I've seen a manufacturer do something stupid. Seems to me there's a switch & AP manufacturer that couldn't do VLANs right.

gwaitsi

@johnpoz I have the pro version, you have the plus. You don't seem to have many of the features that i have.
Time is under System, Management, Time. But like i say, you are missing a whole bunch of stuff like switching, routing, qos, etc

johnpoz

@JKnott said in NTP server issues:

it should have got an address on my LAN

How do you know it didn't query asia.pool.ntp.org or 0.pool.ntp.org, 1.pool.ntp.org, do you have a wildcard setup?

Again any sntp client could query the ntp service that pfsense is running... It is the the ntp server that owns the internet to be honest.. Sniff the traffic.. What do you see.

gwaitsi

@JKnott i removed all timeservers except for the pfsense box and i am getting TCP 66 to 52.211.208.165

JKnott

@johnpoz

While you were posting that, I checked again and it is indeed querying asia.pool.ntp.org. I don't know how I missed that before.

This is one example of where using Wireshark is better than Packet Capture. With Wireshark I can filter on both DNS and NTP at the same time, along with the MAC address, whereas Packet Capture can only filter on 1 protocol at a time, along with MAC or IP address.

Still, given that the tablets are used around the world, it would be better if they just used pool.ntp.org, instead of asia.pool.ntp.org. Using pool.ntp.org will provide a regional address.

Guess it's time for another host override.

johnpoz

yup same sort of nonsense with my smart plugs using uk.pool ;) hehehe

you can download the pcap.. Your pfsense box sure and the F is not 52.211.x.x is it??

That is a aws IP, and is not in the pool.

And I don't show it answering ntp.. And TCP is not what sntp or ntp uses.. anyway. Are you trying to say port tcp 66 to that IP? That sure and the hell has nothing to do with time. Or is the packet size... sniff on port 123 UDP.. And then open it in wireshark - attach a copy..

gwaitsi

@johnpoz how can i export just the 4 ntp lines from wireshark
ntp client
ntp server
ntp client
ntp server

johnpoz

Attache the pcap.. If you sniffed on 123 and the IP of your switch all that would be in there is its query for ntp and answers.

JKnott

@johnpoz

I often filter on the MAC, to capture both IPv4 and IPv6.

johnpoz

Thats fine - is is switch running IPv6.. He made no mention of that.. Here is query from my pc to my ntp server.. That captured with pfsense

Since its routed from my lan to my segment my pi ntp server sites in.

ntp.pcap

You can download that and open in wireshark as example of ntp query and answer.

gwaitsi

@johnpoz ntp_dump.pcap i notice in the two server entries, the first one is giving the address 127.0.0.1 and the second says unidentified reference source 'rate'

johnpoz

That shows that your ntp on pfsense is not in sync... Show us the status of ntp on pfsense.

Here is what typical should look like

Says right there that your clock is unsynced

So no - no clients at all should sync with that..

gwaitsi

@johnpoz

johnpoz

So pfsense can not talk to any ntp servers, so no shit nobody is going to be able to sync time with it.

your reach needs to be 377... And looks like you hae those set as pool placeholders... But they prob never resolved.

example

gwaitsi

@johnpoz i had the interface set to all the LAN/VLAN addresses. Now i changed it to the WAN/VPN and chugging.

johnpoz

Ok once you see it in sync.. Then your switch should sync with it... But when it is unsync like that, no ntp client would sync.

You changed it to what? Your ntp services doesn't need to listen on your wan/vpn... You mean your dns was not using your wan/vpn to resolve with?

gwaitsi

@johnpoz I originally had the Listen Interface selecting only the LAN/VLANS, but not the WAN/VPNs, now i reversed it.

should i listen on both wan and vpn, or just wan?
is europe pool ntp org enough, or should i add individual country pools?
How do i know when it is in sync

stephenw10

Wait until you have entries showing reach 377 and an active peer. Check the logs.
But that looks good now.

gwaitsi

@gwaitsi hmm... interesting.

if i select no interface items, it works.
if i select all the vlan/lan interfaces only, it does not work
if i select the wan/vpn interfaces only , it works
if i select the vlan interfaces only, works
if i select the lan interfaces (3) only, it does not work
if i select any one of the lan interfaces, it does not work.

So the issue seems to be with having ntp listen on a lan interface
*** So i confirmed it, if i enter the pfsense vlan address as the time server, netgear successfully updates ***

so, what can be breaking the NTP if i select the lan interfaces to listen on

gwaitsi

@johnpoz @stephenw10 You can see what happens igb1.50 (vlan) is working.
Then i add a LAN interface to listen on and you see it falls over. why is the lan interface adding 0.0.0.0 instead of the lan address? this is a bug right?

gwaitsi

@stephenw10 where can i find the ntp configuration files to see what is being written

johnpoz

You only need to listen on your interfaces you want to serve time on... My guess since you mention vpn, is you went to manual and now your not natting loopback, so when an application tries to use that as source to talk pfsense doesn't nat it and you can not talk..

So the only why ntp would be able to talk to ntp servers is if bound to interface, the 0.0.0.0 is bind to all... Post up your outbound nats, Bet you a beer you don't have loopback natting..

stephenw10

/var/etc/ntpd.conf

johnpoz

See I am only listening on a few interfaces...

[2.4.5-RELEASE][admin@sg4860.local.lan]/root: cat /var/etc/ntpd.conf 
# 
# pfSense ntp configuration file 
# 

tinker panic 0 
# Orphan mode stratum
tos orphan 12


# Upstream Servers
server 192.168.3.32 iburst maxpoll 9 prefer
pool us.pool.ntp.org iburst maxpoll 9


enable stats
statistics clockstats loopstats peerstats
statsdir /var/log/ntp
logconfig =syncall +clockall +peerall +sysall
driftfile /var/db/ntpd.drift
restrict default kod limited nomodify nopeer notrap
restrict -6 default kod limited nomodify nopeer notrap
restrict source kod limited nomodify notrap
interface ignore all
interface ignore wildcard
interface listen igb3
interface listen igb0
interface listen igb2
interface listen igb2.4
interface listen igb5
[2.4.5-RELEASE][admin@sg4860.local.lan]/root:

gwaitsi

@johnpoz I want to listen on the lan interfaces for switches/access point devices and vlans for end user equip.

I have the same config for each of the VPNs and they work

johnpoz

Exactly I only listen on a few of my local lan side interfaces... I don't see ::1/128 listed there.. maybe its trying to do ipv6?

I really don't see the point of doing manual outbound nat.. Why would you not just use automatic, and set hybrid for the stuff you want to policy route out your vpn..

Do you not have ntp listening on an interface that you nat?

gwaitsi

@johnpoz If i use a single LAN interface, it looks like this

# 
# pfSense ntp configuration file 
# 

tinker panic 0 
# Orphan mode stratum
tos orphan 12


# Upstream Servers
pool nl.pool.ntp.org iburst maxpoll 9
pool de.pool.ntp.org iburst maxpoll 9


enable stats
statistics clockstats loopstats peerstats
statsdir /var/log/ntp
logconfig =syncall +clockall +peerall +sysall
driftfile /var/db/ntpd.drift
restrict default kod limited nomodify nopeer notrap
restrict -6 default kod limited nomodify nopeer notrap
restrict source kod limited nomodify notrap
interface ignore all
interface ignore wildcard
interface listen igb1