NTP Issue

johnpoz

@ARAMP1 one thing I see as different is why are you pointing to multiple pools that are really the same? You run your own ntp server, while sure it should point to some others vs just its gps time.. pfsense really has litte need other than just pointing to your one ntp server that you are running.

Why can it not talk to any of those other ntp servers either.. All of those other ones should be available via wan, that it is seems it talking to the 128.138.141.172 box.

You cut off your config - are you trying to do auth?

If you want to add more later - sure but for troubleshooting I would just point to your 1 local server.. And then sniff do you see an answer?

Can we see your other devices that are syncing off this local server? For example I already showed my windows box syncing - here is one of my switches using it

sg300-28#sho sntp status

Clock is synchronized, stratum 1, reference is 192.168.3.32, unicast

Unicast servers:

Server            : 192.168.3.32
  Source          : Static
  Stratum         : 1
  Status          : up
  Last Response   : 17:03:25.0 web Jun 2 2024
  Offset          : 347.3516068 mSec
  Delay           : -1000 mSec

here is my nas, and printer all pointing to my local ntp - showing working

For troubleshooting - I would set the only interface on to be the one interface where your ntp server is. And I would sniff the traffic to validate you see it sending to it, and getting a response on its own accord, not with the ntpdate command, etc.

Here is my unifi controller setting, and showing that one of the AP is sync'd etc

edit: oh my nas is using old name, which still resolves - but should change that to home.arpa vs local.lan

What are you running for your ntp server? ntp, ntpsec, chrony?

edit2: that you are only able to talk to 1 of the many many ntp servers your pointing to is very odd to me.. If there was something wrong with pfsense, why is talking to anything. If there was some disconnect between pfsense and your ntp server, why is it not talking to all the other servers your pointing to?

Clearly something is amiss - figuring out what it is the puzzle needed to be solved.

ARAMP1

@johnpoz said in NTP Issue:

@ARAMP1 one thing I see as different is why are you pointing to multiple pools that are really the same? You run your own ntp server, while sure it should point to some others vs just its gps time.. pfsense really has litte need other than just pointing to your one ntp server that you are running.

Why can it not talk to any of those other ntp servers either.. All of those other ones should be available via wan, that it is seems it talking to the 128.138.141.172 box.

You cut off your config - are you trying to do auth?

If you want to add more later - sure but for troubleshooting I would just point to your 1 local server.. And then sniff do you see an answer?

Can we see your other devices that are syncing off this local server? For example I already showed my windows box syncing - here is one of my switches using it
sg300-28#sho sntp status

Clock is synchronized, stratum 1, reference is 192.168.3.32, unicast

Unicast servers:

Server            : 192.168.3.32
  Source          : Static
  Stratum         : 1
  Status          : up
  Last Response   : 17:03:25.0 web Jun 2 2024
  Offset          : 347.3516068 mSec
  Delay           : -1000 mSec
here is my nas, and printer all pointing to my local ntp - showing working

For troubleshooting - I would set the only interface on to be the one interface where your ntp server is. And I would sniff the traffic to validate you see it sending to it, and getting a response on its own accord, not with the ntpdate command, etc.

Here is my unifi controller setting, and showing that one of the AP is sync'd etc

edit: oh my nas is using old name, which still resolves - but should change that to home.arpa vs local.lan

What are you running for your ntp server? ntp, ntpsec, chrony?

edit2: that you are only able to talk to 1 of the many many ntp servers your pointing to is very odd to me.. If there was something wrong with pfsense, why is talking to anything. If there was some disconnect between pfsense and your ntp server, why is it not talking to all the other servers your pointing to?

Clearly something is amiss - figuring out what it is the puzzle needed to be solved.

Appreciate the reply.

I'm not doing NTPv3 authentication.

I actually originally had it only pointing to my local server, but when it started acting up, I started adding. I assumed that since I checked "prefer" it would default to that one if it started working.

Here's what I get when I delete all the others.

NTP 5.jpg

I can point anything on the network to my NTP and it seems to work. Here's the windows 10 machine I'm on now.

Sync with NTP.jpg

Show NTP.jpg

Here's a TrueNAS server that I just pointed it to.

NTP Truenas.jpg

Really, I only need it for my Blue Iris machine because I want all of my security cameras synced. Other than that, it's just nice to have it working.

Forgive me, but I'm not sure how do do the ole sniff.

johnpoz

@ARAMP1 the sniff would be just packet capture under the diagnostic menu.. You can have it only capture the ntp from your IP of your ntp server and port 123.

Select the interface used to talk to your ntp server, assuming that is your lan interface.

You can then download that capture and we can look at it with say wireshark.

The info from your nas is good.. what IP is your nas on, I assume its on this same 192.168.10 network.

edit: don't camera's sync their time via ipc?

I don't have ntp setup on my cameras - they sync with the nvr via ipc.. But blue iris could be different for sure. The nvr syncs directly with my ntp server.. Why wouldn't you sync your blue iris and your cameras directly with your ntp server? vs pfsense?

That being said - pfsense for sure should be able to sync with your ntp server.. But if you can just sync your camera stuff with your ntp server, then the pfsense time sync isn't as critical, etc. And can figure out what is going on it with it without having to worry about your camera time being off, etc.

ARAMP1

@johnpoz said in NTP Issue:

@ARAMP1 the sniff would be just packet capture under the diagnostic menu.. You can have it only capture the ntp from your IP of your ntp server and port 123.

Select the interface used to talk to your ntp server, assuming that is your lan interface.

You can then download that capture and we can look at it with say wireshark.

The info from your nas is good.. what IP is your nas on, I assume its on this same 192.168.10 network.

My windows machine is on the 10.10.50 network and that truenas machine is on the 192.168.60 network but I have rules on each of those VLANs to access 192.168.10.120 for NTP (the 192.168.10 network is my management network with all my switches, access points, etc).

For cameras, here's the rules I have in place.

Cameras Port 123.jpg

johnpoz

@ARAMP1 well that is interesting.. So the devices on different networks sync fine - but the pfsense on the same network is not able to reach it??

Your not doing anything with ethernet filtering, nor any floating rules for outbound filtering?

As a test - your truenas can act as ntp server I would assume.. Can you point pfsense to it?

Example - just pointed pfsense to my nas.

notice it shows the 9.10 address of my nas, and it shows its ref as my pi ntp server the 3.32 IP

ARAMP1

Here's a short run of the packet capture...

00:56:11.413566 IP 192.168.60.251.123 > 192.168.10.120.123: UDP, length 48
00:56:11.413612 IP 192.168.60.251.123 > 192.168.10.120.123: UDP, length 48
00:56:11.414620 IP 192.168.10.120.123 > 192.168.60.251.123: UDP, length 48
00:56:11.414652 IP 192.168.10.120.123 > 192.168.60.251.123: UDP, length 48
00:57:17.413522 IP 192.168.60.251.123 > 192.168.10.120.123: UDP, length 48
00:57:17.413531 IP 192.168.60.251.123 > 192.168.10.120.123: UDP, length 48
00:57:17.414624 IP 192.168.10.120.123 > 192.168.60.251.123: UDP, length 48
00:57:17.414659 IP 192.168.10.120.123 > 192.168.60.251.123: UDP, length 48
00:58:23.373022 IP 192.168.60.251.123 > 192.168.10.120.123: UDP, length 48
00:58:23.373061 IP 192.168.60.251.123 > 192.168.10.120.123: UDP, length 48
00:58:23.373684 IP 192.168.10.120.123 > 192.168.60.251.123: UDP, length 48
00:58:23.373718 IP 192.168.10.120.123 > 192.168.60.251.123: UDP, length 48
00:58:42.477178 IP 10.10.50.230.44863 > 192.168.10.120.123: UDP, length 48
00:58:42.477281 IP 10.10.50.230.44863 > 192.168.10.120.123: UDP, length 48
00:58:42.477576 IP 192.168.10.120.123 > 10.10.50.230.44863: UDP, length 48
00:58:42.477597 IP 192.168.10.120.123 > 10.10.50.230.44863: UDP, length 48
00:58:43.586214 IP 10.10.50.230.50913 > 192.168.10.120.123: UDP, length 48
00:58:43.586317 IP 10.10.50.230.50913 > 192.168.10.120.123: UDP, length 48
00:58:43.586555 IP 192.168.10.120.123 > 10.10.50.230.50913: UDP, length 48
00:58:43.586560 IP 192.168.10.120.123 > 10.10.50.230.50913: UDP, length 48
00:58:45.604892 IP 10.10.50.230.41040 > 192.168.10.120.123: UDP, length 48
00:58:45.605003 IP 10.10.50.230.41040 > 192.168.10.120.123: UDP, length 48
00:58:45.605557 IP 192.168.10.120.123 > 10.10.50.230.41040: UDP, length 48
00:58:45.605580 IP 192.168.10.120.123 > 10.10.50.230.41040: UDP, length 48
00:58:47.636203 IP 10.10.50.230.54265 > 192.168.10.120.123: UDP, length 48
00:58:47.636249 IP 10.10.50.230.54265 > 192.168.10.120.123: UDP, length 48
00:58:47.636552 IP 192.168.10.120.123 > 10.10.50.230.54265: UDP, length 48
00:58:47.636578 IP 192.168.10.120.123 > 10.10.50.230.54265: UDP, length 48
00:58:49.778857 IP 10.10.50.230.56448 > 192.168.10.120.123: UDP, length 48
00:58:49.778967 IP 10.10.50.230.56448 > 192.168.10.120.123: UDP, length 48
00:58:49.779532 IP 192.168.10.120.123 > 10.10.50.230.56448: UDP, length 48
00:58:49.779552 IP 192.168.10.120.123 > 10.10.50.230.56448: UDP, length 48
00:58:51.786147 IP 10.10.50.230.36563 > 192.168.10.120.123: UDP, length 48
00:58:51.786273 IP 10.10.50.230.36563 > 192.168.10.120.123: UDP, length 48
00:58:51.786525 IP 192.168.10.120.123 > 10.10.50.230.36563: UDP, length 48
00:58:51.786529 IP 192.168.10.120.123 > 10.10.50.230.36563: UDP, length 48
00:58:53.816726 IP 10.10.50.230.58706 > 192.168.10.120.123: UDP, length 48
00:58:53.816798 IP 10.10.50.230.58706 > 192.168.10.120.123: UDP, length 48
00:58:53.817676 IP 192.168.10.120.123 > 10.10.50.230.58706: UDP, length 48
00:58:53.817681 IP 192.168.10.120.123 > 10.10.50.230.58706: UDP, length 48
00:58:54.780841 IP 10.10.50.230.56213 > 192.168.10.120.123: UDP, length 48
00:58:54.780910 IP 10.10.50.230.56213 > 192.168.10.120.123: UDP, length 48
00:58:54.781525 IP 192.168.10.120.123 > 10.10.50.230.56213: UDP, length 48
00:58:54.781533 IP 192.168.10.120.123 > 10.10.50.230.56213: UDP, length 48
00:58:56.791962 IP 10.10.50.230.46158 > 192.168.10.120.123: UDP, length 48
00:58:56.792080 IP 10.10.50.230.46158 > 192.168.10.120.123: UDP, length 48
00:58:56.792530 IP 192.168.10.120.123 > 10.10.50.230.46158: UDP, length 48
00:58:56.792551 IP 192.168.10.120.123 > 10.10.50.230.46158: UDP, length 48
00:58:58.467266 IP 10.10.50.230.54383 > 192.168.10.120.123: UDP, length 48
00:58:58.467305 IP 10.10.50.230.54383 > 192.168.10.120.123: UDP, length 48
00:58:58.467569 IP 192.168.10.120.123 > 10.10.50.230.54383: UDP, length 48
00:58:58.467574 IP 192.168.10.120.123 > 10.10.50.230.54383: UDP, length 48
00:59:00.508052 IP 10.10.50.230.44608 > 192.168.10.120.123: UDP, length 48
00:59:00.508113 IP 10.10.50.230.44608 > 192.168.10.120.123: UDP, length 48
00:59:00.508564 IP 192.168.10.120.123 > 10.10.50.230.44608: UDP, length 48
00:59:00.508568 IP 192.168.10.120.123 > 10.10.50.230.44608: UDP, length 48
00:59:02.576259 IP 10.10.50.230.54479 > 192.168.10.120.123: UDP, length 48
00:59:02.576371 IP 10.10.50.230.54479 > 192.168.10.120.123: UDP, length 48
00:59:02.576552 IP 192.168.10.120.123 > 10.10.50.230.54479: UDP, length 48
00:59:02.576558 IP 192.168.10.120.123 > 10.10.50.230.54479: UDP, length 48
00:59:04.587321 IP 10.10.50.230.34257 > 192.168.10.120.123: UDP, length 48
00:59:04.587386 IP 10.10.50.230.34257 > 192.168.10.120.123: UDP, length 48
00:59:04.587550 IP 192.168.10.120.123 > 10.10.50.230.34257: UDP, length 48
00:59:04.587556 IP 192.168.10.120.123 > 10.10.50.230.34257: UDP, length 48
00:59:06.612532 IP 10.10.50.230.37136 > 192.168.10.120.123: UDP, length 48
00:59:06.612592 IP 10.10.50.230.37136 > 192.168.10.120.123: UDP, length 48
00:59:06.613552 IP 192.168.10.120.123 > 10.10.50.230.37136: UDP, length 48
00:59:06.613578 IP 192.168.10.120.123 > 10.10.50.230.37136: UDP, length 48
00:59:08.641801 IP 10.10.50.230.33174 > 192.168.10.120.123: UDP, length 48
00:59:08.641915 IP 10.10.50.230.33174 > 192.168.10.120.123: UDP, length 48
00:59:08.642548 IP 192.168.10.120.123 > 10.10.50.230.33174: UDP, length 48
00:59:08.642570 IP 192.168.10.120.123 > 10.10.50.230.33174: UDP, length 48
00:59:29.381935 IP 192.168.60.251.123 > 192.168.10.120.123: UDP, length 48
00:59:29.381971 IP 192.168.60.251.123 > 192.168.10.120.123: UDP, length 48
00:59:29.382608 IP 192.168.10.120.123 > 192.168.60.251.123: UDP, length 48
00:59:29.382641 IP 192.168.10.120.123 > 192.168.60.251.123: UDP, length 48
01:00:13.721904 IP 10.10.50.230.48115 > 192.168.10.120.123: UDP, length 48
01:00:13.722019 IP 10.10.50.230.48115 > 192.168.10.120.123: UDP, length 48
01:00:13.722520 IP 192.168.10.120.123 > 10.10.50.230.48115: UDP, length 48
01:00:13.722524 IP 192.168.10.120.123 > 10.10.50.230.48115: UDP, length 48
01:00:38.413534 IP 192.168.60.251.123 > 192.168.10.120.123: UDP, length 48
01:00:38.413568 IP 192.168.60.251.123 > 192.168.10.120.123: UDP, length 48
01:00:38.414569 IP 192.168.10.120.123 > 192.168.60.251.123: UDP, length 48
01:00:38.414598 IP 192.168.10.120.123 > 192.168.60.251.123: UDP, length 48
01:00:52.473322 IP 192.168.10.1.123 > 192.168.10.120.123: UDP, length 48
01:00:52.473549 IP 192.168.10.120.123 > 192.168.10.1.123: UDP, length 48
01:00:55.450351 IP 192.168.10.1.123 > 192.168.10.120.123: UDP, length 48
01:00:55.450552 IP 192.168.10.120.123 > 192.168.10.1.123: UDP, length 48
01:01:00.525365 IP 192.168.10.1.123 > 192.168.10.120.123: UDP, length 48
01:01:00.525543 IP 192.168.10.120.123 > 192.168.10.1.123: UDP, length 48
01:01:18.317945 IP 10.10.50.230.38268 > 192.168.10.120.123: UDP, length 48
01:01:18.318057 IP 10.10.50.230.38268 > 192.168.10.120.123: UDP, length 48
01:01:18.318613 IP 192.168.10.120.123 > 10.10.50.230.38268: UDP, length 48
01:01:18.318635 IP 192.168.10.120.123 > 10.10.50.230.38268: UDP, length 48
01:01:47.398970 IP 192.168.60.251.123 > 192.168.10.120.123: UDP, length 48
01:01:47.399009 IP 192.168.60.251.123 > 192.168.10.120.123: UDP, length 48
01:01:47.399617 IP 192.168.10.120.123 > 192.168.60.251.123: UDP, length 48
01:01:47.399650 IP 192.168.10.120.123 > 192.168.60.251.123: UDP, length 48
01:02:05.525345 IP 192.168.10.1.123 > 192.168.10.120.123: UDP, length 48
01:02:05.525544 IP 192.168.10.120.123 > 192.168.10.1.123: UDP, length 48
01:02:22.690101 IP 10.10.50.230.44520 > 192.168.10.120.123: UDP, length 48
01:02:22.690213 IP 10.10.50.230.44520 > 192.168.10.120.123: UDP, length 48
01:02:22.690525 IP 192.168.10.120.123 > 10.10.50.230.44520: UDP, length 48
01:02:22.690546 IP 192.168.10.120.123 > 10.10.50.230.44520: UDP, length 48
01:02:54.378938 IP 192.168.60.251.123 > 192.168.10.120.123: UDP, length 48
01:02:54.378974 IP 192.168.60.251.123 > 192.168.10.120.123: UDP, length 48
01:02:54.379592 IP 192.168.10.120.123 > 192.168.60.251.123: UDP, length 48
01:02:54.379623 IP 192.168.10.120.123 > 192.168.60.251.123: UDP, length 48
01:03:09.525230 IP 192.168.10.1.123 > 192.168.10.120.123: UDP, length 48
01:03:09.525545 IP 192.168.10.120.123 > 192.168.10.1.123: UDP, length 48
01:03:27.350112 IP 10.10.50.230.52709 > 192.168.10.120.123: UDP, length 48
01:03:27.350221 IP 10.10.50.230.52709 > 192.168.10.120.123: UDP, length 48
01:03:27.350617 IP 192.168.10.120.123 > 10.10.50.230.52709: UDP, length 48
01:03:27.350637 IP 192.168.10.120.123 > 10.10.50.230.52709: UDP, length 48

johnpoz

@ARAMP1 said in NTP Issue:

01:02:05.525345 IP 192.168.10.1.123 > 192.168.10.120.123: UDP, length 48
01:02:05.525544 IP 192.168.10.120.123 > 192.168.10.1.123: UDP, length 48

I take it that is your pfsense IP at 10.1 - can you download that and attach the pcap.. For example - here is pcap I did earlier

packetcapture-igb3-20240602095409.pcap

ARAMP1

@johnpoz said in NTP Issue:

@ARAMP1 well that is interesting.. So the devices on different networks sync fine - but the pfsense on the same network is not able to reach it??

Your not doing anything with ethernet filtering, nor any floating rules for outbound filtering?

As a test - your truenas can act as ntp server I would assume.. Can you point pfsense to it?

Example - just pointed pfsense to my nas.

notice it shows the 9.10 address of my nas, and it shows its ref as my pi ntp server the 3.32 IP

Just tried pointing it to a couple truenas servers I have on different VLANs. Both are Unreach/Pending.

The floating rules I have in place only have to do with CoDel traffic limiter and pfBlocker.

ARAMP1

packetcapture-ix1-20240602201708.pcap

johnpoz

@ARAMP1 this seems a bit odd

ARAMP1

@johnpoz I'm looking at wireshark now, but I don't know what I'm looking at.

ARAMP1

@johnpoz said in NTP Issue:

@ARAMP1 this seems a bit odd

Something with the DNS server? Here's my setup.

DNS server.jpg

johnpoz

@ARAMP1 no it wouldn't have anything to do with dns, because your pointing to an IP for the server.

I would think that ref ID would point to the IP your asking.. see in mine it shows my 192.168.3.32?

It might be like that because it has never gotten an answer? So it has no ref as of yet?

From your sniff you get an answer, so why would ntp think it can not reach that server?

johnpoz

@ARAMP1 can you turn off ntp and validate nothing is listening on 123..

example

ARAMP1

packetcapture-ix1-20240602204723.pcap

I've still pointed clients at 192.168.10.120 directly, so that would still show something listening on port 123, right?

johnpoz

@ARAMP1 not clicked to not enable it

As you see when I disable it nothing is listening on 123

Also can you one you enable it again, post up your ntpd.conf

[23.09.1-RELEASE][admin@sg4860.home.arpa]/: cat /var/etc/ntpd.conf 
# 
# pfSense ntp configuration file 
# 

tinker panic 0 

# Orphan mode stratum and Maximum candidate NTP peers
tos orphan 12 maxclock 5


# Upstream Servers
server -4 192.168.3.32 iburst minpoll 6 maxpoll 10 prefer


enable stats
statistics clockstats loopstats peerstats
statsdir /var/log/ntp
logconfig =syncall +clockall +peerall +sysall
driftfile /var/db/ntpd.drift
restrict default kod limited nomodify nopeer notrap
restrict -6 default kod limited nomodify nopeer notrap
interface ignore all
interface ignore wildcard
interface listen igb2.1011
interface listen igb3
interface listen igb0
interface listen igb4
interface listen igb2
interface listen igb2.110
interface listen igb2.6
interface listen igb2.4
interface listen igb5
interface listen lo0
[23.09.1-RELEASE][admin@sg4860.home.arpa]/:

ARAMP1

@johnpoz said in NTP Issue:

cat /var/etc/ntpd.conf

cmd prompt.jpg

johnpoz

@ARAMP1 ok the only thing that looks odd is your restrict 192.168.10.120 that is not really a network that is a host address. The network with /24 would be 192.168.10.0

After seeing yours noticed mine wasn't even listed... I removed mine completely because to be honest that is for clients wanting to talk to pfsense as their server. So you shouldn't even need that.

I would remove it, but that would be really crazy if that was the issue.

Nothing is there that makes any sense to why it wouldn't be working that I am seeing..

So you have no native network on the ix1 interface.. Only vlans.. I take it ix1.10 is your 192.168.10 interface..

ARAMP1

@johnpoz I'm not following when you say "I would remove it". What would/did you remove?

I have a native network on the ix1 interface but it's not enabled.

INterfaces 3.jpg

Gertjan

@ARAMP1 said in NTP Issue:

I'm not following when you say "I would remove it". What would/did you remove?

Here :