NTP widget time wrong?

[Ramosel]

A few weeks ago I embarked on making a cheap Pulse Per Second (PPS) disciplined Stratum 1 NTP Time server with a GPS hat (for Raspberry Pi) from Uputronics (UK) that has RTC onboard. I've found 3 builds I like. The first was from Jacob Deane (UK) on his own website, the second was from Kreeblah (Seattle) on Github and the third was from Phil Randal (UK). I've got all of them working to some extent but updates in OS as well as software have strained the build process so the cookbooks need some work and deciphering. I really like the Phil Randal version as he negates the height, motion and location aspects of GPS and concentrates on time via NEMA.

My issue is that no matter which build I use, the time shown on the pfSense dashboard NTP widget always lags behind and noticeably so. If I restart NTP or go away from the pfSense dashboard and back, it always resyncs... then starts to lag again. . I also set a couple of workstations up to go to the NTP server. They are always on the same time as the Raspberry Pi NTP server. I've even set the workstations to talk to the pfSense NTP instance... and they stay in sync and on the correct time. All I did in the NTP service was add the IP of the NTP server (local network) and set it prefer and server.

It all seems to be working.... but appears to me, it's just the widget that lags.

Any Ideas?

Rick

This is on a Netgate SG-4860.

Note the time difference. This was after sitting about 2 hours. This is a workstation synced to the Raspberry Pi NTP server.

Here you can see when a Windows PC updates to the GPS based NTP server, it gets the right time. My 3rd reference is my phone and they are all on the same minute. I'm comfortable with that granularity for this topic as you can see when it sits the drift is in ever growing minutes.

[anthonys]

Hi @Ramosel,

It all seems to be working.... but appears to me, it's just the widget that lags

I believe you are correct. Check the existing discussion Dashboard - NTP widget Server Time

[JonathanLee]

Did you create a NAT rule for NTP?

I have an alias set up for 192.168.1.1 and 127.0.0.1 and anything not going to them "!" negated send to the firewalls IP 192.168.1.1 that will serve the requests. Works great like this. I had issues with manually setting it all the time I set them all to the firewalls IP, as time was jumping all over during tests like my TIME was hacked by 10-15 mins. Weird right, well it was a cyber security class so what do you expect you only learn by trial and error.

[stephenw10]

If it's configured only as a PPS source it will remain accurate but not actually sync the time. Though I would expect that only to apply to local source in pfSense itself.
How is ntp configured in pfSense?

[Ramosel]

@anthonys said in NTP widget time wrong?:

Hi @Ramosel,

It all seems to be working.... but appears to me, it's just the widget that lags

I believe you are correct. Check the existing discussion Dashboard - NTP widget Server Time

Thanks Anthonys. I wonder why your thread didn't show in my search?

I thought I'd messed up the NTP server build and was going through Phil's time refinement process... the more I worked on the GPS-NTP-PPS thing the more the NTP clock drifted (probably not, but it sure felt that way).

Just odd to me that the widget for a "TIME" server has a dysfunctional clock.

[bingo600]

@Ramosel said in NTP widget time wrong?:

Just odd to me that the widget for a "TIME" server has a dysfunctional clock.

What time server are you using ??
pfSense is a firewall, not a Time server.
And my guess is that the NTP server is distributing the correct time, mine does.
It's just the Widget that doesn't spend CPU-Cycles on autorefresh.

IMHO - It's kind'a nitpicking.
I have absolutely no issue, having to hit "Refresh" to update the time. Especially when i have the correct time in the lower left of my PC desktop screen.

If you need a display with an accurate clock, buy one.

SNTP sync'ed - pricey
https://www.american-time.com/product/clock-digital-poe-4-red-6-digit/

Or use TVB's "Java Clock", since your PC is already in sync.
http://leapsecond.com/java/gpsclock.htm

You could even DIY an ESP32 based NTP sync'ed display clock for a decent price.
http://w8bh.net/NTP_DualClock.pdf
And sync it to your new Raspi "Stratum 0" Clock.

/Bingo

[JonathanLee]

@bingo600 do you have a NAT for your NTP set???

[Ramosel]

@stephenw10 said in NTP widget time wrong?:

If it's configured only as a PPS source it will remain accurate but not actually sync the time. Though I would expect that only to apply to local source in pfSense itself.
How is ntp configured in pfSense?

Thanks, Steve! Just odd this is normal considering what the "time server" widget is there for. Is this an issue that is even noted for revisit with the pfSense code slingers?

My NTP is just set with the new GPS/NTP server IP address on the local lan, preferred, server
Then I have two pools, not preferred, pool
2.north-america.pool.ntp.org
2.pfSense.pool.NHP.org
and lastly, one nist server, as a server
time-c-b.nist.gov

That gives me the head count for 3 or more sources.

Off topic: I'm still waiting on OpenWRT 23.05 release to nail down the "switch issue" with the Marvell chipset. Although the configuration you and I worked out has not shown the problem many others were seeing (I think the complexity of the configuration superseded possibility of mishandling routing on the vlans). But, I think the writing is on the wall for the Marvell based products. And, they'll never support mesh. So I've started playing with the e8450 as apparently Mediatek has hired one of the OpenWRT devs to write their drivers... so I'm figuring that is one of the chipsets that will remain viable and supported in OpenWRT for some time to come.

[bingo600]

@JonathanLee
I suppose you mean NAT redirection , to catch "Naugthy clients" that doesn't adhere to the NTP server they get from DHCP

Nope ...
All my NTP clients seem to be well behaved.
I don't log many rogue NTP requests.

I only allow NTP requests to TFW (pfSense IF's) on all but one Vlan.
My pfSense & Linux servers are Synced to my two "stratum 0" Boxes , and have a few selected "stratum 1" servers as backup.

/Bingo

[Ramosel]

@bingo600 said in NTP widget time wrong?:

What time server are you using ??
pfSense is a firewall, not a Time server.
And my guess is that the NTP server is distributing the correct time, mine does.
It's just the Widget that doesn't spend CPU-Cycles on autorefresh.

Bingo, now that I understand what the issue is, I agree. But spending the time to build up a GPS based Stratum 1 time server then have the widget lag... was a bit disconcerting. I spent a lot of time trying to fix something that wasn't broke. Unfortunately, when I searched the issue here, I wasn't shown Anthonys' previous thread. Of all the widgets on the dashboard, I somehow had the mistaken idea the NTP widget would be the ONE to have have accurate "TIME".

The time server I am using is Raspberry Pi based, uses a GPS hat from Uputronics and after a couple of other builds (that showed the same widget issue in pfSense), I settled on the configuration from Phil Randal (Phil's Occasional Blog). His build is much more involved but really ties down the "time" function of GPS as opposed to location or movement. He also makes it easy to use or not use various GPS sources from config files on the server. The other option is to remove the hat and run u-Blox u-Center software to access and configure the GPS settings of the M8/8 chip via USB-C on the hat itself.

All of the time needy devices on my network are getting their time from pfSense, I may need to reassess that. But, until now, it has worked for years with offsite sources.

Hopefully the devs will look at this and fix it at some point... wasted cpu cycles or not. My little SG-4860 generally runs at about 10% load, so I think I have the available cycles to support a fully functional widget.

Until then, maybe an option to NOT show a dysfunctional clock and a disclaimer of it's shortcomings might be a nice stopgap.

[NollipfSense]

@Ramosel said in NTP widget time wrong?:

time-c-b.nist.gov

Just tried adding that...somehow pfSense is not seeing it as a valid address after clicking save. I take it the "c" is for central time.

Tried the IP and it took that...

[johnpoz]

@NollipfSense time-c-g.nist.gov resolves just fine. But you shouldn't be marking things as pools when they are not pools.

;; QUESTION SECTION:                                         
;time-c-g.nist.gov.             IN      A                    
                                                             
;; ANSWER SECTION:                                           
time-c-g.nist.gov.      3600    IN      A       129.6.15.30  

https://tf.nist.gov/tf-cgi/servers.cgi

Looks like you have a space on the end there as well..

[NollipfSense]

@johnpoz Thanks John, I'll fix that.

That worked...

[Ramosel]

@NollipfSense said in NTP widget time wrong?:

@Ramosel said in NTP widget time wrong?:

time-c-b.nist.gov

Just tried adding that...somehow pfSense is not seeing it as a valid address after clicking save. I take it the "c" is for central time.

No, it's just a lesser used IPV4 server in Boulder. Dave MIlls is an old acquaintance and when I spoke with him about my pfSense setup a few years back he said the a-b servers get hammered the most... all the IPV6 guys flock to the d-b and e-b servers... use c-b.

https://tf.nist.gov/tf-cgi/servers.cgi

[bingo600]

@Ramosel
That is actually not a bad price for the "Hat" ... €50 from France.

Right now i'm doing a DIY NTP server, with an OrangePI One.
That will be built in my - RAPCO 1804 L25C GPSDO Receiver

I have chosen the OrangePi over the RasPi's (I have 10+ 3B's in the drawer), due to the Lan CHIP being connected directly to the SoC.
The RasPi 3B Lan chip is actually USB coupled, and that causes a bit of jitter.

This is my current "burn in" test of the OrangePi.
With a DS3231 IIC Module - https://www.aliexpress.com/item/32822420722.html

$ sudo i2cdetect -y 1
     0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f
00:                         -- -- -- -- -- -- -- -- 
10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
30: 30 -- 32 33 34 35 36 -- -- -- -- -- -- -- -- -- 
40: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
50: 50 51 52 53 54 55 56 57 58 59 5a 5b 5c 5d 5e 5f 
60: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
70: -- -- -- -- -- -- -- --                

$ sudo hwclock --verbose
hwclock from util-linux 2.36.1
System Time: 1695225080.007973
Trying to open: /dev/rtc0
Using the rtc interface to the clock.
Last drift adjustment done at 1695223549 seconds after 1969
Last calibration done at 1695223549 seconds after 1969
Hardware clock is on UTC time
Assuming hardware clock is kept in UTC time.
Waiting for clock tick...
...got clock tick
Time read from Hardware Clock: 2023/09/20 15:51:41
Hw clock time : 2023/09/20 15:51:41 = 1695225101 seconds since 1969
Time since last adjustment is 1552 seconds
Calculated Hardware Clock drift is 0.000000 seconds
2023-09-20 17:51:39.554254+02:00

The hardware will be "transplanted" into the GPSDO Box , and i will piggyback on the existing Trimble GPS receiver (RX + 1PPS).
Then install & run GPSD + NTP w. 1-PPS support

/Bingo

[JonathanLee]

@bingo600 yes, that is what I mean to NAT to your NTP server's IP address for clients not manually going to it. For me I use authenticated NTP so the firewall points to those when requests hit the firewall.

[Ramosel]

@bingo600 said in NTP widget time wrong?:

@Ramosel
That is actually not a bad price for the "Hat" ... €50 from France.

Right now i'm doing a DIY NTP server, with an OrangePI One.
That will be built in my - RAPCO 1804 L25C GPSDO Receiver

I like that, sweet. Thanks for the info. Most of these builds are turning off the USB and using GPIO. The i2c gets really simple. Phil gets really deep into fine tuning after you get it running... I need to go back and do more, but I though the Pi NTP server was causing the widget issue. Now that I know it's not, I'll dig deeper into the GPS/NTP/PPS dialog.

pi@RPi4b-NTP-GPS:~ $ i2cdetect -y 1
0 1 2 3 4 5 6 7 8 9 a b c d e f
00: -- -- -- -- -- -- -- --
10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
40: -- -- 42 -- -- -- -- -- -- -- -- -- -- -- -- --
50: -- -- UU -- -- -- -- -- -- -- -- -- -- -- -- --
60: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
70: -- -- -- -- -- -- -- --

I'm curious if Phil's build would work on the Orange Pi??
http://www.philrandal.co.uk/blog/archives/2019/04/entry_213.html

[NollipfSense]

Cannot get the devices to agree on time, one will be -0.046s while another device will be +0.045s

[bingo600]

@Ramosel said in NTP widget time wrong?:

I'm curious if Phil's build would work on the Orange Pi??
http://www.philrandal.co.uk/blog/archives/2019/04/entry_213.html

The Trimble TSIP protocol is somewhat special, compared to NMEA.
But i'm in contact with the maintainer of the TSIP part of GPSD, and he's quite sure GPSD will work.
Even with a receive only (Uart RX + 1-PPS) connection. I can't (won't) connect TX to the OrangePI, as that "belongs to the Rapco".

If i go totally crazy i could test for "missing 1-PPS" for ie. 20 sec , and then switch over to a 1-PPS divided down from the 10MHz OCXO in the Rapco.
That way "I'll be somewhat in sync .. Depends on the holdover routine in the Rapco" even with a GPS outage.

Hmmm .... Maybe the HW dependent part should be moved to the "Off Topic thread" you made.
https://forum.netgate.com/topic/182811/raspberry-gps-based-time-server

/Bingo

[Ramosel]

@bingo600 said in NTP widget time wrong?:

Hmmm .... Maybe the HW dependent part should be moved to the "Off Topic thread" you made.
https://forum.netgate.com/topic/182811/raspberry-gps-based-time-server

Sure, now that I know I'm not crazy and the NTP clock is really grunged... I'm good here. I doubt that I'd ever build what you have, but you've got me interested and I'd sure like to follow the build. Use my thread or start your own, I'm cool either way.

Rick