NTP / PPS GPS_ntpd - Time Error - PPS Jitter Exceeded

pete · Jan 10, 2022, 3:43 PM

On Sure GPS / U-Blox GPS and Haswell chipset Hardware.

Jan 10 09:34:49	ntpd	59827	kernel reports TIME_ERROR: 0x2307: PPS Time Sync wanted but PPS Jitter exceeded
Jan 10 09:19:05	ntpd	59827	kernel reports TIME_ERROR: 0x2307: PPS Time Sync wanted but PPS Jitter exceeded
Jan 10 09:03:21	ntpd	59827	kernel reports TIME_ERROR: 0x2307: PPS Time Sync wanted but PPS Jitter exceeded
Jan 10 09:01:13	ntpd	59827	kernel reports TIME_ERROR: 0x2307: PPS Time Sync wanted but PPS Jitter exceeded
Jan 10 08:58:17	ntpd	59827	kernel reports TIME_ERROR: 0x2307: PPS Time Sync wanted but PPS Jitter exceeded
Jan 10 08:53:13	ntpd	59827	kernel reports TIME_ERROR: 0x2307: PPS Time Sync wanted but PPS Jitter exceeded
Jan 10 08:51:37	ntpd	59827	kernel reports TIME_ERROR: 0x2307: PPS Time Sync wanted but PPS Jitter exceeded

Any suggestions to eliminate these log entries?

stephenw10 · Jan 10, 2022, 6:25 PM

There are some sysctls you can adjust that might increase the allowed jitter. I've never had to set them though when PPS is actually working. You might argue it's not worth using a PPS signal that jittery.

pete · Jan 10, 2022, 9:56 PM

@stephenw10

Thank you.

PPS appears to be working fine on the two GPS/PPS set ups other than the log entries. Never really paid attention to the NTP log entries until recently.

I tried adding:

kern.timecounter.hardware ==> ACPI-fast

Which sort of worked for a couple of hours then the log entries came back.

I also changed the source of the 5VDC power from the PC to an aux power supply which did not do anything.

stephenw10 · Jan 10, 2022, 11:52 PM

Hmm, changing the hardware timecounter to something less accurate might stop it logging but seems counter productive.

bingo600 · Jan 11, 2022, 9:06 AM

There could be a few hints here
https://sudonull.com/post/160635-Very-accurate-time-server-stratum-1-on-Garmin-GPS

@pete
Do you use USB-Serial for 1-PPS too ?
USB (Poll) could be to "Jittery" for the 1-PPS pulse.

I see your rs-232 converter is "max3232" based
https://forum.netgate.com/topic/168711/ntp-pps-ublox-status_ntpd-gps-information-not-working

Just watch out w. those adapters , rumors are that the max3232 on most , are fake.
And some get hot and "burn out" (latchup)

But i suppose if you receive NMEA sentences , that it's ok for now.

I bought some of those adapters , and a batch of "real max3232", and changed the chip.
Even at the cost of replacing the chip, it still made the adapters a good buy.

Edit: This post seems to indicate, that you might loose GPS Fix
https://titanwolf.org/Network/Articles/Article?AID=00d0cf84-2c6e-4fc7-bd1b-7844601c8ed0

login-to-view

pete · Jan 11, 2022, 1:21 PM

Thank you guys

The old Sure GPS has a direct RS-232 output on the the board.

login-to-view

The newer UBlox GPS is connected to a TTL/RS-232 MAX3232 chipped (Amazon HiLetgo 5 pieces for $6.99).

I disabled on the Sure GPS set up.
Enable PPS signal processing (default: checked).
Unchecking it.

I see no PPS Jitter Exceeded log entries which would be expected.

login-to-view
GPS Info indicates: in view 13, in use 6

ntptime
ntp_gettime() returns code 0 (OK)
  time e587ff95.f2f0e0a8  Tue, Jan 11 2022  7:07:33.948, (.948988776),
  maximum error 35904 us, estimated error 9601 us, TAI offset 0
ntp_adjtime() returns code 0 (OK)
  modes 0x0 (),
  offset -7702.365 us, frequency -27.296 ppm, interval 256 s,
  maximum error 35904 us, estimated error 9601 us,
  status 0x2101 (PLL,PPSSIGNAL,NANO),
  time constant 4, precision 0.001 us, tolerance 496 ppm,
  pps frequency -28.814 ppm, stability 69.051 ppm, jitter 61.300 us,
  intervals 4932, jitter exceeded 27627, stability exceeded 88, errors 439.

So with PPS enabled see close to zero jitter on the NTP status but I see PPS Jitter Exceeded in the NTP logs. Most likely this the jitter logging has been there for years and I should just ignore it. It caught my eye because I was building two more of the NTP/GPS devices for a PFSense Peer.

Geez still remember setting my clock on my Hammerlund HQ-170 short wave radio in the 1960's using WWV in Fort Collins Colorado when I was a kid.

stephenw10 · Jan 11, 2022, 1:18 PM

8ms jitter is very high for PPS, I would expect to see single digit μs there. It's not using PPS in that screenshot. That's after disabling it I assume?

pete · Jan 11, 2022, 1:36 PM

@stephenw10

Yes disabling the default enabled check mark in the configuration page: (un checking it)

Enable PPS signal processing (default: checked).

This is on my PFSense box (Haswell chip set) with the old Sure GPS card as depicted.

stephenw10 · Jan 11, 2022, 1:34 PM

Ok. So when it's enabled do you actually see jitter at the expected far lower level?

pete · Jan 11, 2022, 1:55 PM

@stephenw10

Yes.

Jitter is much closer to zero on the NTP status page.

That is when I see the NTP Jitter exceeded log entries.

Just changed it back to the default settings and waiting for it to settle and will post a screen shot.

login-to-view

When upgrading to 2.5.X-RELEASE (amd64) I was more concerned about IPv6 stuff than looking at the GPS NTP logs stuff.

pete · Jan 11, 2022, 2:06 PM

Here is the U-Blox PFSense box at site #2 NTP info page.
Here I never touched the NTP PPS configuration settings.

Note: Both sites show same when configured identically except that one is using the old Sure GPS and site #2 is using the U-Blox GPS.

login-to-view

ntptime
ntp_gettime() returns code 0 (OK)
 time e5880c71.dc213a0c  Tue, Jan 11 2022  8:02:25.859, (.859882524),
 maximum error 2500 us, estimated error 8 us, TAI offset 0
ntp_adjtime() returns code 0 (OK)
 modes 0x0 (),
 offset -1.354 us, frequency -43.581 ppm, interval 256 s,
 maximum error 2500 us, estimated error 8 us,
 status 0x2107 (PLL,PPSFREQ,PPSTIME,PPSSIGNAL,NANO),
 time constant 4, precision 0.001 us, tolerance 496 ppm,
 pps frequency -43.581 ppm, stability 0.024 ppm, jitter 2.266 us,
 intervals 4257, jitter exceeded 20520, stability exceeded 2, errors 2.

bingo600 · Jan 11, 2022, 2:29 PM

@pete
It could be that your GPS is loosing Fix. And stops sending 1-PPS, at the times listed.

I have a South/East GPS antenna that is totally blind to north, it's rare i don't have 1 or 2 birds in view. But at times i don't have 3 , which is required for most NON-Timing GPS'es for a fix.

I tried a RGM2000 GPS to begin with (Sirf II LP chipset) , but it stopped sending 1-PPS when there wasn't 3 birds in view.

I switched to LEA-5T Timing GPS'es (and static coordinates) in order to be able to keep time with even just 1 GPS bird in view.

I would have loved LEA-6T or LEA-8T but they are still serious $$

pete · Jan 11, 2022, 3:15 PM

@bingo600

Thank you.

Both Sure GPS and U-Blox GPS modules are using external antennas.

Site #2 with the U-Blox GPS has the antenna mounted outside magnetically attached to the DTV dish and seeing using 8 out of 11 satellites in GPS Information page and GGA sentence sees:

$GPGGA,145638.00,4XXX.30682,N,08XXX.22024,W,1,09,0.88,218.7,M,-33.7,M,,*69

Fix type #1 = 1 = Autonomous GPS fix, no correction data used.
Number of satellites in use => 9

Site #1 with Sure GPS has antenna mounted on window sil and seeing 6 out of 11 satellites on GPS info page and sees:

$GPGGA,145938.000,4XXX.3924,N,08XXX.9252,W,2,5,2.22,233.9,M,-34.0,M,0000,0000*61

Fix type ==> 2 DGPS fix, using a local DGPS base station or correction service such as WAAS or EGNOS.
Number of satellites in use = 5

Back of both houses where GPS antennas are have a clear view of the South / SouthEastern skies (with no trees).

stephenw10 · Jan 11, 2022, 3:14 PM

Mmm, my antenna location was nothing special (before my cat broke it!) and the number of satellites in view varied quite a lot but I don't ever recall seeing those messages. I think you may have something else introducing jitter.

pete · Jan 11, 2022, 3:37 PM

@stephenw10

The oldest PFSense box was originally patched way long time ago to use the Sure GPS as discussed over here a few years ago. Looking at my old blog on Cocoontech it was around 2013 that I was posting about this on this forum.

http://cocoontech.com/forums/blog/29/entry-415-from-anyone-using-pfsense-as-a-firewall/

That said I never looked at the NTP logs.

I cannot find my old posts here relating to introduction of the Sure GPS. I recall it was much of a manual effort with adding patches.

Today Site #1 is using an updated DIY'd Haswell chipped motherboard with 6 Gb nics and Site #2 is using a Qotom firewall with 2 Gb nics. These are mine.

Two new set ups are using Qotom i5's in TX with U-Blox GPS configurations (done pro bono for peer). These will be on line next week. Curious if I will see the same. Another one going on line in Tampa, FL with same set up (pro bono GPS)

pete · Jan 15, 2022, 1:01 PM

Still seeing the Jitter logs in both the SureGPS/PPS and U-Blox GPS with PPS on the two PFSense boxes. I keep tinkering with the settings and nothing seems to work.

This past week made an adjustment to the U-Blox PFSense box which resulted in the box not booting properly.

Using the documentation relating to:

Adjust timecounter hardware settings put one setting in and kept adjusting the values.

kern.timecounter.hardware

kern.timecounter.choice: TSC-low(1000) ACPI-safe(850) i8254(0) dummy(-1000000)

The dummy setting made the PFSense box not boot or hang.

I had to go to user mode, command line prompt and mount the drive in RW mode. I then restored a back up configuration to get it working again. Once it booted I removed the setting. I had to remove the PFSense box and connect it to a keyboard and monitor to get it going again.

I got it going again then noticed that my OpenVPN server is not working now. Thinking though with the constant reboots the WAN IP changed. (hopefully that is all that changed?)

pete · Jan 30, 2022, 1:26 AM

I see your rs-232 converter is "max3232" based
https://forum.netgate.com/topic/168711/ntp-pps-ublox-status_ntpd-gps-information-not-working

Just watch out w. those adapters , rumors are that the max3232 on most , are fake.
And some get hot and "burn out" (latchup)

But i suppose if you receive NMEA sentences , that it's ok for now.

I bought some of those adapters , and a batch of "real max3232", and changed the chip.
Even at the cost of replacing the chip, it still made the adapters a good buy.

Built 3 of these using bulk purchased Amazon RS-232 to TTL boards.

The burned up last week and are not functioning.

Ordered new ones this week. Pictures show Max chips clearly marked. The 5 I purchased I cannot read any markings on the Max chips.

Went to using these from Mouser electronics. Working fine.
https://www.mouser.com/ProductDetail/713-103990363

The originally built device is still running fine after a couple of years.

Testing here looks like made one serial port on a Qotom PFSense box disfunctional. Had to connect the device to serial port 2.

I am still seeing the NTP log jitter errors on the SureGPS (untouched) and in production Qotom box.

pete · Jan 30, 2022, 8:39 AM

Fixed on the UBlox and Sure GPS with PPS configurations.

Not sure if I did this correctly.

I unchecked this option

Services / NTP / Serial GPS / Flags

Enable kernel PPS clock discipline (default: checked).

Not seeing PPS Jitter Exceeded in the NTP logs.

Would this be considered for a bug report?

stephenw10 · Jan 30, 2022, 1:23 AM

Hmm, not sure that's a bug. Might be something worth documenting though, which you could open a request for.

pete · Jan 30, 2022, 1:23 AM

@stephenw10

Thank you.