NTP and stratum issue



  • I'm trying to use pfsense (2.2.6) as a time server for the LAN.

    Clients are failing because the stratum being returned is 16 when I debug with 'ntpdate -d pfsense':

    stratum 16, precision -6, leap 11, trust 000

    When I use ntpq on the pfsense box - the 'rv' command tells me the stratum is 2.

    ntpq> rv
    associd=0 status=0615 leap_none, sync_ntp, 1 event, clock_sync,
    version="ntpd 4.2.8p4@1.3265-o Mon Oct 26 14:28:17 UTC 2015 (1)",
    processor="amd64", system="FreeBSD/10.1-RELEASE-p25", leap=00, stratum=2,
    precision=-23, rootdelay=63.249, rootdisp=10.466, refid=132.163.4.101,
    reftime=da3174e2.f45f84b6  Fri, Jan  1 2016 16:57:54.954,
    clock=da3175dc.165390e2  Fri, Jan  1 2016 17:02:04.087, peer=16120, tc=7,
    mintc=3, offset=2.566951, frequency=18.819, sys_jitter=2.789931,
    clk_jitter=4.064, clk_wander=0.035

    I'm pretty confused on how to fix this. I'd like the firewall to be the time sever, but I don't know how to debug this any further.

    For completeness, here's the output of peers, and as.

    ntpq> peers
        remote          refid      st t when poll reach  delay  offset  jitter

    *time-a.timefreq  .ACTS.          1 u  122  128  377  63.249    2.567  2.406
    +host-24-56-178- .ACTS.          1 u  51  128  377  55.824  12.147  0.818
    +gps.layer42.net .GPS.            1 u  59  128  377  72.650  15.905  3.152
    -time-b.nist.gov .ACTS.          1 u  57  128  173  70.204  47.305  1.220

    ntpq> as

    ind assid status  conf reach auth condition  last_event cnt

    1 16120  963a  yes  yes  none  sys.peer    sys_peer  3
      2 16121  944d  yes  yes  none candidate              4
      3 16122  9424  yes  yes  none candidate  reachable  2
      4 16123  9324  yes  yes  none  outlier  reachable  2


  • Rebel Alliance Global Moderator

    You also got a Server dropped: Leap not in sync in your ntpdate did you not… I see your leap bit set to 11 there..  But your rv doesn't show that.. You sure your not just resolving pfsense to the wrong time server?



  • I'm positive I'm hitting the right IP for the pfsense box.

    I'll delete the unresponsive time server, and see what that does for us.



  • More info - if I do a ntpdate -ud localhost, I get results that match ntpq rv. If I run the same command from any machine on the LAN, I get stratum 16.

    NOTE: I also shut the NTP service down on the pfsense box, and I confirmed that I could no longer reach the service from the clients and localhost.

    10.1.1.1 is the pfsense box

    $ ssh admin@10.1.1.1
    *** Welcome to pfSense 2.2.6-RELEASE-pfSense (amd64) on pfSense ***

    Running against localhost

    [2.2.6-RELEASE][admin@pfSense]/root: ntpdate -ud localhost
    1 Jan 17:54:08 ntpdate[13292]: ntpdate 4.2.8p4@1.3265-o Mon Oct 26 14:28:19 UTC 2015 (1)
    Looking for host localhost and service ntp
    127.0.0.1 reversed to localhost
    host found : localhost
    transmit(127.0.0.1)
    receive(127.0.0.1)
    transmit(::1)
    receive(::1)
    transmit(127.0.0.1)
    receive(127.0.0.1)
    transmit(::1)
    receive(::1)
    transmit(127.0.0.1)
    receive(127.0.0.1)
    transmit(::1)
    receive(::1)
    transmit(127.0.0.1)
    receive(127.0.0.1)
    transmit(::1)
    receive(::1)
    server 127.0.0.1, port 123
    stratum 3, precision -23, leap 00, trust 000
    refid [127.0.0.1], delay 0.02567, dispersion 0.00000
    transmitted 4, in filter 4
    reference time:    da3181e4.9049c7d3  Fri, Jan  1 2016 17:53:24.563
    originate timestamp: da318216.52f855dc  Fri, Jan  1 2016 17:54:14.324
    transmit timestamp:  da318216.52f477d5  Fri, Jan  1 2016 17:54:14.324
    filter delay:  0.02573  0.02568  0.02567  0.02568
            0.00000  0.00000  0.00000  0.00000
    filter offset: 0.000024 -0.00000 -0.00000 -0.00000
            0.000000 0.000000 0.000000 0.000000
    delay 0.02567, dispersion 0.00000
    offset -0.000002

    server ::1, port 123
    stratum 3, precision -23, leap 00, trust 000
    refid [::1], delay 0.02568, dispersion 0.00000
    transmitted 4, in filter 4
    reference time:    da3181e4.9049c7d3  Fri, Jan  1 2016 17:53:24.563
    originate timestamp: da318216.862b731b  Fri, Jan  1 2016 17:54:14.524
    transmit timestamp:  da318216.8627c7d6  Fri, Jan  1 2016 17:54:14.524
    filter delay:  0.02573  0.02570  0.02568  0.02570
            0.00000  0.00000  0.00000  0.00000
    filter offset: 0.000018 0.000004 0.000003 0.000004
            0.000000 0.000000 0.000000 0.000000
    delay 0.02568, dispersion 0.00000
    offset 0.000003

    1 Jan 17:54:14 ntpdate[13292]: adjust time server 127.0.0.1 offset -0.000002 sec

    Running from a client

    [root@test] ~# ntpdate -ud 10.1.1.1
    1 Jan 17:54:52 ntpdate[8615]: ntpdate 4.2.4p5-a (1)
    transmit(10.1.1.1)
    receive(10.1.1.1)
    transmit(10.1.1.1)
    receive(10.1.1.1)
    transmit(10.1.1.1)
    transmit(10.1.1.1)
    transmit(10.1.1.1)
    10.1.1.1: Server dropped: strata too high
    server 10.1.1.1, port 123
    stratum 16, precision -6, leap 11, trust 000
    refid [10.1.1.1], delay 0.04140, dispersion 24.07111
    transmitted 4, in filter 4
    reference time:    00000000.00000000  Thu, Feb  7 2036  1:28:16.000
    originate timestamp: da31823c.d7c0ee0f  Fri, Jan  1 2016 17:54:52.842
    transmit timestamp:  da31823d.ba086f2d  Fri, Jan  1 2016 17:54:53.726
    filter delay:  0.14259  0.04140  0.00000  0.00000
            0.00000  0.00000  0.00000  0.00000
    filter offset: 0.142123 -0.00008 0.000000 0.000000
            0.000000 0.000000 0.000000 0.000000
    delay 0.04140, dispersion 24.07111
    offset -0.000080

    1 Jan 17:54:54 ntpdate[8615]: no server suitable for synchronization found


  • Rebel Alliance Global Moderator

    Dude your not hitting the same server it looks to me.. You forwarding traffic?

    On pfsense change your host to your pfsense IP and do rv, and do same thing from localhost

    They should be exactly the same

    ntpq> host localhost
    current host set to localhost
    ntpq> rv
    associd=0 status=0618 leap_none, sync_ntp, 1 event, no_sys_peer,
    version="ntpd 4.2.8p4@1.3265-o Mon Oct 26 14:28:17 UTC 2015 (1)",
    processor="amd64", system="FreeBSD/10.1-RELEASE-p25", leap=00, stratum=3,
    precision=-18, rootdelay=13.319, rootdisp=59.874, refid=192.168.9.40,
    reftime=da319e93.4960f3d0  Fri, Jan  1 2016 18:55:47.286,
    clock=da31a29e.32685bb3  Fri, Jan  1 2016 19:13:02.196, peer=20504, tc=8,
    mintc=3, offset=0.475810, frequency=-55.601, sys_jitter=0.230866,
    clk_jitter=0.087, clk_wander=0.002
    ntpq> host 192.168.9.253
    current host set to 192.168.9.253
    ntpq> rv
    associd=0 status=0618 leap_none, sync_ntp, 1 event, no_sys_peer,
    version="ntpd 4.2.8p4@1.3265-o Mon Oct 26 14:28:17 UTC 2015 (1)",
    processor="amd64", system="FreeBSD/10.1-RELEASE-p25", leap=00, stratum=3,
    precision=-18, rootdelay=13.319, rootdisp=60.129, refid=192.168.9.40,
    reftime=da319e93.4960f3d0  Fri, Jan  1 2016 18:55:47.286,
    clock=da31a2ae.87be472a  Fri, Jan  1 2016 19:13:18.530, peer=20504, tc=8,
    mintc=3, offset=0.475810, frequency=-55.601, sys_jitter=0.230866,
    clk_jitter=0.087, clk_wander=0.002
    ntpq>



  • ummm - yeah I am. The weird thing is that 'ntpdate -ud 10.1.1.1' and 'ntpq -c rv 10.1.1.1' return different results on a client box when pointed to the pfsense box.

    I know ntpdate is being depreciated but this is just a bit nuts.

    I've clients, e.g. the Freenas box, that won't accept the pfsense box as a time server because of the stratum issue. I'm wondering if this is a software version mismatch issue?

    This is all done on the pfsense box

    ntpq> host localhost
    current host set to localhost
    ntpq> rv
    associd=0 status=0615 leap_none, sync_ntp, 1 event, clock_sync,
    version="ntpd 4.2.8p4@1.3265-o Mon Oct 26 14:28:17 UTC 2015 (1)",
    processor="amd64", system="FreeBSD/10.1-RELEASE-p25", leap=00, stratum=2,
    precision=-23, rootdelay=24.781, rootdisp=585.121, refid=204.9.54.119,
    reftime=da31a54c.232af4c7  Fri, Jan  1 2016 20:24:28.137,
    clock=da31a561.e76f2bd6  Fri, Jan  1 2016 20:24:49.904, peer=58444, tc=6,
    mintc=3, offset=28.685194, frequency=18.604, sys_jitter=117.371580,
    clk_jitter=7.981, clk_wander=0.000
    ntpq> host 10.1.1.1
    current host set to 10.1.1.1
    ntpq> rv
    associd=0 status=0615 leap_none, sync_ntp, 1 event, clock_sync,
    version="ntpd 4.2.8p4@1.3265-o Mon Oct 26 14:28:17 UTC 2015 (1)",
    processor="amd64", system="FreeBSD/10.1-RELEASE-p25", leap=00, stratum=2,
    precision=-23, rootdelay=24.781, rootdisp=585.436, refid=204.9.54.119,
    reftime=da31a54c.232af4c7  Fri, Jan  1 2016 20:24:28.137,
    clock=da31a576.73554a5e  Fri, Jan  1 2016 20:25:10.450, peer=58444, tc=6,
    mintc=3, offset=28.685194, frequency=18.604, sys_jitter=117.371580,
    clk_jitter=7.981, clk_wander=0.000
    ntpq>

    This is done on a Freenas box

    ntpq> host 10.1.1.1
    current host set to 10.1.1.1
    ntpq> rv
    assID=0 status=0615 leap_none, sync_ntp, 1 event, event_clock_reset,
    version="ntpd 4.2.8p4@1.3265-o Mon Oct 26 14:28:17 UTC 2015 (1)",
    processor="amd64", system="FreeBSD/10.1-RELEASE-p25", leap=00,
    stratum=2, precision=-23, rootdelay=25.191, rootdisp=67.854,
    refid=204.9.54.119,
    reftime=da31a5c9.235647a3  Fri, Jan  1 2016 20:26:33.138,
    clock=da31a622.b3b0f7cd  Fri, Jan  1 2016 20:28:02.701, peer=58444,
    tc=6, mintc=3, offset=-0.122, frequency=18.603, sys_jitter=22.813403,
    clk_jitter=12.192, clk_wander=0.000
    ntpq> q
    [root@freenas] ~# ntpdate -ud 10.1.1.1
    1 Jan 20:28:46 ntpdate[15064]: ntpdate 4.2.4p5-a (1)
    transmit(10.1.1.1)
    receive(10.1.1.1)
    transmit(10.1.1.1)
    receive(10.1.1.1)
    transmit(10.1.1.1)
    transmit(10.1.1.1)
    transmit(10.1.1.1)
    10.1.1.1: Server dropped: strata too high
    server 10.1.1.1, port 123
    stratum 16, precision -6, leap 11, trust 000
    refid [10.1.1.1], delay 0.02571, dispersion 24.01199
    transmitted 4, in filter 4
    reference time:    00000000.00000000  Thu, Feb  7 2036  1:28:16.000
    originate timestamp: da31a64e.39fe35ff  Fri, Jan  1 2016 20:28:46.226
    transmit timestamp:  da31a64f.3a2ea971  Fri, Jan  1 2016 20:28:47.227
    filter delay:  0.02571  0.04134  0.00000  0.00000
            0.00000  0.00000  0.00000  0.00000
    filter offset: 0.023948 -0.00004 0.000000 0.000000
            0.000000 0.000000 0.000000 0.000000
    delay 0.02571, dispersion 24.01199
    offset 0.023948

    1 Jan 20:28:48 ntpdate[15064]: no server suitable for synchronization found
    [root@freenas] ~#


  • Rebel Alliance Global Moderator

    you are using an older version of ntpdate.. Why don't you just update it?

    Current dev version is 4.3.88 it is syncing fine to pfsense ntp server

    user@clean:~$ ntpdate -d pfsense.local.lan
    1 Jan 21:46:42 ntpdate[29545]: ntpdate 4.3.88@1.2483 Mon Dec 28 22:23:20 UTC 2015 (1)
    Looking for host pfsense.local.lan and service ntp
    192.168.9.253 reversed to pfSense.local.lan
    host found : pfSense.local.lan
    transmit(192.168.9.253)
    receive(192.168.9.253)
    transmit(192.168.9.253)
    receive(192.168.9.253)
    transmit(192.168.9.253)
    receive(192.168.9.253)
    transmit(192.168.9.253)
    receive(192.168.9.253)
    server 192.168.9.253, port 123
    stratum 3, precision -18, leap 00, trust 000
    refid [192.168.9.253], delay 0.02600, dispersion 0.00003
    transmitted 4, in filter 4
    reference time:    da31c5f2.447ed54e  Fri, Jan  1 2016 21:43:46.267
    originate timestamp: da31c6a8.e20bc1c1  Fri, Jan  1 2016 21:46:48.882
    transmit timestamp:  da31c6a8.e1e17584  Fri, Jan  1 2016 21:46:48.882
    filter delay:  0.02606  0.02605  0.02606  0.02600
            0.00000  0.00000  0.00000  0.00000
    filter offset: 0.000203 0.000183 0.000202 0.000250
            0.000000 0.000000 0.000000 0.000000
    delay 0.02600, dispersion 0.00003
    offset 0.000250

    1 Jan 21:46:48 ntpdate[29545]: adjust time server 192.168.9.253 offset 0.000250 sec
    user@clean:~$



  • Thanks for the help! I love being the first to find version mismatch bugs.

    Rather than update the entire office of OSX clients and Freenas servers to a newer version of ntpdate, I think we're just going to make ntpdate an alias to 'ntpd -qg -c <special config="">' or something along those lines. That way we don't have to worry about dependencies etc.

    I'll probably file a bug with Freenas also so someone else doesn't waste an afternoon chasing this one down.</special>


  • Rebel Alliance Global Moderator

    Dude but isn't 4.2.4p5 really really freaking old??  didn't 4.2.4p5 come out in like Aug of 2008??  What version of os x and freebsd are you running that that would be the ntp version?

    So help validate this, I grabbed an OLD version for windows - couldn't find the exact 4.2.4p5 but p8 pretty close, and yeah fails..




  • @fatsailor:

    stratum 16, precision -6, leap 11, trust 000

    See exactly the same with a older CentOS (guess 6.4) Server, Debian 7/8 works.


  • Rebel Alliance Global Moderator

    Yeah it looks like there is a issue between versions of ntp..  What version of ntp is the centos box running



  • @johnpoz:

    What version of ntp is the centos box running

    Don't know which version was affected, now it is a CentOS 6.7 with ntp.x86_64 4.2.6p5-5.el6.centos.2 and it working as expected.



  • ntpdate is rather bitrotted and has been on the pathway to deprecation for some time.

    Depending on the usage case, sntp or ntpd -gq should be used - the link in the previous paragraph gives some background and implementation considerations.

    FreeBSD base system only abandoned ntp 4.2.4 in the second half of 2015.



  • @fatsailor:

    Rather than update the entire office of OSX clients and Freenas servers to a newer version of ntpdate, I think we're just going to make ntpdate an alias to 'ntpd -qg -c <special config="">' or something along those lines. That way we don't have to worry about dependencies etc.

    I'll probably file a bug with Freenas also so someone else doesn't waste an afternoon chasing this one down.</special>

    It was reported here that choosing certain parameters in NTP access restrictions would work around this bug in older ntpdate client versions.  I haven't tested myself, but if it works then you wouldn't have to touch each client.

    Yes, 'ntpd -gq' is the right approach going forward, but it may not be practical to upgrade client machines.


  • Rebel Alliance Global Moderator

    So looking at that thread.. And a simple test with old ntpdate, its the KOD packet that is killing the old version for some reason..

    If you uncheck the Kiss-o'-death, then ntpdate from old client works… If you have it enabled then it fails with stratum 16, and leap 11

    edit:  Seems if kod is enabled, it also places a limited in the restrictions, which is a conflict when you have no monitor set as well, which is in the conf file but don't see anyway in the gui to edit that.

    Jan 5 04:05:38 ntpd[11014]: restrict: 'monitor' cannot be disabled while 'limited' is enabled

    If you uncheck kod, then the kod is removed from the conf as well as the limited.

    disable monitor
    statsdir /var/log/ntp
    logconfig =syncall +clockall +peerall +sysall
    driftfile /var/db/ntpd.drift
    restrict default nomodify nopeer notrap
    restrict -6 default nomodify nopeer notrap

    If you check the kod option in access restrictions you get
    disable monitor
    statsdir /var/log/ntp
    logconfig =syncall +clockall +peerall +sysall
    driftfile /var/db/ntpd.drift
    restrict default kod limited nomodify nopeer notrap
    restrict -6 default kod limited nomodify nopeer notrap






  • @johnpoz:

    And a simple test with old ntpdate, its the KOD packet that is killing the old version for some reason..

    If you uncheck the Kiss-o'-death, then ntpdate from old client works… If you have it enabled then it fails with stratum 16, and leap 11

    In this case, kiss-of-death is working as intended, but ntpdate earlier that some time during the 4.2.7 cycle fails to understand it.

    Dave Hart posted an excellent analysis of this scenario in the comp.protocols.time.ntp newsgroup.

    I found:
    discard average 6 minimum 0
    got my ntp servers working with old ntpdate binaries and kiss-of-death enabled.


  • Rebel Alliance Global Moderator

    why wouldn't you update vs downgrade..  I show it working with current ntpdate.



  • @johnpoz:

    why wouldn't you update vs downgrade..  I show it working with current ntpdate.

    It's a compatibility thing. Nothing here runs ntp older than 4.2.8p4 (4.2.8p5 has just released), but there are circumstances where older remote clients need to run against my servers.



  • Thank you for this topic. Was pulling my hair out why my Thecus nas was the only device not able to sync with time my pfsense box.
    All test done from my windows pc showed no issues. Looking on the nas I found out it is not using ntpd / ntpq but ntpdate (ntpdate 4.2.4p8 to be exact)

    throug a long way of searching came to this topic.

    After setting "ACL - Custom Access Restrictions" in pfSense for the IP of my NAS and disabeling KOD, all is working fine


  • Rebel Alliance Global Moderator

    Glad the topic was of use of you HeMan.. When I saw this show up as something new it - was at first ah shit some spammer necro an old thread ;)  Nice to see it was someone actual used the info contained in old threads for what they are meant for ;)