Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    NTP server not working post-2.2 upgrade

    Scheduled Pinned Locked Moved General pfSense Questions
    12 Posts 4 Posters 4.9k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • C Offline
      charliem
      last edited by

      Try the '-u' option on ntpdate to use an unprivileged port.

      port 123 udp needs to be open

      1 Reply Last reply Reply Quote 0
      • S Offline
        sporkme
        last edited by

        Thanks - I don't think either of those is really applicable though.

        From what I can gather, this is a config generation issue.

        The ntpd crash also seems to have been addressed previously, but I don't see exactly what the current status is.

        1 Reply Last reply Reply Quote 0
        • C Offline
          cmb
          last edited by

          Most likely:
          https://redmine.pfsense.org/issues/4155

          Under some circumstance that isn't exactly replicable (that I've found or that anyone else has been able to provide), ntpd crashes on 32 bit when there is an interface change of some sort. If you have a scenario or config where that's reliably replicable, that would help.

          Using 64 bit instead will fix the problem.

          1 Reply Last reply Reply Quote 0
          • C Offline
            charliem
            last edited by

            @sporkme:

            Thanks - I don't think either of those is really applicable though.

            For some reason you are not getting all the required responses from the server, see below.  With ntpdate, default number of sample requests sent is four; your devel machine sent four queries, but only the first two were answered, or answered properly.

            @sporkme:

            [spork@devel4 ~]$ ntpdate -d 192.168.11.1
            2 Apr 22:32:16 ntpdate[5180]: ntpdate 4.2.4p5-a (1)
            transmit(192.168.11.1)     
            receive(192.168.11.1)          <== T/R OK
            transmit(192.168.11.1)
            receive(192.168.11.1)          <== T/R OK
            transmit(192.168.11.1)        <== No response
            transmit(192.168.11.1)        <== No response
            transmit(192.168.11.1)        <== No response
            192.168.11.1: Server dropped: strata too high        <== ntpdate bails, something is wrong with 192.168.11.1
            server 192.168.11.1, port 123
            stratum 16, precision -6, leap 11, trust 000
            refid [192.168.11.1], delay 0.02605, dispersion 24.00186
            transmitted 4, in filter 4
            reference time:    00000000.00000000  Thu, Feb  7 2036  1:28:16.000
            originate timestamp: d8c87a30.69123d7c  Thu, Apr  2 2015 22:32:16.410
            transmit timestamp:  d8c87a31.6916bd01  Thu, Apr  2 2015 22:32:17.410
            filter delay:  0.02605  0.04166  0.00000  0.00000      <== See only two valid values in delay history samples
                    0.00000  0.00000  0.00000  0.00000
            filter offset: 0.003524 -0.00021 0.000000 0.000000
                    0.000000 0.000000 0.000000 0.000000
            delay 0.02605, dispersion 24.00186
            offset 0.003524

            2 Apr 22:32:18 ntpdate[5180]: no server suitable for synchronization found

            The crashing bug is something else you're right, probably 4155 as Chris mentioned.  Seems to be triggered when ntpd is killed & restarted when any of the newwanip scripts are run, maybe related to 32 bit or [openvpn, ipsec] interfaces.  I'm running my systems with the patch I posted here, letting ntpd handle interface changes on its own:
            https://forum.pfsense.org/index.php?topic=88359.msg496016#msg496016.  Not because I had any crashing issues, but ntpd has better timekeeping when it's not stopped and restarted often.  If you have success with that patch, can you post back here or in the bug report?

            If your ntpd happens to crash after answering the first two queries, that could explain your client ntpdate issue.

            1 Reply Last reply Reply Quote 0
            • B Offline
              bplein
              last edited by

              @cmb:

              Using 64 bit instead will fix the problem.

              I can reproduce this on 64bit pfSense.

              2.2.1-RELEASE (amd64)
              built on Fri Mar 13 08:16:49 CDT 2015

              Attempting from a FreeBSD 9.x system on my network:

              # ntpdate -d 172.17.0.1
               9 Apr 23:30:45 ntpdate[5173]: ntpdate 4.2.4p5-a (1)
              transmit(172.17.0.1)
              receive(172.17.0.1)
              transmit(172.17.0.1)
              receive(172.17.0.1)
              transmit(172.17.0.1)
              transmit(172.17.0.1)
              transmit(172.17.0.1)
              172.17.0.1: Server dropped: strata too high
              server 172.17.0.1, port 123
              stratum 16, precision -6, leap 11, trust 000
              refid [172.17.0.1], delay 0.02620, dispersion 24.00563
              transmitted 4, in filter 4
              reference time:    00000000.00000000  Thu, Feb  7 2036  0:28:16.000
              originate timestamp: d8d1d075.a3ebf7c2  Thu, Apr  9 2015 23:30:45.640
              transmit timestamp:  d8d1d076.a3bed31b  Thu, Apr  9 2015 23:30:46.639
              filter delay:  0.02620  0.04164  0.00000  0.00000 
                       0.00000  0.00000  0.00000  0.00000 
              filter offset: 0.011050 -0.00020 0.000000 0.000000
                       0.000000 0.000000 0.000000 0.000000
              delay 0.02620, dispersion 24.00563
              offset 0.011050
              
              
              1 Reply Last reply Reply Quote 0
              • C Offline
                charliem
                last edited by

                @bplein:

                Attempting from a FreeBSD 9.x system on my network:

                # ntpdate -d 172.17.0.1
                 9 Apr 23:30:45 ntpdate[5173]: ntpdate 4.2.4p5-a (1)
                
                

                @sporkme:

                
                [spork@devel4 ~]$ ntpdate -d 192.168.11.1
                 2 Apr 22:32:16 ntpdate[5180]: ntpdate 4.2.4p5-a (1)
                
                

                These both seem to be problems with ntpdate, or rather how older ntpdate reacts to modern ntpd server versions.  This is not related to bug 4155, where ntpd crashes with (at least) openvpn interface changes.

                I was able to duplicate the "only two response when there should be 4 or more" with ntpdate 4.2.4p5-a on a FreeBSD 9.x and 10.1 system against an ntpd 4.2.8p1 server.  (But a linux ntpdate 4.2.6p5 had no problem).  Then I built ntpdate from the ports collection and it successfully sync'd using the same command (below, 192.168.2.128 is my pfsense 2.2.2 server running ntpd 4.2.8p1 from ports):

                Failure with ntpdate 4.2.4p5-a:

                root@:~ # uname -a
                FreeBSD  10.1-RELEASE FreeBSD 10.1-RELEASE #0 r274401: Tue Nov 11 21:02:49 UTC 2014    root@releng1.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC  amd64
                root@:~ # ntpdate -d 192.168.2.128
                10 Apr 18:33:28 ntpdate[63086]: ntpdate 4.2.4p5-a (1)
                transmit(192.168.2.128)
                receive(192.168.2.128)
                transmit(192.168.2.128)
                receive(192.168.2.128)
                transmit(192.168.2.128)
                transmit(192.168.2.128)
                transmit(192.168.2.128)
                192.168.2.128: Server dropped: strata too high
                server 192.168.2.128, port 123
                stratum 16, precision -6, leap 11, trust 000
                refid [192.168.2.128], delay 0.02649, dispersion 24.00044
                transmitted 4, in filter 4
                reference time:    00000000.00000000  Thu, Feb  7 2036  6:28:16.000
                originate timestamp: d8d295f8.59639498  Fri, Apr 10 2015 18:33:28.349
                transmit timestamp:  d8d295f9.595945d6  Fri, Apr 10 2015 18:33:29.349
                filter delay:  0.02649  0.04248  0.00000  0.00000
                        0.00000  0.00000  0.00000  0.00000
                filter offset: 0.000283 -0.00062 0.000000 0.000000
                        0.000000 0.000000 0.000000 0.000000
                delay 0.02649, dispersion 24.00044
                offset 0.000283

                10 Apr 18:33:30 ntpdate[63086]: no server suitable for synchronization found

                Success on the same client with ntpdate 4.2.8p1 from ports:

                root@:~ # /usr/local/sbin/ntpdate -d 192.168.2.128
                10 Apr 18:37:12 ntpdate[63089]: ntpdate 4.2.8p1@1.3265-o Fri Apr 10 17:57:46 UTC 2015 (1)
                Looking for host 192.168.2.128 and service ntp
                192.168.2.128 reversed to pfsense.localdomain
                host found : pfsense.localdomain
                transmit(192.168.2.128)
                receive(192.168.2.128)
                transmit(192.168.2.128)
                receive(192.168.2.128)
                transmit(192.168.2.128)
                receive(192.168.2.128)
                transmit(192.168.2.128)
                receive(192.168.2.128)
                server 192.168.2.128, port 123
                stratum 2, precision -21, leap 00, trust 000
                refid [192.168.2.128], delay 0.02620, dispersion 0.00047
                transmitted 4, in filter 4
                reference time:    d8d295e2.b5bc7f8a  Fri, Apr 10 2015 18:33:06.709
                originate timestamp: d8d296de.6d307fc2  Fri, Apr 10 2015 18:37:18.426
                transmit timestamp:  d8d296de.6d182b34  Fri, Apr 10 2015 18:37:18.426
                filter delay:  0.02649  0.02632  0.02620  0.02658
                        0.00000  0.00000  0.00000  0.00000
                filter offset: -0.00052 -0.00021 0.000272 -0.00019
                        0.000000 0.000000 0.000000 0.000000
                delay 0.02620, dispersion 0.00047
                offset 0.000272

                10 Apr 18:37:18 ntpdate[63089]: adjust time server 192.168.2.128 offset 0.000272 sec

                Note that ntpdate is being deprecated, in favor of using ntpd in the 'one-shot' mode: "ntpd -qg".  Here the -q means quit as soon as the client is synchronized, and -g means step the clock, a lot, if necessary.  The drawbacks to this method are needing to make sure that ntpd is not already running as a daemon, and also not being able to specify a host on the command line (the normal servers configured in /var/etc/ntpd.conf are used).  This approach does work with the older distributed versions of ntpd, so if you can't build ntpd from ports, that's an option you can use with older clients.

                One could argue that there's a bug with ntpd, in that clients that were able to use ntpdate are no longer able to do so, but that may not get much traction as ntpdate is on its way out.  In any case, it's not a pfSense bug

                1 Reply Last reply Reply Quote 0
                • B Offline
                  bplein
                  last edited by

                  Interesting.

                  But that doesn't help for clients, like the IPMI client embedded in one of my servers, where I can't control their implementation.

                  I guess I will need to point various systems at external sources rather than my pfSense router.

                  1 Reply Last reply Reply Quote 0
                  • S Offline
                    sporkme
                    last edited by

                    @charliem:

                    One could argue that there's a bug with ntpd, in that clients that were able to use ntpdate are no longer able to do so, but that may not get much traction as ntpdate is on its way out.  In any case, it's not a pfSense bug

                    True, but the old ntpdate had no issues after I changed ntp settings away from defaults on pfsense:

                    If I uncheck everything in the "access restrictions" section, all is well.

                    That's the section under Services->NTP labelled "Access restrictions" with the odd note that says "these options control access to NTP from the WAN", which seems odd as they actually seem to have an effect on LAN clients and I can't imagine anyone adding the WAN interface to the list of IPs without firewalling that off.  For completeness, these are the parameters that if I uncheck them allow ntpdate and ntpd to work across all the LAN hosts:

                    So that's one issue, and that's dealt with (for me at least, others running older ntpdate will have to google).

                    The crashing issue persists, so I'm trying your suggestion of commenting out the ntpd restarts in rc.newwanip and rc.newwanipv6.  I think all the clock skew that causes is also triggering issues with rekeying on one of my ipsec tunnels, so maybe that will get fixed as well.

                    What's interesting about this ntpd crash is how regular it is.  It's not every tun up/down, but there's a pattern:

                    
                    pid 27217 (ntpd), uid 0: exited on signal 11 (core dumped)
                    ovpns1: link state changed to DOWN
                    ovpns1: link state changed to UP
                    ovpns1: link state changed to DOWN
                    ovpns1: link state changed to UP
                    ovpns1: link state changed to DOWN
                    ovpns1: link state changed to UP
                    ovpns1: link state changed to DOWN
                    ovpns1: link state changed to UP
                    pid 29157 (ntpd), uid 0: exited on signal 11 (core dumped)
                    ovpns1: link state changed to DOWN
                    ovpns1: link state changed to UP
                    ovpns1: link state changed to DOWN
                    ovpns1: link state changed to UP
                    ovpns1: link state changed to DOWN
                    ovpns1: link state changed to UP
                    ovpns1: link state changed to DOWN
                    ovpns1: link state changed to UP
                    pid 84000 (ntpd), uid 0: exited on signal 11 (core dumped)
                    ovpns1: link state changed to DOWN
                    ovpns1: link state changed to UP
                    ovpns1: link state changed to DOWN
                    ovpns1: link state changed to UP
                    ovpns1: link state changed to DOWN
                    ovpns1: link state changed to UP
                    ovpns1: link state changed to DOWN
                    ovpns1: link state changed to UP
                    
                    

                    And I'm I the only one that gets nervous about software with this kind of security track record just falling over with a sig 11?  Who would put money on whether something exploitable lurks in this crash?

                    1 Reply Last reply Reply Quote 0
                    • C Offline
                      charliem
                      last edited by

                      @sporkme:

                      That's the section under Services->NTP labelled "Access restrictions" with the odd note that says "these options control access to NTP from the WAN", which seems odd as they actually seem to have an effect on LAN clients and I can't imagine anyone adding the WAN interface to the list of IPs without firewalling that off.  For completeness, these are the parameters that if I uncheck them allow ntpdate and ntpd to work across all the LAN hosts:

                      Any idea which one allows the older ntpdate to work?  Does the preferred method "ntpd -gq" work for you with the defaults?

                      The crashing issue persists, so I'm trying your suggestion of commenting out the ntpd restarts in rc.newwanip and rc.newwanipv6.  I think all the clock skew that causes is also triggering issues with rekeying on one of my ipsec tunnels, so maybe that will get fixed as well.

                      Interested in results you see.

                      And I'm I the only one that gets nervous about software with this kind of security track record just falling over with a sig 11?  Who would put money on whether something exploitable lurks in this crash?

                      I doubt it's exploitable, as it seems to be triggered by a race condition stopping/starting ntpd.

                      1 Reply Last reply Reply Quote 0
                      • S Offline
                        sporkme
                        last edited by

                        @charliem:

                        @sporkme:

                        That's the section under Services->NTP labelled "Access restrictions" with the odd note that says "these options control access to NTP from the WAN", which seems odd as they actually seem to have an effect on LAN clients and I can't imagine anyone adding the WAN interface to the list of IPs without firewalling that off.  For completeness, these are the parameters that if I uncheck them allow ntpdate and ntpd to work across all the LAN hosts:

                        Any idea which one allows the older ntpdate to work?  Does the preferred method "ntpd -gq" work for you with the defaults?

                        Not sure, I can test again at some point, but I've already annoyed people enough with my nagios alerts on ntp skew. :)

                        @charliem:

                        The crashing issue persists, so I'm trying your suggestion of commenting out the ntpd restarts in rc.newwanip and rc.newwanipv6.  I think all the clock skew that causes is also triggering issues with rekeying on one of my ipsec tunnels, so maybe that will get fixed as well.

                        Interested in results you see.

                        So far so good.  No ntpd crashes, and it might be too soon to tell, but no IPSEC VPN drops either, which I assume is just a side-effect of more accurate timekeeping.

                        1 Reply Last reply Reply Quote 0
                        • First post
                          Last post
                        Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.