Status NTP never displays any peers
-
Alix 2D13 256MB nanoBSD
2.2-BETA (i386)
built on Wed Dec 03 02:58:05 CST 2014
FreeBSD 10.1-RELEASE
(and as long as I can remember looking)Status->NTP always says:
No peers found, is the ntp service running?.
My NTP has all default settings. In config.xml the only occurrence of ntp is a blank:
and
<timeservers>0.pfsense.pool.ntp.org</timeservers>/var/etc/ntpd.conf is:
# # pfSense ntp configuration file # tinker panic 0 # Orphan mode stratum tos orphan 12 # Upstream Servers server 0.pfsense.pool.ntp.org iburst maxpoll 9 disable monitor statsdir /var/log/ntp logconfig =syncall +clockall driftfile /var/db/ntpd.drift restrict default kod limited nomodify nopeer notrap restrict -6 default kod limited nomodify nopeer notrapIf I go to the shell and try the NTP command that is done in status_ntpd.php I get:
[2.2-BETA][root@testoffice-rt-01.xxx.xxx]/root: /usr/local/sbin/ntpq -pn /usr/local/sbin/ntpq: write to localhost failed: Operation not permittedSo I guess that is why the webGUI cannot display the NTP status.
I go to a test APU system, 64-bit, SD card nanoBSD… and NTP Status on the webGUI works, and the command gives good output also:
[2.2-BETA][root@apu22.localdomain]/root: /usr/local/sbin/ntpq -pn remote refid st t when poll reach delay offset jitter ============================================================================== *210.23.18.205 .PPS. 1 u 56 64 1 109.969 66.964 23.977On both systems, ntpd.conf is identical, and the process running ntpd has the same command and is user root.
Any clues on what might be causing this "Operation not permitted" error?
-
Check your rules for ipv4 and ipv6 localhost (udp).
-
Can you please explain what you mean, charliem?
-
ntpq operates by sending a query packet over the loopback interface to the ntpd daemon. If that query cannot be sent, then you see the message quoted above "write to localhost failed: Operation not permitted".
So, either the daemon is not listening on the loopback interface(s), or there is a firewall rule in place that is blocking packets to the loopback interface(s). Since Phil has two machines configured similarly, it's more likely that rules are different between the two, and one is inadvertently blocking localhost.
You can check the interfaces ntpd is listening on by looking at the ntp log file; here are my loopback interfaces:
Nov 27 06:53:42 pfsense ntpd[66066]: Listen normally on 5 lo0 127.0.0.1:123 Nov 27 06:53:42 pfsense ntpd[66066]: Listen normally on 6 lo0 [::1]:123I'm sure there are better ways to check loopback rules (I'm a pf rule neophyte), but this is a start:
[2.2-BETA][root@pfsense.localdomain]/root: grep loop /tmp/rules.debug loopback = "{ lo0 }" # loopback pass in on $loopback inet all tracker 1000006861 label "pass IPv4 loopback" pass out on $loopback inet all tracker 1000006862 label "pass IPv4 loopback" pass in on $loopback inet6 all tracker 1000006863 label "pass IPv6 loopback" pass out on $loopback inet6 all tracker 1000006864 label "pass IPv6 loopback" -
Here's mine:
Dec 4 00:03:13 ntpd[51683]: Listen normally on 9 em0_vlan210 192.168.51.1:123 Dec 4 00:03:13 ntpd[51683]: setsockopt IPV6_MULTICAST_IF 0 for fe80::21b:21ff:fecf:8c1f%9 fails: Can't assign requested address Dec 4 00:03:13 ntpd[51683]: Listen normally on 8 em0_vlan210 [fe80::21b:21ff:fecf:8c1f%9]:123 Dec 4 00:03:13 ntpd[51683]: Listen normally on 7 em0_vlan120 10.1.2.1:123 Dec 4 00:03:13 ntpd[51683]: setsockopt IPV6_MULTICAST_IF 0 for fe80::21b:21ff:fecf:8c1f%7 fails: Can't assign requested address Dec 4 00:03:13 ntpd[51683]: Listen normally on 6 em0_vlan120 [fe80::21b:21ff:fecf:8c1f%7]:123 Dec 4 00:03:13 ntpd[51683]: Listen normally on 5 lo0 [::1]:123 Dec 4 00:03:13 ntpd[51683]: Listen normally on 4 lo0 127.0.0.1:123 Dec 4 00:03:13 ntpd[51683]: Listen normally on 3 em0 192.168.50.1:123 Dec 4 00:03:13 ntpd[51683]: setsockopt IPV6_MULTICAST_IF 0 for fe80::21b:21ff:fecf:8c1f%1 fails: Can't assign requested address Dec 4 00:03:13 ntpd[51683]: Listen normally on 2 em0 [fe80::21b:21ff:fecf:8c1f%1]:123 Dec 4 00:03:13 ntpd[51683]: Listen and drop on 1 v4wildcard 0.0.0.0:123 Dec 4 00:03:13 ntpd[51683]: Listen and drop on 0 v6wildcard [::]:123loopback = "{ lo0 }" # loopback pass in on $loopback inet all tracker 1000008961 label "pass IPv4 loopback" pass out on $loopback inet all tracker 1000008962 label "pass IPv4 loopback" pass in on $loopback inet6 all tracker 1000008963 label "pass IPv6 loopback" pass out on $loopback inet6 all tracker 1000008964 label "pass IPv6 loopback"At least here, seems ok.
but still:
2.2-BETA][root@firewall]/root: ntpq -pn ntpq: write to localhost failed: Operation not permitted -
NTP logs for;
- APU that works OK:
Dec 4 03:14:37 ntpd[49485]: Listen normally on 10 lo0 [fe80::1%6]:123 Dec 4 03:14:37 ntpd[49485]: Listen normally on 9 lo0 [::1]:123 Dec 4 03:14:37 ntpd[49485]: Listen normally on 8 lo0 127.0.0.1:123- Alix with "Operation not permitted" message:
Dec 3 20:43:53 ntpd[47185]: Listen normally on 9 lo0 [fe80::1%7]:123 Dec 3 20:43:53 ntpd[47185]: Listen normally on 8 lo0 [::1]:123 Dec 3 20:43:53 ntpd[47185]: Listen normally on 7 lo0 127.0.0.1:123They both look reasonable and similar.
Mentions of loopback or lo0 in /tmp/rules.debug are the same as Hugovsky's post.
Edit, add: And I disabled all policy-routing rules on the Alix (it has 2 WAN, APU has 1 WAN) and checked that there are no mentions of any of the GW/GWGs in the rules - so all traffic that gets through the rule set must fall through to the normal routing table. That should eliminate any possibility that the localhost write is being redirected out a gateway.
Now to think about what else is different between the 2 systems (32 and 64 bit - but I hope it is not a 32-bit-specific bug!)
-
Hmm … Can you see if there is a difference when specifying -4 versus specifying -6? _Turning up the debugging with -ddd:
[2.2-BETA][root@pfsense.localdomain]/root: ntpq -ddd -6 -pn | head -10
[31276] [31275] [31274] [31273]
4 associations total
Opening host localhost (AF_INET6)
Sending 12 octets
Got packet, size = 28
Packet okay
1 packets reassembled into response
remote refid st t when poll reach delay offset jitterSending 12 octets
Got packet, size = 428
Packet okay[2.2-BETA][root@pfsense.localdomain]/root: ntpq -ddd -4 -pn | head -10
[31276] [31275] [31274] [31273]
4 associations total
Opening host localhost (AF_INET)
Sending 12 octets
Got packet, size = 28
Packet okay
1 packets reassembled into response
remote refid st t when poll reach delay offset jitterSending 12 octets
Got packet, size = 424
Packet okayWithout -4 or -6, my system here goes for AF_INET6, but they all work. Note that I'm running an earlier snapshot (64bit, built on Tue Nov 25 01:23:50 CST 2014), so perhaps the issue is recent. I'll try a later snapshot this weekend and see if I can break mine too._
-
On working APU:
[2.2-BETA][root@apu22.localdomain]/root: /usr/local/sbin/ntpq -pn remote refid st t when poll reach delay offset jitter ============================================================================== *210.23.18.205 .PPS. 1 u 33 64 377 116.861 248.125 109.124 [2.2-BETA][root@apu22.localdomain]/root: /usr/local/sbin/ntpq -pn -4 remote refid st t when poll reach delay offset jitter ============================================================================== *210.23.18.205 .PPS. 1 u 35 64 377 116.861 248.125 109.124 [2.2-BETA][root@apu22.localdomain]/root: /usr/local/sbin/ntpq -pn -6 remote refid st t when poll reach delay offset jitter ============================================================================== *210.23.18.205 .PPS. 1 u 38 64 377 116.861 248.125 109.124 [2.2-BETA][root@apu22.localdomain]/root: sockstat -6 | grep ntpd root ntpd 49485 20 udp6 *:123 *:* root ntpd 49485 23 udp6 fe80:1::20d:b9ff:fe33:8888:123 *:* root ntpd 49485 24 udp6 fe80:2::20d:b9ff:fe33:8889:123 *:* root ntpd 49485 27 udp6 fe80:3::20d:b9ff:fe33:888a:123 *:* root ntpd 49485 29 udp6 ::1:123 *:* root ntpd 49485 30 udp6 fe80:6::1:123 *:* root ntpd 49485 31 udp6 fe80:8::20d:b9ff:fe33:8888:123 *:*On Alix where it does not work:
[2.2-BETA][root@testoffice-rt-01.np.net.inf.org]/root: /usr/local/sbin/ntpq -pn /usr/local/sbin/ntpq: write to localhost failed: Operation not permitted [2.2-BETA][root@testoffice-rt-01.np.net.inf.org]/root: /usr/local/sbin/ntpq -pn -4 remote refid st t when poll reach delay offset jitter ============================================================================== *27.114.150.13 160.45.10.8 2 u 50 64 3 122.322 12.623 18.597 [2.2-BETA][root@testoffice-rt-01.np.net.inf.org]/root: /usr/local/sbin/ntpq -pn -6 /usr/local/sbin/ntpq: write to localhost failed: Operation not permitted [2.2-BETA][root@testoffice-rt-01.xxx.xxx]/root: sockstat -6 | grep ntpd root ntpd 54268 20 udp6 *:123 *:* root ntpd 54268 22 udp6 fe80:1::20d:b9ff:fe24:59c0:123 *:* root ntpd 54268 24 udp6 fe80:2::20d:b9ff:fe24:59c1:123 *:* root ntpd 54268 25 udp6 fe80:3::20d:b9ff:fe24:59c2:123 *:* root ntpd 54268 28 udp6 ::1:123 *:* root ntpd 54268 29 udp6 fe80:7::1:123 *:* root ntpd 54268 30 udp6 fe80:9::6202:b4ff:fe0c:4539:123 *:* root ntpd 54268 32 udp6 fe80:a::20d:b9ff:fe24:59c0:123 *:* root ntpd 54268 34 udp6 fe80:b::20d:b9ff:fe24:59c0:123 *:* root ntpd 54268 36 udp6 fe80:c::20d:b9ff:fe24:59c0:123 *:* root ntpd 54268 38 udp6 fe80:d::20d:b9ff:fe24:59c0:123 *:* root ntpd 54268 40 udp6 fe80:e::20d:b9ff:fe24:59c0:123 *:* root ntpd 54268 42 udp6 fe80:f::20d:b9ff:fe24:59c0:123 *:* root ntpd 54268 44 udp6 2001:470:35:a7c::2:123 *:* root ntpd 54268 45 udp6 fe80:10::20d:b9ff:fe24:59c0:123 *:* root ntpd 54268 46 udp6 fe80:11::20d:b9ff:fe24:59c0:123 *:* root ntpd 54268 48 udp6 fe80:12::20d:b9ff:fe24:59c0:123 *:*So it is defaulting to using IPv6.
I looked at System->Advanced, Networking. On the Alix, "Allow IPv6" was unchecked. I checked it, and now the NTP command responds fine. And Status->NTP displays in the webGUI.
So I guess the firewall is blocking all IPv6 when that box is unchecked.
I wonder what should be done about this?
If that box is unchecked, then NTP could be configured to use just IPv4, or at least use IPv4 first. -
yes changed the allow ipv6 fixed mine too
-
This fixes it in a simple way by doing "ntpq -4" if ipv6allow is off.
Works on both my systems, switching ipv6allow back and forth.
https://github.com/pfsense/pfsense/pull/1361 has been committed.Another obscure little thing tracked down.
There might be other little things that happen like this due to the general IPv6 block rule. Others might like to think what else is likely to have odd effects and need to be explicitly told to use IPv4. And soon it will be time to remove that check box - the world will have to progress some day!
-
true soon it we will have native ipv6 just won't be from my 2 isp options windstream or fairpoint
-
Damn… never thought of ipv6 checkbox... Mine is unchecked too. Nice debug! But I'm almost sure it was working maybe a month ago or so.
-
G Gertjan referenced this topic on
-
G Gertjan referenced this topic on