Issues with IPV6
-
Hello,
Since a couple of weeks I have issues with IPV6 for unkown reasons. Without knowing the cause of the issues, I expect that there are multiple causes.
I noticed the following:
- PPOE interface fails after some time; A restart fixes this;
- the IPV6 DHCP-server did not provide addresses with settings which used to work in the past. Returning to the default values seems to have solved that problem. Note that my original settings did shorten all kind of hold times to the minimum, in order to have them expired as soon as possible in favor of testing.
- Now and than I have a connection, but no internet due to lack of an (IPV6) IP. Probably related to either the failing PPOE or the DHCP-server.
I noticed that there are some issues in the bug list, no idea if my observations are related.
A couple of month ago my provider did make changes to their network, that could be a factor as well. However I do not think so, since the problems I am observing now seems to start on a later moment than that provider network change.
The described issues are a bit vague I know, however something is not properly working!! So this is partly an attention signal as well as a request to hear your experiences.
-
Today, running the latest build, I did lose IPV6 connectivity again. As soon as you notice the issue, it can simply be ^fixed^ by restating the WAN ..... but something is not OK.
Louis
-
Below some alarms copied from the log
Jul 31 10:20:15 pfSense dpinger[7853]: WAN_DHCP6 fe80::9217:3fff:fe7f:e4a1%pppoe1: Alarm latency 2900us stddev 3734us loss 22%
Jul 29 13:09:36 pfSense dpinger[7853]: send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr fe80::9217:3fff:fe7f:e4a1%pppoe1 bind_addr fe80::6a05:caff:fe5a:813d%pppoe1 identifier "WAN_DHCP6 "
Jul 29 13:09:36 pfSense dpinger[7607]: send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr 195.190.228.107 bind_addr 77.174.182.228 identifier "WAN_PPPOE "
Jul 29 13:09:36 pfSense dpinger[65137]: exiting on signal 15
Jul 29 13:09:36 pfSense dpinger[66074]: exiting on signal 15
Jul 29 13:09:35 pfSense dpinger[66074]: send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr fe80::9217:3fff:fe7f:e4a1%pppoe1 bind_addr fe80::6a05:caff:fe5a:813d%pppoe1 identifier "WAN_DHCP6 "
Jul 29 13:09:35 pfSense dpinger[65137]: send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr 195.190.228.107 bind_addr 77.174.182.228 identifier "WAN_PPPOE "
Jul 29 13:09:35 pfSense dpinger[82037]: exiting on signal 15
Jul 29 13:09:35 pfSense dpinger[82385]: exiting on signal 15
Jul 29 13:09:33 pfSense dpinger[82385]: send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr fe80::9217:3fff:fe7f:e4a1%pppoe1 bind_addr fe80::6a05:caff:fe5a:813d%pppoe1 identifier "WAN_DHCP6 "
Jul 29 13:09:33 pfSense dpinger[82037]: send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr 195.190.228.107 bind_addr 77.174.182.228 identifier "WAN_PPPOE "
Jul 29 13:09:33 pfSense dpinger[31117]: exiting on signal 15
Jul 29 13:09:31 pfSense dpinger[31117]: send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr fe80::9217:3fff:fe7f:e4a1%pppoe1 bind_addr fe80::6a05:caff:fe5a:813d%pppoe1 identifier "WAN_DHCP6 "
Jul 29 13:09:25 pfSense dpinger[89346]: exiting on signal 15
Jul 29 13:09:25 pfSense dpinger[89737]: exiting on signal 15
Jul 29 13:09:24 pfSense dpinger[89346]: WAN_PPPOE 195.190.228.107: sendto error: 65
Jul 29 13:09:24 pfSense dpinger[89737]: WAN_DHCP6 fe80::9217:3fff:fe7f:e4a1%pppoe1: sendto error: 50
Jul 29 13:09:24 pfSense dpinger[89346]: WAN_PPPOE 195.190.228.107: sendto error: 65
Jul 29 13:09:23 pfSense dpinger[89346]: WAN_PPPOE 195.190.228.107: sendto error: 65
Jul 29 10:19:55 pfSense dpinger[89737]: WAN_DHCP6 fe80::9217:3fff:fe7f:e4a1%pppoe1: Alarm latency 3032us stddev 3890us loss 21%
Jul 29 09:21:44 pfSense dpinger[89737]: send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr fe80::9217:3fff:fe7f:e4a1%pppoe1 bind_addr fe80::6a05:caff:fe5a:813d%pppoe1 identifier "WAN_DHCP6 "
Jul 29 09:21:44 pfSense dpinger[89346]: send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr 195.190.228.107 bind_addr 77.174.182.228 identifier "WAN_PPPOE " -
Today I again noticed that the IPV6 gateway was gone!
Turning off and on the gateway did re-activate IPV6, however something is definitively not OK
Below some alarms !
Louis
Aug 15 12:01:39 pfSense dpinger[332]: WAN_DHCP6 fe80::9217:3fff:fe7f:e4a1%pppoe1: Alarm latency 3229us stddev 4333us loss 21%
Aug 11 21:39:49 pfSense dpinger[332]: send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr fe80::9217:3fff:fe7f:e4a1%pppoe1 bind_addr fe80::6a05:caff:fe5a:813d%pppoe1 identifier "WAN_DHCP6 "
Aug 11 21:39:49 pfSense dpinger[99533]: send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr 195.190.^C.D^ bind_addr 77.174.^A.B^ identifier "WAN_PPPOE "
Aug 11 21:39:49 pfSense dpinger[85113]: exiting on signal 15
Aug 11 21:39:49 pfSense dpinger[85685]: exiting on signal 15
Aug 11 21:39:47 pfSense dpinger[85685]: send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr fe80::9217:3fff:fe7f:e4a1%pppoe1 bind_addr fe80::6a05:caff:fe5a:813d%pppoe1 identifier "WAN_DHCP6 "I also noticed the strange alarm below, but I do not expect it to be related.
-
@louis2
Have you tried 'Disable Gateway Monitoring Action'?- Go to: System / Routing / Gateways
- Edit the gateway with description Interface WAN_DHCP6 Gateway
- check Gateway Action
Disable Gateway Monitoring Action - Save and apply changes
We can investigate further if it doesn't work or if you want to keep this function.
-
No, its important to know (=monitor) if the gateways are working correctly!
Next to that, I do not know what I have to think from all those dpringer alarms
-
Here:
@louis2 said in Issues with IPV6:Jul 29 13:09:24 pfSense dpinger[89346]: WAN_PPPOE 195.190.228.xxx: sendto error: 65
Jul 29 13:09:24 pfSense dpinger[89737]: WAN_DHCP6 fe80::9217:3fff:fe7f:e4a1%pppoe1: sendto error: 50
Jul 29 13:09:24 pfSense dpinger[89346]: WAN_PPPOE 195.190.228.xxx: sendto error: 65
Jul 29 13:09:23 pfSense dpinger[89346]: WAN_PPPOE 195.190.228.xxx: sendto error: 65
Jul 29 10:19:55 pfSense dpinger[89737]: WAN_DHCP6 fe80::9217:3fff:fe7f:e4a1%pppoe1: Alarm latency 3032us stddev 3890us loss 21%the entire uplink goes bad; not only IPv6.
Btw : the only relevant lines are :Jul 31 10:20:15 pfSense dpinger[7853]: WAN_DHCP6 fe80::9217:3fff:fe7f:e4a1%pppoe1: Alarm latency 2900us stddev 3734us loss 22%
22% means : some IPv6 pings came back - some didn't = connection bad ?
Again, it not just IPv6, its also IPv4 - the entire WAN_PPPOE goes down ones.All the others are dpinger exiting and restarting (because there was some WAN interface .event ?).
-
Hum:
- In the GUI I never saw the IPV4-interface down, or noticed that it was down !!
- I have a fiber connection from kpn.nl which is a first class provider;
- I recent months I noticed a couple of times that the IPV6-GW was Red, when I logged into pfSense to check or modify something. So it does not look like a short glitch. Restarting the interface "solves" the problem.
The provider is providing the internet (and voice) via vlan6 and tv via vlan4 (I use internet only).
The arriving fiber is terminated on a provider owned fiber switch/termination point. From there the vlan's are passed towards pfSense via a managed Zyxel switch.
So apart form the fiber-switch and the vlan's transported via the rock stable managed switch there is "nothing", what could spoil the connectivity. And that used to work, it is only relatively recent that I now and than notice that the IPV6 is gone / GW is red.
-
I had a lot of these
in my Status > System Logs > System > Gateways log page.
All on August 20.
I remember doing some testing with pfBlockerng-devel, so I force reloaded the settings. That will trigger what I would call an Interface event, and interface related tasks will restart, like dpinger.Since 20/08 and now : not a single message from dpinger : no restart, no ping loss.
Btw : I use pfBlockerng-devel,, and only sync files (feeds) once a week, not every hour (!!), as most feeds are rarely updated anyway.
So my dpinger only gets restarted once a week (I guess)I'm not sure if lowering the number of dpinger restarts will help for you, but I guess it's worth it to try.
My ISP in Franc is Orange, using an VDSL, IPv6 from Orange sucks, as they only give a /64 == totally useless for a multi LAN setup. I use the IPv6 from tunnel.he.net, the last free mondial IPv6 ISP with a pop in Paris.
@louis2 said in Issues with IPV6:
from kpn.nl which is a first class provider
Ik heb enige ervaring met die andere ISPs ..... niet echt great idd ;)
-
No, its important to know (=monitor) if the gateways are working correctly!
Of course, I was thinking it might just be a lazy ping response. It will still monitor the gateway but the sensitivity might need tuning. Is there any noticeable pattern to when the disruptions occur, e.g. time of day, hours between occurence? Are there any DHCP/DHCP6 logs in the time leading up to a disruption?
About the ARP warning, do you recognize the MAC address, is it one of your devices, or maybe a phone, laptop, or another router?
7c:b0:c2belongs to Intel, the gateway appears to be Huawei. Any pattern to the occurence?How many leases does your ISP allow, e.g. I may request two IPv4 leases, and up to five IPv6 /56 delegations. After a reset, do you get the same ip/ip6 addresses every time?
What's the Zyxel switch for? Untagging, monitoring, or is pfSense one-armed? Any IGMP configuration?
-
Related to your question: "Is there any noticeable pattern to when the disruptions occur, e.g. time of day, hours between occurrence? Are there any DHCP/DHCP6 logs in the time leading up to a disruption?"
I do not know, to analyze that I have to download the logs and analyze the them offline. I will consider that option.Related to the ARP messages, perhaps it is related to Synching firewall event. See below. There seems to be an relation. I have no idea to which device the mac-address belongs. I also wonder why there is the word "bogon".
Aug 23 21:59:58 pfSense arpwatch[58415]: bogon 0.0.0.0 7c:b0:c2:f2:df:55
Aug 23 21:59:57 pfSense arpwatch[58415]: bogon 0.0.0.0 7c:b0:c2:f2:df:55
Aug 23 10:55:00 pfSense sshguard[22416]: Now monitoring attacks.
Aug 23 10:55:00 pfSense sshguard[73281]: Exiting on signal.
Aug 23 10:53:29 pfSense check_reload_status[481]: Syncing firewallRelated to leases. I have one fixed IPV4 address and an /48 IPV6-range. So I will constantly have the same addresses.
The Zyxel switch is my 1G-main/central switch. Related to this, it is used to separate and forward vlans from the Fiber access point to pfSense. As example originally I did forward the internet vlan (vlan6) to pfsense but not the TV-vlan (vlan4).
-
@louis2 said in Issues with IPV6:
Related to the ARP messages, perhaps it is related to Synching firewall event. See below. There seems to be an relation. I have no idea to which device the mac-address belongs. I also wonder why there is the word "bogon".
For info Arpwatch reports bogons frequently
From what I make of it : DHCP clients broadcast intially with a 0.0.0.0.0.0 MAC to trigger a DHCP server reply.
The Arpwatch 'scanner' will detect this, and add a bogon line, as he knows that 0.0.0.0.0.0 is actually known as for example aa.bb.cc.dd.ee.ff.
The bogon messages are nearly always inoffensive. -
This morning I noticed that the IPV6-gateway was gone again
Stopping and starting the wan does ^fix^ the problem, but of course that is not OK.Since previous time this happened, I did setup a syslog server (graylog) in order to fetch all potential related alarms.
In the logging I found three messages which draw my attention. when filtering the log using message ^like "ppp"^
<12>1 2022-09-09T04:12:53.468066+02:00 pfSense.lan dpinger 76225 - - WAN_DHCP6 fe80::9217:3fff:fe7f:e4a1%pppoe1: Alarm latency 3558us stddev 4502us loss 22%
Using the time as new selection criteria I selected the alarms occurring around this time. Starting and ending with an alarm which IMHO is not related. Here they are:
<134>1 2022-09-09T04:12:35.879325+02:00 pfSense.lan filterlog 72340 - - 1316,,,1559467905,lagg0.13,match,pass,in,4,0x0,,1,64738,0,none,103,pim,46,192.168.13.1,224.0.0.13,datalength=26
<12>1 2022-09-09T04:12:53.468066+02:00 pfSense.lan dpinger 76225 - - WAN_DHCP6 fe80::9217:3fff:fe7f:e4a1%pppoe1: Alarm latency 3558us stddev 4502us loss 22%
<30>1 2022-09-09T04:12:53.474315+02:00 pfSense.lan rc.gateway_alarm 30961 - - >>> Gateway alarm: WAN_DHCP6 (Addr:fe80::9217:3fff:fe7f:e4a1%pppoe1 Alarm:1 RTT:3.558ms RTTsd:4.502ms Loss:22%)
<13>1 2022-09-09T04:12:53.474964+02:00 pfSense.lan check_reload_status 391 - - updating dyndns WAN_DHCP6
<13>1 2022-09-09T04:12:53.474990+02:00 pfSense.lan check_reload_status 391 - - Restarting OpenVPN tunnels/interfaces
<13>1 2022-09-09T04:12:53.474983+02:00 pfSense.lan check_reload_status 391 - - Restarting IPsec tunnels
<13>1 2022-09-09T04:12:53.474997+02:00 pfSense.lan check_reload_status 391 - - Reloading filter
<30>1 2022-09-09T04:12:55.499322+02:00 pfSense.lan unbound 21401 - - [21401:3] info: 2a02:a468:c48e:11::f051 printer.lan. AAAA IN
Note that I am a bit surprised to see messages related to OpenVPN as well .... since I am not using OpenVPN
-
@louis2 said in Issues with IPV6:
since I am not using OpenVPN
And neither IPSEC, right ?
When an interface event arrives, processes that use that interface are restarted.
If OpenVPN, or IPSEC are not running, then that's ok. Nothing happens.
You also saw "updating dyndns WAN_DHCP6" as it would be logical to do a DynDNS operation if the WAN_DHCP6 was used by the a Dyndns settings, the WAN IPv6 could have been changed, and the Dyndns (might) need an update.I don't know if dpinger triggers an interface event when the loss is 22 %, as 78 % of the v6 ping packets still made it back, so the IPv6 connection still working, but 'not for "100 %".
Was it slowly degrading, dpinger wioll take action : a simple IPv6 interface restart (down and up) could correct the situation. Or not .... all depends why IPv6 went down (upstream).
Your IPv6 was probably getting ok (long/some) time afterwards, but the immediate dpinger stop start on the interface didn't work out. It stays down, until far later on you restart it.My theory, of course.
-
I assume that Dyndns is related to a dynamic address, however .... I have a fixed IP{V4 address and a fixed IPV6-range.
I can imagine .... that there was a maintenance action at the provider leading (not sure at all that that is what happend), however, that the situation does not recover ...... is not ok ....
And yep simple interface restart ^fixes^ the issue ..... but that is not OK of course
-
Today the GUI again showed that the IPV6 was gone.
To test if the IPV6 was really gone, I did ping a couple of IPV6-addresses. And ..... I could reach them
Pinging 2a00:1450:400e:801::200e with 32 bytes of data:
Reply from 2a00:1450:400e:801::200e: time=4ms
Reply from 2a00:1450:400e:801::200e: time=4ms
Reply from 2a00:1450:400e:801::200e: time=4ms
Reply from 2a00:1450:400e:801::200e: time=4msPing statistics for 2a00:1450:400e:801::200e:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 4ms, Maximum = 4ms, Average = 4msSo the conclusion seems to be that the gateway is available .... however that the GUI is not correctly updated ....
-
@louis2 said in Issues with IPV6:
So the conclusion seems to be that the gateway is available
It should be the gateway.
You've see many people people using (IPv4) 8.8.8.8 - ir 8.8.8.8 is not a gateway ;)Who are you pinging ?
To find out 'for sure' : use
grep ax | grep 'dpinger' and look at the command line used.
-
@gertjan said in Issues with IPV6:
grep ax | grep 'dpinger'
[2.7.0-DEVELOPMENT][admin@pfSense.lan]/root: nslookup google.com
Server: 127.0.0.1
Address: 127.0.0.1#53Non-authoritative answer:
Name: google.com
Address: 142.251.39.110
Name: google.com
Address: 2a00:1450:400e:800::200e[2.7.0-DEVELOPMENT][admin@pfSense.lan]/root: ping6 2a00:1450:400e:800::200e
PING6(56=40+8+8 bytes) 2a02:a468:c48e:1::1 --> 2a00:1450:400e:800::200e
16 bytes from 2a00:1450:400e:800::200e, icmp_seq=0 hlim=61 time=6.048 ms
16 bytes from 2a00:1450:400e:800::200e, icmp_seq=1 hlim=61 time=5.979 ms
16 bytes from 2a00:1450:400e:800::200e, icmp_seq=2 hlim=61 time=5.980 ms
16 bytes from 2a00:1450:400e:800::200e, icmp_seq=3 hlim=61 time=6.106 ms
16 bytes from 2a00:1450:400e:800::200e, icmp_seq=4 hlim=61 time=5.989 ms
16 bytes from 2a00:1450:400e:800::200e, icmp_seq=5 hlim=61 time=6.030 ms
16 bytes from 2a00:1450:400e:800::200e, icmp_seq=6 hlim=61 time=6.032 ms
16 bytes from 2a00:1450:400e:800::200e, icmp_seq=7 hlim=61 time=6.020 ms--- 2a00:1450:400e:800::200e ping6 statistics ---
12 packets transmitted, 12 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 5.979/6.024/6.106/0.036 ms
[2.7.0-DEVELOPMENT][admin@pfSense.lan]/root: grep ax | grep 'dpinger'
^C
[2.7.0-DEVELOPMENT][admin@pfSense.lan]/root: -
@louis2 said in Issues with IPV6:
grep ax | grep 'dpinger'
sorry,
ps ax | grep 'dpinger'
My IPv6 interface (he.net' :
78401 - Is 2:11.06 /usr/local/bin/dpinger -S -r 0 -i HENETV6_TUNNELV6 -B 2001:470:beef:5c0::2 -p /var/run/dpinger_HENETV6_TUNNELV6~2001:470:beef:5c0::2~2001:470:beef:5c0::1.pid -u /var/run/dpinger_HENETV6_TUNNELV6~2001:470:beef:5c0::2~2001:470:beef:5c0::1.sock -C /etc/rc.gateway_alarm -d 0 -s 500 -l 2000 -t 60000 -A 1000 -D 500 -L 20 2001:470:beef:5c0::1
where 2001:470:beef:5c0::1 is actually my IPv6 gateway
I can ping it - like dpinger :
[22.05-RELEASE][admin@pfSense.overhere.net]/root: ping6 2001:470:beef:5c0::1
PING6(56=40+8+8 bytes) 2001:470:beef:5c0::2 --> 2001:470:beef:5c0::1
16 bytes from 2001:470:beef:5c0::1, icmp_seq=0 hlim=64 time=45.301 ms
16 bytes from 2001:470:beef:5c0::1, icmp_seq=1 hlim=64 time=43.746 ms
16 bytes from 2001:470:beef:5c0::1, icmp_seq=2 hlim=64 time=44.482 ms
^C -
[2.7.0-DEVELOPMENT][admin@pfSense.lan]/root: ps ax | grep 'dpinger'
31465 - Is 0:09.22 /usr/local/bin/dpinger -S -r 0 -i WAN_PPPOE -B 77.174.abc.def -p /var/ru
31795 - Is 0:07.53 /usr/local/bin/dpinger -S -r 0 -i WAN_DHCP6 -B fe80::6a05:caff:fe5a:813d
88478 1 S+ 0:00.00 grep dpinger
[2.7.0-DEVELOPMENT][admin@pfSense.lan]/root: -
L louis2 referenced this topic on
-
S stephenw10 referenced this topic on
