IPv6 broken beyond pfSense after 2.5 upgrade
-
I'm now on 2.5.1-RC trying to get IPv6 to work. It looks like we're getting close, but I'm now having an issue with the interfaces going up and down in a loop if I try to enable IPv6 on WAN.
Config is WAN:
DHCPv6, /56, send prefix hintLAN, vlan1,vlan2,vlan3 all set to track interface.
With the latest snapshot and change cca31114b0ac041e41865c586d587558f82979d6 from github I'm getting a gateway all the way on my PC on LAN, but the interfaces keep resetting on a loop on pfSense.
Mar 22 17:11:39 kernel config_aqm Unable to configure flowset, flowset busy!
Mar 22 17:11:39 kernel config_aqm Unable to configure flowset, flowset busy!
Mar 22 17:11:37 check_reload_status 700 Reloading filter
Mar 22 17:11:37 check_reload_status 700 updating dyndns opt1
Mar 22 17:11:36 check_reload_status 700 Reloading filter
Mar 22 17:11:36 check_reload_status 700 updating dyndns opt3
Mar 22 17:11:36 kernel config_aqm Unable to configure flowset, flowset busy!
Mar 22 17:11:36 kernel config_aqm Unable to configure flowset, flowset busy!
Mar 22 17:11:36 php-fpm 69246 /rc.linkup: The command '/usr/local/sbin/unbound -c /var/unbound/unbound.conf' returned exit code '1', the output was '[1616425896] unbound[88636:0] error: bind: address already in use [1616425896] unbound[88636:0] fatal error: could not open ports'
Mar 22 17:11:35 php-fpm 74596 /rc.linkup: The command '/usr/local/sbin/unbound -c /var/unbound/unbound.conf' returned exit code '1', the output was '[1616425895] unbound[17921:0] error: bind: address already in use [1616425895] unbound[17921:0] fatal error: could not open ports'
Mar 22 17:11:35 kernel config_aqm Unable to configure flowset, flowset busy!
Mar 22 17:11:35 check_reload_status 700 Reloading filter
Mar 22 17:11:35 check_reload_status 700 updating dyndns opt2
Mar 22 17:11:34 check_reload_status 700 Reloading filter
Mar 22 17:11:34 check_reload_status 700 updating dyndns lan
Mar 22 17:11:33 php-fpm 28703 /rc.linkup: The command '/usr/local/sbin/unbound -c /var/unbound/unbound.conf' returned exit code '1', the output was '[1616425893] unbound[24925:0] error: bind: address already in use [1616425893] unbound[24925:0] fatal error: could not open ports'
Mar 22 17:11:30 php-fpm 29377 /rc.linkup: Gateway, NONE AVAILABLE
Mar 22 17:11:30 php-fpm 69246 /rc.linkup: Gateway, NONE AVAILABLE
Mar 22 17:11:30 check_reload_status 700 Restarting ipsec tunnels
Mar 22 17:11:30 php-fpm 28703 /rc.linkup: Gateway, NONE AVAILABLE
Mar 22 17:11:30 check_reload_status 700 Restarting ipsec tunnels
Mar 22 17:11:30 php-fpm 74596 /rc.linkup: Gateway, NONE AVAILABLE
Mar 22 17:11:30 php-fpm 74596 /rc.linkup: HOTPLUG: Configuring interface opt3
Mar 22 17:11:30 php-fpm 74596 /rc.linkup: DEVD Ethernet attached event for opt3
Mar 22 17:11:30 php-fpm 28703 /rc.linkup: HOTPLUG: Configuring interface opt2
Mar 22 17:11:30 php-fpm 28703 /rc.linkup: DEVD Ethernet attached event for opt2
Mar 22 17:11:30 php-fpm 69246 /rc.linkup: HOTPLUG: Configuring interface opt1
Mar 22 17:11:30 php-fpm 69246 /rc.linkup: DEVD Ethernet attached event for opt1
Mar 22 17:11:30 php-fpm 29377 /rc.linkup: HOTPLUG: Configuring interface lan
Mar 22 17:11:30 php-fpm 29377 /rc.linkup: DEVD Ethernet attached event for lan
Mar 22 17:11:29 check_reload_status 700 Linkup starting em1.200
Mar 22 17:11:29 check_reload_status 700 Linkup starting em1.101
Mar 22 17:11:29 check_reload_status 700 Linkup starting em1.100
Mar 22 17:11:29 kernel em1.200: link state changed to UP
Mar 22 17:11:29 kernel em1.101: link state changed to UP
Mar 22 17:11:29 kernel em1.100: link state changed to UP
Mar 22 17:11:29 kernel em1: link state changed to UP
Mar 22 17:11:29 check_reload_status 700 Linkup starting em1
Mar 22 17:11:26 check_reload_status 700 Reloading filter
Mar 22 17:11:26 check_reload_status 700 Reloading filter
Mar 22 17:11:26 php-fpm 39491 /rc.linkup: DEVD Ethernet detached event for opt3
Mar 22 17:11:26 php-fpm 74596 /rc.linkup: DEVD Ethernet detached event for opt2
Mar 22 17:11:26 php-fpm 28703 /rc.linkup: DEVD Ethernet detached event for opt1
Mar 22 17:11:26 php-fpm 69246 /rc.linkup: DEVD Ethernet detached event for lan
Mar 22 17:11:25 check_reload_status 700 Linkup starting em1.200
Mar 22 17:11:25 check_reload_status 700 Linkup starting em1.101
Mar 22 17:11:25 check_reload_status 700 Linkup starting em1.100
Mar 22 17:11:25 check_reload_status 700 Linkup starting em1
Mar 22 17:11:25 kernel em1.200: link state changed to DOWN
Mar 22 17:11:25 kernel em1.101: link state changed to DOWN
Mar 22 17:11:25 kernel em1.100: link state changed to DOWN
Mar 22 17:11:25 kernel em1: link state changed to DOWN
Mar 22 17:11:25 php 85589 [pfBlockerNG] filterlog daemon started
Mar 22 17:11:25 tail_pfb 85065 [pfBlockerNG] Firewall Filter Service started
Mar 22 17:11:25 lighttpd_pfb 83328 [pfBlockerNG] DNSBL Webserver started
Mar 22 17:11:25 php_pfb 82575 [pfBlockerNG] filterlog daemon stopped
Mar 22 17:11:25 tail_pfb 82531 [pfBlockerNG] Firewall Filter Service stopped
Mar 22 17:11:25 lighttpd_pfb 81336 [pfBlockerNG] DNSBL Webserver stopped
Mar 22 17:11:24 php-fpm 670 /rc.start_packages: Skipping STARTing packages process because previous/another instance is already running
Mar 22 17:11:23 php-fpm 74596 /rc.start_packages: Skipping STARTing packages process because previous/another instance is already running
Mar 22 17:11:23 ladvd 25994 ladvd 1.1.2 running
Mar 22 17:11:23 check_reload_status 700 Starting packages
Mar 22 17:11:23 php-fpm 28703 /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - -> 10.10.14.1 - Restarting packages.
Mar 22 17:11:23 php-fpm 28703 /rc.newwanip: rc.newwanip called with empty interface.
Mar 22 17:11:23 php-fpm 28703 /rc.newwanip: rc.newwanip: on (IP address: 10.10.14.1) (interface: []) (real interface: ovpns2).
Mar 22 17:11:23 php-fpm 28703 /rc.newwanip: rc.newwanip: Info: starting on ovpns2.
Mar 22 17:11:23 php-fpm 29377 /rc.start_packages: Restarting/Starting all packages.
Mar 22 17:11:22 check_reload_status 700 Starting packages
Mar 22 17:11:22 php-fpm 39491 /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - -> 10.10.13.1 - Restarting packages.
Mar 22 17:11:22 check_reload_status 700 Reloading filter
Mar 22 17:11:22 php-fpm 39491 /rc.newwanip: rc.newwanip called with empty interface.
Mar 22 17:11:22 php-fpm 39491 /rc.newwanip: rc.newwanip: on (IP address: 10.10.13.1) (interface: []) (real interface: ovpns1).
Mar 22 17:11:22 kernel config_aqm Unable to configure flowset, flowset busy!
Mar 22 17:11:22 kernel config_aqm Unable to configure flowset, flowset busy!
Mar 22 17:11:22 php-fpm 39491 /rc.newwanip: rc.newwanip: Info: starting on ovpns1.
Mar 22 17:11:22 check_reload_status 700 rc.newwanip starting ovpns2
Mar 22 17:11:22 check_reload_status 700 Starting packages
Mar 22 17:11:22 php-fpm 74596 /rc.newwanipv6: pfSense package system has detected an IP change or dynamic WAN reconnection - 2111:14xx:3183:2a2d::1 -> 2111:14xx:3183:2a33::1 - Restarting packages.
Mar 22 17:11:22 php-fpm 74596 /rc.newwanipv6: Creating rrd update script
Mar 22 17:11:22 kernel ovpns2: link state changed to UP
Mar 22 17:11:22 php-fpm 74596 OpenVPN PID written: 31365
Mar 22 17:11:22 kernel ovpns2: link state changed to DOWN
Mar 22 17:11:22 php-fpm 74596 OpenVPN terminate old pid: 33830
Mar 22 17:11:21 check_reload_status 700 rc.newwanip starting ovpns1
Mar 22 17:11:21 kernel ovpns1: link state changed to UP
Mar 22 17:11:21 check_reload_status 700 Reloading filter
Mar 22 17:11:21 php-fpm 74596 OpenVPN PID written: 88755
Mar 22 17:11:21 check_reload_status 700 Reloading filter
Mar 22 17:11:21 kernel ovpns1: link state changed to DOWN
Mar 22 17:11:21 php-fpm 74596 OpenVPN terminate old pid: 77669
Mar 22 17:11:21 php-fpm 74596 /rc.newwanipv6: Resyncing OpenVPN instances for interface WAN.
Mar 22 17:11:20 php-fpm 74596 /rc.newwanipv6: phpDynDNS (): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
Mar 22 17:11:17 kernel config_aqm Unable to configure flowset, flowset busy!
Mar 22 17:11:16 php-fpm 670 /rc.dyndns.update: phpDynDNS (): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
Mar 22 17:11:16 php-fpm 74596 /rc.newwanipv6: The command '/sbin/ifconfig em0 inet6 2111:14xx:3183:2a2d::1 delete' returned exit code '1', the output was 'ifconfig: ioctl (SIOCDIFADDR): Can't assign requested address'
Mar 22 17:11:16 check_reload_status 700 Reloading filter
Mar 22 17:11:16 php-fpm 74596 /rc.newwanipv6: Gateway, NONE AVAILABLE
Mar 22 17:11:15 php-fpm 69246 /interfaces.php: Creating rrd update script
Mar 22 17:11:15 check_reload_status 700 Reloading filter
Mar 22 17:11:13 check_reload_status 700 updating dyndns wan
Mar 22 17:11:12 php-fpm 74596 /rc.newwanipv6: The command '/usr/local/sbin/unbound -c /var/unbound/unbound.conf' returned exit code '1', the output was '[1616425872] unbound[21998:0] error: bind: address already in use [1616425872] unbound[21998:0] fatal error: could not open ports'
Mar 22 17:11:11 kernel config_aqm Unable to configure flowset, flowset busy!
Mar 22 17:11:11 kernel config_aqm Unable to configure flowset, flowset busy!
Mar 22 17:11:09 check_reload_status 700 Reloading filter
Mar 22 17:11:09 php-fpm 28703 /rc.newwanip: rc.newwanip: on (IP address: 84.253.242.158) (interface: WAN[wan]) (real interface: em0).
Mar 22 17:11:09 php-fpm 28703 /rc.newwanip: rc.newwanip: Info: starting on em0.
Mar 22 17:11:08 check_reload_status 700 Restarting ipsec tunnels
Mar 22 17:11:08 php-fpm 69246 /interfaces.php: Gateway, NONE AVAILABLE
Mar 22 17:11:08 check_reload_status 700 rc.newwanip starting em0
Mar 22 17:11:07 php-fpm 69246 /interfaces.php: Shutting down Router Advertisment daemon cleanly
Mar 22 17:11:00 check_reload_status 700 Reloading filter
Mar 22 17:11:00 check_reload_status 700 updating dyndns opt1
Mar 22 17:11:00 check_reload_status 700 Syncing firewall
Mar 22 17:11:00 kernel config_aqm Unable to configure flowset, flowset busy!
Mar 22 17:11:00 kernel config_aqm Unable to configure flowset, flowset busy!
Mar 22 17:10:59 check_reload_status 700 Reloading filter
Mar 22 17:10:59 check_reload_status 700 updating dyndns opt2
Mar 22 17:10:59 kernel config_aqm Unable to configure flowset, flowset busy!
Mar 22 17:10:59 kernel config_aqm Unable to configure flowset, flowset busy!
Mar 22 17:10:58 check_reload_status 700 Reloading filter
Mar 22 17:10:58 check_reload_status 700 updating dyndns lan
Mar 22 17:10:57 check_reload_status 700 Reloading filter
Mar 22 17:10:57 check_reload_status 700 updating dyndns opt3
Mar 22 17:10:56 php-fpm 74596 /rc.newwanipv6: The command '/usr/local/sbin/unbound -c /var/unbound/unbound.conf' returned exit code '1', the output was '[1616425856] unbound[43451:0] error: bind: address already in use [1616425856] unbound[43451:0] fatal error: could not open ports'
Mar 22 17:10:56 php-fpm 29377 /rc.linkup: The command '/usr/local/sbin/unbound -c /var/unbound/unbound.conf' returned exit code '1', the output was '[1616425856] unbound[21722:0] error: bind: address already in use [1616425856] unbound[21722:0] fatal error: could not open ports'
Mar 22 17:10:56 php-fpm 36283 /rc.linkup: The command '/usr/local/sbin/unbound -c /var/unbound/unbound.conf' returned exit code '1', the output was '[1616425856] unbound[19055:0] error: bind: address already in use [1616425856] unbound[19055:0] fatal error: could not open ports'
Mar 22 17:10:56 php-fpm 39491 /rc.linkup: The command '/usr/local/sbin/unbound -c /var/unbound/unbound.conf' returned exit code '1', the output was '[1616425856] unbound[844:0] error: bind: address already in use [1616425856] unbound[844:0] fatal error: could not open ports'
Mar 22 17:10:54 php-fpm 74596 /rc.newwanipv6: rc.newwanipv6: on (IP address: 2111:14xx:3183:2a33::1) (interface: wan) (real interface: em0).
Mar 22 17:10:54 php-fpm 74596 /rc.newwanipv6: rc.newwanipv6: Info: starting on em0.
Mar 22 17:10:53 php-fpm 36283 /rc.linkup: Gateway, NONE AVAILABLE
Mar 22 17:10:53 php-fpm 670 /rc.linkup: Gateway, NONE AVAILABLE
Mar 22 17:10:53 check_reload_status 700 Restarting ipsec tunnels
Mar 22 17:10:53 php-fpm 39491 /rc.linkup: Gateway, NONE AVAILABLE
Mar 22 17:10:53 check_reload_status 700 Restarting ipsec tunnels
Mar 22 17:10:53 php-fpm 29377 /rc.linkup: Gateway, NONE AVAILABLE
Mar 22 17:10:53 php-fpm 28703 /rc.newwanipv6: rc.newwanipv6: No IPv6 address found for interface WAN [wan].
Mar 22 17:10:53 php-fpm 28703 /rc.newwanipv6: rc.newwanipv6: Info: starting on em0.
Mar 22 17:10:53 php-fpm 28703 /rc.newwanipv6: rc.newwanipv6: No IPv6 address found for interface WAN [wan].
Mar 22 17:10:53 php-fpm 28703 /rc.newwanipv6: rc.newwanipv6: Info: starting on em0.
Mar 22 17:10:53 php-fpm 670 /rc.linkup: HOTPLUG: Configuring interface opt3
Mar 22 17:10:53 php-fpm 670 /rc.linkup: DEVD Ethernet attached event for opt3
Mar 22 17:10:53 php-fpm 36283 /rc.linkup: HOTPLUG: Configuring interface opt2
Mar 22 17:10:53 php-fpm 36283 /rc.linkup: DEVD Ethernet attached event for opt2
Mar 22 17:10:53 php-fpm 29377 /rc.linkup: HOTPLUG: Configuring interface opt1
Mar 22 17:10:53 php-fpm 29377 /rc.linkup: DEVD Ethernet attached event for opt1
Mar 22 17:10:53 php-fpm 39491 /rc.linkup: HOTPLUG: Configuring interface lan
Mar 22 17:10:53 php-fpm 39491 /rc.linkup: DEVD Ethernet attached event for lan
Mar 22 17:10:52 check_reload_status 700 Linkup starting em1.200
Mar 22 17:10:52 check_reload_status 700 Linkup starting em1.101
Mar 22 17:10:52 check_reload_status 700 Linkup starting em1.100
Mar 22 17:10:52 kernel em1.200: link state changed to UP
Mar 22 17:10:52 kernel em1.101: link state changed to UP
Mar 22 17:10:52 kernel em1.100: link state changed to UP
Mar 22 17:10:52 kernel em1: link state changed to UP
Mar 22 17:10:52 check_reload_status 700 Linkup starting em1 -
@fragged I don't know if you've tested the IPV6 gateway with gateway monitoring OFF or set with a monitoring address (such as google.com or their IPV6 DNS for example)- it's a known issue with 2.5 and at least some ISPs. So a quick test of that would be just go into SYSTEM/ROUTING and turn IPV6 gateway monitoring OFF to see what happens. You can then try supplying a monitoring IP and turning it on if it works.
I was lucky in that IPV6 didn't work if monitoring was ON with my previous ISP so I already had it off. I didn't bother to turn it on when I switched ISPs, so didn't have that problem when I upgraded to 2.5. I have since enabled it with google's IPV6 as the monitoring IP. So it's worth a try if you haven't tried that yet. -
I felt for sure this was the problem I was having.
But I have struggled in a seemingly identical way.I am on 2.5.0 (also now running the 2.5.1-RC but seems the same)
I tried disabling the gateway monitor as noted above.I have PPPoE WAN which requests a 48 prefix from my ISP. (Giganet in the UK)
Tried other prefix sizes and various option on the WAN side i.e. prefix hints only request prefix etc.My Wan has a proper IPv4 and IPv6 IP.
I can always ping ipv6.google.com from the router ping page.However My LAN which is set to track interface WAN never gets an IPv6 IP assigned to it.
Consequently I can never get any ipv6 assignements to my LAN.
Any debug hints?
Cheers,
Tom
-
Hello,
I attempted to upgrade to 2.5.1 last night but I believe that I encountered the problem above. I couldn't get IPv6 addresses or routing on any of my WAN connections. I tried troubleshooting for several hours but had to give up and rollback to 2.4.5_1.
From the messages above and in the bug tracker, it seems like this might be fixed but in a version that isn't released yet. Is that accurate? (I've been reading from https://redmine.pfsense.org/issues/11454 and they mention installing 2.5.1 plus a patch..) Is there any way to know when this should be published and I could re-attempt upgrading?
Thanks,
Ryan -
I am running version 2.5.2-RELEASE and my situation is the same. Here goes some details:
- Enable IPv6 w/ DHCPv6 on WAN (IA_PD only and prefix /56 as ISP instructions)
- dhcp.log file shows PD being received but no address or further delegations to the tracking interfaces
- WAN does not assign IPv6 for itself
- Gateway shows the fe80:: link local address as gateway.
In this scenario, if I manually set an IP to WAN interface (ifconfig igb0 inet6 <ipv6>), the link local (fe80::) disappears (odd) but I get IPv6 connectivity. However, all tracking interfaces are not assigned. Also, if I manually add an IP to them, it seems to work but interface become unresponsive.
-
@jrbenito said in IPv6 broken beyond pfSense after 2.5 upgrade:
WAN does not assign IPv6 for itself
Do you have Request only an IPv6 prefix on the WAN page select? Regardless, a WAN address other than link local is not needed.
Gateway shows the fe80:: link local address as gateway
Entirely normal. Link local addresses are often used for routing. You'll see the same thing on the LAN side.
-
Thanks for clarification.
A couple of hours into debugging and I was able to have it partially working. IPv6 is working as expect. The problem is being caused by LAN side VLANs. I have no idea why, but when any of the VLANs (I have 4) is marked to track interface WAN, it starts flapping.
If I keep doing reload the interface page or issuing
ifconfig
in a ssh session, I can see the interfac asactive
(with IPv4 and IPv6) and a split of second later asno carrier
. Looking to the physical port at the router, LEDs goes off, come back on for about 2 or 3 seconds, goes off again. Because this flapping behavior, it seems that there is no connection (and there is no connection on that port, however on another port I can see everything ok).My VLANs are set all together on interface
igb3
, hence I haveigb3.20
,igb3.30
,igb3.40
,igb3.199
. No matter if I set all to track IPv6 from WAN or only one, once one is set to track, all of them goes tono carrier
state (because the issue is probably onigb3
itself). I have no clue where to look for more information on this.Ah, and yes, IPv6 on wan is set to send
IA_PD
only and now I understood that way wouter won't have an/64
for itself. IPv6 on WAN was not the problem, but the IPv6 on LAN. At least now I know it.BRs,
José -
Check the IPv6 Prefix ID. It must be unique for each interface or VLAN. I get a /56 prefix from my ISP, so I can choose anything in the range 0-ff.
-
@jknott said in IPv6 broken beyond pfSense after 2.5 upgrade:
Check the IPv6 Prefix ID. It must be unique for each interface or VLAN. I get a /56 prefix from my ISP, so I can choose anything in the range 0-ff.
It is same here, I receive a /56 prefix from my ISP. The IPv6 Prefix ID is good call and I checked it, no luck. The flapping starts with even only on VLAN using the IPv6 and, btw, it happens if I set IPv6 manually (static) anyway. I need to dig deeper into it. I will need a length ethernet cable to wire my computer to the spare port of the router so I have connection even when ports start flapping. Otherwise I have to walk down the comm closet with laptop on hand to reset config and have internet/connection back to the main computer.
-
Just for documentation purposes, what is happening on my end is:
-> when set any vlan interface to track WAN, all vlan interfaces on the same physical ethernet start to flapping.
-> longing into the router through ssh (another ethernet) and verifying
/var/etc/dhcp6c_wan.conf
shows that configuration was not written to there (even after applying it)I guess that at this point, pfsense is trying to track wan interface without proper configuration on DHCP and it causes the flapping.
-> I reboot gracefully through ssh session and after it, DHCP config was properly written and IPv6 working on VLAN.
If I don't have the spare ethernet port to connect and control I would have lost a way to gracefully reboot, I don't know if a hardreset would have same effect. Anyway, I now have it working.
Thanks for your insights.
Cheers
-
@mrsunfire I'm running 2.5.2 and this is still an issue. Disable the 2nd wan and ipv6 just works.
-
@mrsunfire I just upgraded to 2.6 and this is STILL an issue....