pfSense CE 2.5.1 RC update/upgrade

[pete]

2nd of April, 2021 update(s).

Updated twice yesterday. With each update lost my main WAN IP6 connection. I did as before which was disabling all IP6 on WAN and LAN, rebooting, re-enabling IP6, rebooting and all was well.

This morning updated to:
2.5.1.r.20210402.0300

Before update disabled WAN2 IP6 configuration. WAN2 is a CPE on T-Mobile. I am using a LAN interface on the device which is in a DMZ. I have it configured IP6 wise identical to the Comcast connection on main WAN.

With the updates to PFSense this WAN2 connection always stayed up IP4/IP6; but not the XFinity IP6 connection.

This time before the update disabled IP6 on WAN2.

I did the update and it worked fine and I never lost the IP6 connectivity on main WAN.

I do not know why having IP6 running on the failover CPE WAN2 connection would prevent IP6 from running on WAN connection when doing an update / rebooting.

So update went fine this time after reboot and after disabling IP6 on the WAN2 CPE connection.

All appears fine after update. (with WAN2 IP6 off)

No updates have affected the NTP server. Working fine here.
I did move the GPS from the attic talking serially to the basement installed PFSense box to the basement with the GPS antenna near a SW windows with a cover on it 6 feet below ground level.

Configured WAN2 - TMobile CPE with IP6.
CPE ==> LAN interface ==> PFSense WAN2 interface.
I cannot bridge the interface. I can shut off firewall pieces and directly connect to WAN port but then I cannot get to the CPE interface on the LAN side.
The CPE is an el cheapo Yeacomm 4G LTE CPE Router on Amazon.

Here is a direct connection to the CPE interface GUI.

[pete]

Update 2147 c time 2nd April, 2021

XFinity has been up and down twice in the last 8 hours.

The default gateway IPv6 icon is no longer present.

[pete]

3rd of April, 2021

Upgraded to 2.5.1.r.20210403.0300

LAN device (laptop) would not pass IP6 test on internet after update reboot.

The assumption is that RA wasn't working?

Both IP4 and IP6 gateway icons (globes) came back.

After 2-3 hours of not seeing IP6 on my laptop removed all references to IP6 on WAN and LAN interfaces, rebooted, re-entered all IP6 stuff, rebooted and all was well.

[pete]

Update 4th of April, 2021 to:

2.5.1.r.20210404.0300

After reboot all functions continued to work; specfically IP6 came back fine and IP6 internet test worked fine.

Have kept the WAN #2 IP6 pieces disabled for the last couple of days.

Note here I have disabled IP6 on WAN2 for time bean as for whatever reason that seemed to be an issue after updates. I am not sure why this would cause a loss of IP6 on the primary WAN interface after an update / reboot.

The CPE is router:

I am attaching to IP4 via a NAT'd DHCP address on the LAN side of the CPE.

I have the IP6 configuration for TMobile CPE same as the Comcast connection which has worked for me:

IPv6 Configuration Type: DHCP6
Use IPv4 connectivity as parent interface - checked
DHCPv6 Prefix Delegation size - /60
Send IPv6 prefix hint

For failover only using IP4 on WAN #1 and WAN #2.

I do see 4 gateways in the router configuration:

1 - WAN IP4 gateway
2 - WAN IP6 gateway
3 - WAN #2 IP4 gateway
4 - WAN #2 IP6 gateway

If I enable IP6 on WAN #2 a reboot causes WAN #1 IP6 to disconnect and not come back unless I remove IP6 from all of the interfaces, reboot, reinstall IP6 and reboot.

T-Mobile gives me a 26XX address on the WAN #2 link. PFSense is connected to a LAN link on the CPE. So I get a private IP4 LAN address and a 26XX address on same LAN link but that is the same as the CPE WAN link.

I cannot bridge the interface on the CPE. I can though shut off the router piece and directly connect to the WAN interface but then I do not get to see the GUI on the CPE. So if I connect my laptop to the LAN port or WLAN port of the CPE I do get internet connectivity with an IP4 and IP6 T-Mobile address.

[jimp]

There are known issues with having multiple DHCPv6 WANs. It's not technically supported currently. It may have partially worked in the past by chance alone, but it's not viable at the moment.

https://redmine.pfsense.org/issues/6880

[pete]

@jimp

Thank you.

[pete]

5th of April, 2021

Updated to:

2.5.1.r.20210405.0300

Left WAN#2 IP6 off, updated, rebooted and all is well.

[encyklopedi]

Please write ipv4 and ipv6, not ip4 and ip6.

[pete]

Will do.

Update 2200c 6th of April, 2021 to:

2.5.1.r.20210406.1302

Left failover WAN 2 IPv6 settings off when updating.

Noticed that I do not see Dashboard status on IPSec server when active tunnel is present.

[pete]

Update:
8th of April, 2021 - 1200c
2.5.1.r.20210406.1302

Looks like XFinity / Comcast rebooted my SB 6190 modem this morning...

Software version on SB6190 ==> 9.1.103AA65L

Uptime: 0 d: 1 h: 42 m

PFsense lost WAN #1 connection around the same time as modem being rebooted by XFinity.

Failed over to WAN #2 at this time.

PFSense didn't automagically reconnect to the modem (WAN #1) after XFinity restarted Modem as Dashboard indicated no WAN IPv4 or IPv6 address on the WAN interface.

I had to manually reboot PFSense to get it to connect to the SB6190.

[Gertjan]

@pete said in pfSense CE 2.5.1 RC update/upgrade:

I had to manually reboot PFSense

Not optimal.
Better - not still not best - would be Status > Interfaces and try a Release and re connect for the concerning WAN interface.
This should give a kick to the DHCP client.
If that works, you know that the issue is that "DHCP client" didn't manage to get a new WAN IP on it's own. Check the DHCP logs if it was doing so.
When the "DHCP client" issue is confirmed, goto Interface > WAN and check DHCP Client Configuration > Advanced Configuration
Now you have some extra options that you can be used to 'back off' when the modem goes down.

[pete]

@gertjan

Thank you.

It happened again around 3AM here and the IPv4/IPv6 interface did not come back to life.

Will test a release and reapply of the DHCP WAN client.

The release and reapply of the WAN IPv4/IPv6 side did not work.

Logs show:

Apr 9 09:42:33 dhclient 24862 DHCPDISCOVER on em1 to 255.255.255.255 port 67 interval 12
Apr 9 09:42:28 dhclient 24862 DHCPDISCOVER on em1 to 255.255.255.255 port 67 interval 5
Apr 9 09:42:26 dhclient 24862 DHCPDISCOVER on em1 to 255.255.255.255 port 67 interval 2
Apr 9 09:42:25 dhclient 24862 DHCPDISCOVER on em1 to 255.255.255.255 port 67 interval 1
Apr 9 09:42:24 dhclient 24862 DHCPDISCOVER on em1 to 255.255.255.255 port 67 interval 1
Apr 9 09:42:23 dhclient 24862 DHCPDISCOVER on em1 to 255.255.255.255 port 67 interval 1
Apr 9 09:42:17 dhclient 24862 DHCPREQUEST on em1 to 255.255.255.255 port 67
Apr 9 09:42:12 dhclient 24862 DHCPREQUEST on em1 to 255.255.255.255 port 67
Apr 9 09:42:12 dhclient 24862 DHCPREQUEST on em1 to 255.255.255.255 port 67
Apr 9 09:42:12 dhclient 44358 PREINIT
Apr 9 09:42:08 dhclient 24862 Cannot open or create pidfile: No such file or directory
Apr 9 09:42:06 dhcp6c 999 exiting
Apr 9 09:42:05 dhclient 97160 exiting.
Apr 9 09:42:05 dhclient 97160 connection closed
Apr 9 09:42:05 dhcp6c 999 Sending Solicit
Apr 9 09:42:04 dhcpd 27848 Server starting service.

Watching the Dashboard seeing an IPv4 come up then go away. Nothing seen on the IPv6 interface in the Dashboard.

Looks like it is trying. What changes can I make to the DHCP advanced section of the WAN interface to maybe fix this?

Rebooted PFSense and all is well for time bean.

This issue appears to have cropped up with latest upgrade.

2.5.1.r.20210406.1302

[Gertjan]

As you can see, the dhclient stops. Probably because their is no reason for it to keep running as the WAN interface went down. That is, the other end - the modem - was rebooting.
As usual, it takes a small eternity for it to come back on-line.
So, dhclient is launched again.

As the last lease wasn't expired, a couple of DHCPREQUEST are fired away.
The modem doesn't reply.
After 3 DHCPREQUEST, dhclient start to obtain a new lease :
After 0,1 second, a DHCPDISCOVER is fired : no reply.
0,2 seconds another DHCPDISCOVER is fired : no reply.
after 0,5 seconds another DHCPDISCOVER is fired : no reply.
after 1 second another DHCPDISCOVER is fired : no reply.
after 2 second another DHCPDISCOVER is fired : no reply.
after 5 second another DHCPDISCOVER is fired : no reply.
after 10 second another DHCPDISCOVER is fired : no reply.
Etc.

yes, we all love these modems.

As said, with the DHCP log measure the time the modem decides to reply, and add 30 seconds margin.

Now, as said above, use the DHCP Client Configuration > Advanced Configuration and use these option to introduce an initial hold-off time.

Or get a modem that is a bit faster and/or actually replies after it started up.

If IPv4 is down, then IPv6 is also down, that's normal.

[pete]

@gertjan

Thank you.

The modem is an Arris Surfboard 6190.

While I do see an uptime of a few hours I do not see a reboot of the modem in the modem logs. The reboot of the modem is typically around a minute or so.

I have been considering updating the owned modem to a new Arris Surfboard SB8200.

Guessing here a PFSense / Interfaces / WAN / advanced DHCP configuration setting to Protocal timing:

The options are timeout, retry, select timeout, reboot, back off or initial interval.

Which option should I use? and for how long?

See the first option default timeout to be 60 seconds. Should I change this to 120 seconds?

[pete]

10th of April, 2021

2.5.1.r.20210406.1302

No reset or reboot of the Modem and all is well this morning.

Looking at the Surfboard logs looks more like it was a disconnect rather an a reboot of the modem.

Looking at the upstream / downstream signals see that here signals are OK but almost marginal and in house #2 they are better. Here and house #2 only use Internet and use DTV Satellite for TV (and OTA). Here have connected a thin RG6 cable to the ingress (no splitters) to go to my Leviton 42" media can. Might do a test here replacing that cable with standard RG6 cable.

In the old house XFinity replaced the underground RG6 cable with a newer much thicker cable. Only they left it on the lawn for 2 months before I made a big deal of it and then they buried it.

I have not upgraded PFSense on house #2 yet...and not running IPv6 there...will do that...

House #2
Current Base System ==> 2.4.5_1
Latest Base System ==> 2.5.0

Gonna ask XFinity for a truck roll to house #1 to check signals from the outside box then the ingress to the house...

[pete]

11th of April, 2021

Installed a PCT active return amp on the coaxial cable today.

Modem downstream channel power is a bit high - going to switch to using Forward Path Attenuator this week. I read somewhere that this might not work as modem will self correct after installation.

Have not changed advance settings mentioned above.

Saw this almost failover and return to normal earlier this morning.

Notifications in this message: 3

1:15:49 MONITOR: WAN_DHCP is available now, adding to routing group Failover
8.8.8.8|w.x.y.z|WAN_DHCP|11.63ms|0.401ms|0.0%|online|none
1:18:24 MONITOR: WAN_DHCP has packet loss, omitting from routing group Failover
8.8.8.8|w.x.y.z|WAN_DHCP|17.221ms|5.948ms|44%|down|highloss
1:18:26 MONITOR: WAN_DHCP is available now, adding to routing group Failover
8.8.8.8|w.x.y.z|WAN_DHCP|13.756ms|5.308ms|20%|online|loss

[pete]

13th of April, 2021

Back to monitoring the WAN link. It is not resetting any more like before. The issues below appeared when I was streaming AOD last night. Working with XFinity to maybe correct these issues.

Mon 4/12/2021

18:59:30 MONITOR: WAN_DHCP has packet loss, omitting from routing group Failover

8.8.8.8|24.XX.XX.XX|WAN_DHCP|13.267ms|1.276ms|33%|down|highloss

18:59:46 MONITOR: WAN_DHCP is available now, adding to routing group Failover

8.8.8.8|24.XX.XX.XX|WAN_DHCP|15.906ms|5.549ms|20%|online|loss

18:59:57 MONITOR: WAN_DHCP has packet loss, omitting from routing group Failover

8.8.8.8|24.XX.XX.XX|WAN_DHCP|16.688ms|7.034ms|25%|down|highloss

19:00:01 MONITOR: WAN_DHCP is available now, adding to routing group Failover

8.8.8.8|24.XX.XX.XX|WAN_DHCP|15.823ms|5.291ms|14%|online|loss

19:00:04 MONITOR: WAN_DHCP has packet loss, omitting from routing group Failover

8.8.8.8|24.XX.XX.XX|WAN_DHCP|27.761ms|31.182ms|30%|down|highloss

19:00:12 MONITOR: WAN_DHCP is available now, adding to routing group Failover

8.8.8.8|24.XX.XX.XX|WAN_DHCP|17.575ms|7.674ms|10%|online|none

19:13:36 MONITOR: WAN_DHCP has packet loss, omitting from routing group Failover

8.8.8.8|24.XX.XX.XX|WAN_DHCP|18.989ms|11.389ms|33%|down|highloss

19:13:38 MONITOR: WAN_DHCP is available now, adding to routing group Failover

8.8.8.8|24.XX.XX.XX|WAN_DHCP|17.675ms|10.165ms|20%|online|loss

21:25:23 MONITOR: WAN_DHCP has packet loss, omitting from routing group Failover

8.8.8.8|24.XX.XX.XX|WAN_DHCP|15.089ms|5.734ms|21%|down|highloss

21:27:26 MONITOR: WAN_DHCP is available now, adding to routing group Failover

8.8.8.8|24.XX.XX.XX|WAN_DHCP|14.49ms|5.424ms|16%|online|loss

All PFSense functions appear to work OK except for not seeing VPN status on Dashboard.