Re: PFsense random loss of WAN gateway

I just wanted to add my thanks!

I have a Telia Fiber connection and it would lose WAN every six hours. Turns out that the Telia DHCP server only allows a limited number of renewals after which it demands a broadcast again.

The above option to always broadcast works fine.

It took me several month to find this solution! Thanks again!

@johnpoz @stephenw10

The problem with pfBlockerNG and config saves is even bigger as that many MANY configuration changes are all SYNCED to a CARP member triggering a HUGE number of unnecessary reloads and changes on the secondary node. And as pfBNG doesn't really sync the lists but the config only, the second node still has to run its own instance of pfB and do the whole download and install of the lists AGAIN, so you nearly have double the config changes to the standby node. Also in a bigger setup you have to either completely disable the sync because of this or you have to time the standby node down to do updates e.g. only daily or all 12h as otherwise you get hit with the sync job that triggers a reload of MANY services of the standby node, then have the node perform its own pfB download and saving configs. So config history is completely broken and unusable in a cluster where pfB is enabled as you won't see anything older then a day or two with that many checkpoints. Also the sync adds even more on the standby AND triggers high load and temporary RPC/sync unavailability as the node gets simply swamped by syncs and reloads (talking about a big node here with many VPNs, big ruleset etc. - datacenter firewall). That's a really big minus of pfB currently. I already mentioned that to BBcan/Tony several times but never came to tackle down the issue (with various others concerning a CARP setup like the interface creation in DNSBL mode etc.)

Cheers

Yup, but he said he still didn't get access when the DMZ mode was disabled so pfSense gets a private WAN IP. Which is unexpected.

Trunk is actually a Cisco term but is commonly used to refer to a link that carries more than one VLAN. A trunk can carry tagged and untagged traffic but in order to keep them separated only one VLAN can be untagged.

It's probably a routing conflict. The laptop has an IP and gateway in the LAN subnet and that must be routed outside the tunnel but at the same time the WG server is sending it routes to the LAN subnet via the tunnel.
I would expect to see some routing error logged somewhere.

Putting the WIFI on a different subnet would workaround it.

You could also block access to the WG server from the LAN so the tunnel cannot connect when you're on that subnet.

Steve

The installer image contains a FAT32 partition for exactly this purpose. See:
https://docs.netgate.com/pfsense/en/latest/backup/restore-during-install.html#restore-configuration-from-usb-during-install

Steve

Ok I created a bug for it but that may get changed to a feature request because that is the expected behaviour in the code. It's not what I would expect as a user though.

https://redmine.pfsense.org/issues/15228

Yup, usually you won't notice the difference. We have seen some situations where it is required though. Others where the throughput can be significantly increased by disabling it.
It's worth testing disabling it if you are not seeing the expected throughput and have local access to revert that change if required.

Steve

Almost always because the system clock changed so the data becomes invalid.

Many modules that are shown as Intel compatible will work there. Those listed are just what we've tested locally.

The most common problem people hit is trying to use a 10/1G module at 1G. That often requires setting a 1G fixed link speed and some modules don't expose that option. DAC cables usually don't offer that.

Steve

Hmm, still can't replicate it. It must be something in your config somehow. Are you able to test it with a default config? Or upload your config to us to check?

@JKnott said in Netgate 6100 SFP+ connection error rate of 0.0055%. Should I be worried?:

@ChrisJenk said in Netgate 6100 SFP+ connection error rate of 0.0055%. Should I be worried?:

Name Mtu Network Address Ipkts Ierrs Idrop Opkts Oerrs Coll
ix1 9000 <Link#6> 90:ec:77:7f:c9:d5 31488071 1452 0 74590982 0 0
ix1 - fe80::%ix1/64 fe80::92ec:77ff:fe7f:c9d5%ix1 1060 - - 98 - -

Is that your WAN or LAN? While there's no problem with jumbo frames on the LAN, assuming other devices can handle them, you shouldn't be sending them to the WAN. PfSense should be sending ICMP too big messages when a jumbo frame tries to leave your LAN. You shouldn't be using jumbo frames on the WAN side, with the possible exception of if you're on Internet2.

It's my LAN and I do use Jumbo frames on that (carefully). However, the jumbo frames are restricted to two VLANs neither of which are configured on the NetGate so those frames should actually never reach the unit. I think that MTU is a hangover from an older config; I will set it back to the default.

However, in my current setup the LAN is now ix0 and it has an MTU of 1500.

Name Mtu Network Address Ipkts Ierrs Idrop Opkts **Oerrs** Coll ix0 1500 <Link#5> 90:ec:77:7f:c9:d4 17034814 **387** 0 20616901 0 0 ix0 - fe80::%ix0/64 fe80::92ec:77ff:fe7f:c9d4%ix0 4078 - - 18875 - - ix0 - 10.0.200.0/24 router 8993 - - 22104 - - ix0 - fd00::/64 router 8936 - - 9272 - - ix0 - xxxxxxxxx::/64 router.xxxxxxxxxxxx 0 - - 25628 - - ix0 - yyyyyyyyy::/64 yyyyyyyyyyyyyy::1 0 - - 42 - -

Hmm, none jumps out there but I would try removing them one at a time if you're able.

Ok, what does the Status > Interfaces page show?

What sort of connection does your ISP provide? Is it just DHCP?

You may need to reboot the modem if it's locked to the MAC address of the Asus router.

Steve

@Popolou This is from my Windows laptop, I don't have the native apps installed. But, it did prompt me to test with another Windows device that hadn't been moved to the pfSense firewall yet. That did seem to run much better, so it may well be a device issue, rather than the firewall.

Strangely though, the troublesome laptop does work very smoothly when it's on a raw internet connection.

Thanks to you both for your answers.

Thanks. I reinstalled yesterday evening. Seems to be booting fine now after the fresh installation. Just have to restore the backup and I'm good.

Thank you so much for the input though!

@stephenw10 OK I just deleted them all and created new ones - now 7 of them are green and 1 is red (100% packet loss) - while this is less than ideal we can live with that in the short term. What I'll do is open a support call with the provider that these tunnels go to and see what they have to say. If we identify issues with pfSense I will likely open up a new thread on this forum and link to it from here.

As for the LAN drops, I think that problem is solved, we've had 2 whole working days now and its been clean. As mentioned we did rather too many changes at once to isolate which one of these might of been the solution, but thems the breaks.

Of course your help has been invaluable so thank you very much for that.

@stephenw10
Unfortunately the NUT service would not restart after all this. I had to delete the package and reinstall. I think I should just wait for the officially updated package (hopefully to be released soon!)

Hmm well I know nothing about them but others may.

Thank you @stephenw10

Turned out to be a config issue on my site which I missed. (forgot to add the sync states checkbox on the secondary node)