PPPoE WAN up and IP but No Internet

on3moda

So I thought I would type this all up on my mobile phone after days of wrestling with this.

I have an existing config that was working using PPPoE and VLAN tagging. However I want to migrate firewalls so I needed to offload the tagging onto something else because the new firewall doesn't support a VLAN subininterface on PPPoE ports.

So I configured my edge switch and pfsense didn't like any configuration for the WAN to come up. I noticed that interface changes didn't like to work after a period of time so I rebooted and wrestled some more. Long story short I switched over to the replacement firewall to validate my VLAN setting were correct before I wrestled more with PFSense. Got my edge switch working with my supported firewall. I was able to ping the usual suspect 8.8.8.8.

I switched over back to PFSense putting the edge switch properly configured back in line so we have ONT -> Edge Switch -> PFSense Wan. After that I removed all VLAN tagging from the config. I restarted PFSense however I noticed it hung on "do you want to assign VLANs?" So I hit no and it was in a loop till I hit Y then N. PfSense booted to find all my interfaces and their names deleted however I could add them again. Instead of trying to remember how everything was setup I reverted back to the old config to get everything back from an interface perspective and then swung over to PPPoE again. And that's where we are today. Everything can ping everything locally and even the WAN gateway from both internally and externally. However nothing gets to the next hop after that on both sides. From external I can't ping my WAN IP that was assigned. From a PC on the LAN I can't trace route past the 192.168.1.1. Destination host unreachable. Trying to use the ping tools from the WAN interface I can ping 192.168.1.1 and from the lan interface I can ping my carrier gateway.

The last puzzle piece was when wrestling with the interfaces on PFSense originally I deleted the gateway when trying to switch between static and DHCP to troubleshoot why my WAN wasn't coming up originally.

It seems the gateway gets added automatically on the WAN interface.

Next I think I will check to see if my default route got deleted when deleting the gateway and see if it didn't get added back to the routing table however I thought I would start this in case anyone else has or will run I to this issue.

on3moda

Fixed it. When mucking around with the static and DHCP. When deleting the gateway it removed the default gateway config and selected automatic.

TLDR; Lessons learned when placing a edge switch to offload the VLAN tagging from PPPoE and VLAN tagging on the WAN interface:

Configure the WAN interface to PPPoE only.
Remove the VLAN
Reboot. PFSense seems to pull the PPPoE settings in at reboots.

Leaving this in hopes it can help someone else and save time.