How to get pfSense WAN to accept VLAN 0

[Jarhead]

@stephenw10 said in How to get pfSense WAN to accept VLAN 0:

Hmm, looks like they are using identical drivers.

Did you test with em or igb specifically? Though they are the same driver.... except if they're not!

I use a Protectli Vault with em interfaces.
Plug it right into the ONT and it works.

[Jarhead]

@c45p32 said in How to get pfSense WAN to accept VLAN 0:

I loaded opnsense onto that same backup router and I can’t get it to work on em0. I even went into the attic, got the ISP router to formally release the IP and still no go. So not sure how people are getting it to work on opnsense. I’m on Frontier fiber College Station, Texas.

What version of Opnsense?
I see the latest has all kinds of problems.
Try 22.7. Works with em drivers. Plug drirectly into the ONT.

[stephenw10]

If you run a pcap can you see the vlan0 tagged dhcp responses?

[michaellacroix]

@stephenw10

I just tested the latest pfsense dev image using em0 as my wan interface and it was able to pickup an IP address. I'm using frontier fiber and was directly connected to the ONT.
This is hopeful...

[stephenw10]

Yes that is useful.

It would be very interesting to see if those DHCP replies appear in a pcap. I wasn't able to see vlan0 tagged pings but perhaps some off-loading allows the udp dhcp packets....

[stephenw10]

Ok, after a bunch of testing it appears the difference here is in OPN pretty much all hardware offloading is disabled by default and we have most things enabled. Including VLAN Hardware Filtering which appears to be broken for VLAN0 in the iflib e1000 driver.

Manually disabling it allowed my test boxes to pull a dhcp lease from a server that is priority tagging replies:

ifconfig em2 -vlanhwfilter

01:04:04.652627 00:90:7f:87:dc:77 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328)
    0.0.0.0.68 > 255.255.255.255.67: [udp sum ok] BOOTP/DHCP, Request from 00:90:7f:87:dc:77, length 300, xid 0x7b209734, Flags [none] (0x0000)
01:04:04.654209 90:ec:77:1f:8a:62 > 00:90:7f:87:dc:77, ethertype 802.1Q (0x8100), length 346: vlan 0, p 4, ethertype IPv4, (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328)
    10.14.0.1.67 > 10.14.0.12.68: [udp sum ok] BOOTP/DHCP, Reply, length 300, xid 0x7b209734, Flags [none] (0x0000)

None of my tests allowed any client to see the servers replies with vlan hardware filtering still enabled. However I could imagine there might be em or igb NICs that don't support VLAN hardware filtering that would not be affected. That could explain the conflicting results we see here.
Anyway if you're seeing this issue try disabling VLAN_HWFILTER.

Steve

[c45p32]

@stephenw10 Well holy crap batman, that worked. With all default settings:
Hardware Checksum Offloading: enable
Disable hardware TCP segmentation offload: disabled
Disable hardware large receive offload: disabled
and just setting ifconfig em0 -vlanhwfilter I was able to pull an IP address and surf the internet.

[Jarhead]

@c45p32
Are you saying this works on 2.6?

[c45p32]

@jarhead No, I couldn't get it to work on 2.6.0 (22.05) with only the driver and no script on igb0. My testing above was done on my back-up router with em0 and the lastest dev 2.7

You can get 2.6.0 (22.05) to work with the following instructions:
I've also attached my actual script that I keep in /root and I create a earlyshellcmd that points to /root/./vlanzero.sh
vlanzero.zip

1. Use this post (https://github.com/MonkWho/pfatt/issues/67#issuecomment-1043358822) to install the driver for igb0.
2. Reboot
3. Download vlan script from here: [vlanzero.sh.tar.gz] (https://forum.netgate.com/assets/uploads/files/1614644825333-vlanzero.sh.tar.gz) and unzip it.
4. Edit vlanzero.sh and change ONT_IF='xx0' to your WAN assignment, usually igb0. You can find this under Interfaces > Assignments
5. In the same file change ONT_ETHER_ADDR='xx:xx:xx:xx:xx:xx' to your mac address for your WAN. You can usually see this under Status > Interfaces Under the WAN interface heading. Save and close the file.
6. Goto Diagnostics > Command Prompt and Upload the vlanzero.sh file you just edited.
7. ssh into pfSense
8. type the following commands:
   mv /tmp/vlanzero.sh /root/vlanzero.sh
   chmod +x vlanzero.sh
   ./vlanzero.sh
9. Go to Interface > Assignments and change your WAN to ngeth0 click save and apply
10. Make sure you have the package shellcmd installed. Go to Services > Shellcmd
11. Click "Add" for Command put "/root/./vlanzero.sh" without quotes, shellcmd type set it to "earlyshellcmd", and then name it what ever you want. Save
12. Reboot and you should be working.

Note you need to either release your IP using the Frontier box, or just unplug your ONT for a few min. I usually just unplug easier.

[stephenw10]

Yeah, it won't work in 2.6/22.05 because it doesn't have the vlan0 bpf patch for dhclient. 2.7/22.11 does.

[Jarhead]

@stephenw10
So Shouldn't 2.7 "just work" without needing to do anything?

[stephenw10]

It would for any NIC except an e1000. At least any that I've tested.
For em/igb you need to disable vlan_hwfilter. Currently.

Steve

[michaellacroix]

@stephenw10

Great find!

[stephenw10]

What would confirm that as the source of confusion would be if the cases where two seemingly identical boxes had different behaviour actually had different NIC chips and only one supported vlan_hwfilter.

Steve

[michaellacroix]

@stephenw10

Hi Stephen, does netgate make any type of release notes for the daily snapshots? As they move along I'm curious as to what issues were addressed. Thanks

[stephenw10]

Not beyond the commit logs in github. At least not publicly:
https://github.com/pfsense/pfsense/commits/master

Steve

[Schwiing]

Just got Frontier Fiber installed today (2Gig Service). Hooked the FRX523 to my PFsense router and it got an IP immediately. No need to strip the tag. I did have a few weird speed tests (940/920, 200/600) but after that it's been almost consistent with 2.5/2.5G.

Not that this is extremely useful here, but it's a data point nonetheless.

[stephenw10]

In 2.6 or 2.7-dev?

[Schwiing]

@stephenw10 in 22.05. My thought is just that Frontier isn't using Vlan0 here. But I'm also not entirely sure one way or the other aside from the result. Is there a way for me to check?

[michaellacroix]

@schwiing

I took a pcap a couple of weeks ago and the vlan 0 tag was there. I also have frontier in CT. Are you familiar with wireshark?