How to get pfSense WAN to accept VLAN 0

stephenw10

If you run a pcap can you see the vlan0 tagged dhcp responses?

michaellacroix

I just tested the latest pfsense dev image using em0 as my wan interface and it was able to pickup an IP address. I'm using frontier fiber and was directly connected to the ONT.
This is hopeful...

stephenw10

Yes that is useful.

It would be very interesting to see if those DHCP replies appear in a pcap. I wasn't able to see vlan0 tagged pings but perhaps some off-loading allows the udp dhcp packets....

stephenw10

Ok, after a bunch of testing it appears the difference here is in OPN pretty much all hardware offloading is disabled by default and we have most things enabled. Including VLAN Hardware Filtering which appears to be broken for VLAN0 in the iflib e1000 driver.

Manually disabling it allowed my test boxes to pull a dhcp lease from a server that is priority tagging replies:

ifconfig em2 -vlanhwfilter

01:04:04.652627 00:90:7f:87:dc:77 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328)
    0.0.0.0.68 > 255.255.255.255.67: [udp sum ok] BOOTP/DHCP, Request from 00:90:7f:87:dc:77, length 300, xid 0x7b209734, Flags [none] (0x0000)
01:04:04.654209 90:ec:77:1f:8a:62 > 00:90:7f:87:dc:77, ethertype 802.1Q (0x8100), length 346: vlan 0, p 4, ethertype IPv4, (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328)
    10.14.0.1.67 > 10.14.0.12.68: [udp sum ok] BOOTP/DHCP, Reply, length 300, xid 0x7b209734, Flags [none] (0x0000)

None of my tests allowed any client to see the servers replies with vlan hardware filtering still enabled. However I could imagine there might be em or igb NICs that don't support VLAN hardware filtering that would not be affected. That could explain the conflicting results we see here.
Anyway if you're seeing this issue try disabling VLAN_HWFILTER.

Steve

c45p32

@stephenw10 Well holy crap batman, that worked. With all default settings:
Hardware Checksum Offloading: enable
Disable hardware TCP segmentation offload: disabled
Disable hardware large receive offload: disabled
and just setting ifconfig em0 -vlanhwfilter I was able to pull an IP address and surf the internet.

Jarhead

@c45p32
Are you saying this works on 2.6?

c45p32

@jarhead No, I couldn't get it to work on 2.6.0 (22.05) with only the driver and no script on igb0. My testing above was done on my back-up router with em0 and the lastest dev 2.7

You can get 2.6.0 (22.05) to work with the following instructions:
I've also attached my actual script that I keep in /root and I create a earlyshellcmd that points to /root/./vlanzero.sh
vlanzero.zip

Use this post (https://github.com/MonkWho/pfatt/issues/67#issuecomment-1043358822) to install the driver for igb0.

Reboot

Download vlan script from here: [vlanzero.sh.tar.gz] (https://forum.netgate.com/assets/uploads/files/1614644825333-vlanzero.sh.tar.gz) and unzip it.

Edit vlanzero.sh and change ONT_IF='xx0' to your WAN assignment, usually igb0. You can find this under Interfaces > Assignments

In the same file change ONT_ETHER_ADDR='xx:xx:xx:xx:xx:xx' to your mac address for your WAN. You can usually see this under Status > Interfaces Under the WAN interface heading. Save and close the file.

Goto Diagnostics > Command Prompt and Upload the vlanzero.sh file you just edited.

ssh into pfSense

type the following commands:
mv /tmp/vlanzero.sh /root/vlanzero.sh
chmod +x vlanzero.sh
./vlanzero.sh

Go to Interface > Assignments and change your WAN to ngeth0 click save and apply

Make sure you have the package shellcmd installed. Go to Services > Shellcmd

Click "Add" for Command put "/root/./vlanzero.sh" without quotes, shellcmd type set it to "earlyshellcmd", and then name it what ever you want. Save

Reboot and you should be working.

Note you need to either release your IP using the Frontier box, or just unplug your ONT for a few min. I usually just unplug easier.

stephenw10

Yeah, it won't work in 2.6/22.05 because it doesn't have the vlan0 bpf patch for dhclient. 2.7/22.11 does.

Jarhead

@stephenw10
So Shouldn't 2.7 "just work" without needing to do anything?

stephenw10

It would for any NIC except an e1000. At least any that I've tested.
For em/igb you need to disable vlan_hwfilter. Currently.

Steve

michaellacroix

@stephenw10

Great find!

stephenw10

What would confirm that as the source of confusion would be if the cases where two seemingly identical boxes had different behaviour actually had different NIC chips and only one supported vlan_hwfilter.

Steve

michaellacroix

@stephenw10

Hi Stephen, does netgate make any type of release notes for the daily snapshots? As they move along I'm curious as to what issues were addressed. Thanks

stephenw10

Not beyond the commit logs in github. At least not publicly:
https://github.com/pfsense/pfsense/commits/master

Steve

Schwiing

Just got Frontier Fiber installed today (2Gig Service). Hooked the FRX523 to my PFsense router and it got an IP immediately. No need to strip the tag. I did have a few weird speed tests (940/920, 200/600) but after that it's been almost consistent with 2.5/2.5G.

Not that this is extremely useful here, but it's a data point nonetheless.

stephenw10

In 2.6 or 2.7-dev?

Schwiing

@stephenw10 in 22.05. My thought is just that Frontier isn't using Vlan0 here. But I'm also not entirely sure one way or the other aside from the result. Is there a way for me to check?

michaellacroix

@schwiing

I took a pcap a couple of weeks ago and the vlan 0 tag was there. I also have frontier in CT. Are you familiar with wireshark?

stephenw10

Yes, it would fail in 22.05 if they were tagging. You can run a packet capture on WAN for port 68 and then renew the dhcp lease. If you then view it at full detail you will see any tagging in the replies.

Steve

Schwiing

@stephenw10 @michaellacroix

I'm familiar with Wireshark but have only run it once and that was a long time ago. I may need some help to figure that one out.

I'm sure there's some good guides on the web though. Can report back once I figure it out