How to get pfSense WAN to accept VLAN 0

stephenw10

Hmm, looks like they are using identical drivers.

Did you test with em or igb specifically? Though they are the same driver.... except if they're not!

Schwiing

@stephenw10 would anyone be able to test with ix?

stephenw10

It works with ix with the netgraph script in 2.6 and I expect it to work with ix without a script in 2.7 because main has the vlan0 dhclient patch.

It's worth trying that on em/igb too since I can't see any different there. Though I can't see how it could work given it doesn't even appear in a pcap. I could be missing something....

c45p32

I loaded opnsense onto that same backup router and I can’t get it to work on em0. I even went into the attic, got the ISP router to formally release the IP and still no go. So not sure how people are getting it to work on opnsense. I’m on Frontier fiber College Station, Texas.

stephenw10

Ah, well that would make more sense. Still seem to be conflicting reports.

I suspect there are other issues at work and people have just assumed it was the e1000 driver problem because that's known to exist.

As far as I can see the new CE snapshots should respond exactly like the current OPNSense does. Still running some tests though.

Steve

Jarhead

@stephenw10 said in How to get pfSense WAN to accept VLAN 0:

Hmm, looks like they are using identical drivers.

Did you test with em or igb specifically? Though they are the same driver.... except if they're not!

I use a Protectli Vault with em interfaces.
Plug it right into the ONT and it works.

Jarhead

@c45p32 said in How to get pfSense WAN to accept VLAN 0:

I loaded opnsense onto that same backup router and I can’t get it to work on em0. I even went into the attic, got the ISP router to formally release the IP and still no go. So not sure how people are getting it to work on opnsense. I’m on Frontier fiber College Station, Texas.

What version of Opnsense?
I see the latest has all kinds of problems.
Try 22.7. Works with em drivers. Plug drirectly into the ONT.

stephenw10

If you run a pcap can you see the vlan0 tagged dhcp responses?

michaellacroix

@stephenw10

I just tested the latest pfsense dev image using em0 as my wan interface and it was able to pickup an IP address. I'm using frontier fiber and was directly connected to the ONT.
This is hopeful...

stephenw10

Yes that is useful.

It would be very interesting to see if those DHCP replies appear in a pcap. I wasn't able to see vlan0 tagged pings but perhaps some off-loading allows the udp dhcp packets....

stephenw10

Ok, after a bunch of testing it appears the difference here is in OPN pretty much all hardware offloading is disabled by default and we have most things enabled. Including VLAN Hardware Filtering which appears to be broken for VLAN0 in the iflib e1000 driver.

Manually disabling it allowed my test boxes to pull a dhcp lease from a server that is priority tagging replies:

ifconfig em2 -vlanhwfilter

01:04:04.652627 00:90:7f:87:dc:77 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328)
    0.0.0.0.68 > 255.255.255.255.67: [udp sum ok] BOOTP/DHCP, Request from 00:90:7f:87:dc:77, length 300, xid 0x7b209734, Flags [none] (0x0000)
01:04:04.654209 90:ec:77:1f:8a:62 > 00:90:7f:87:dc:77, ethertype 802.1Q (0x8100), length 346: vlan 0, p 4, ethertype IPv4, (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328)
    10.14.0.1.67 > 10.14.0.12.68: [udp sum ok] BOOTP/DHCP, Reply, length 300, xid 0x7b209734, Flags [none] (0x0000)

None of my tests allowed any client to see the servers replies with vlan hardware filtering still enabled. However I could imagine there might be em or igb NICs that don't support VLAN hardware filtering that would not be affected. That could explain the conflicting results we see here.
Anyway if you're seeing this issue try disabling VLAN_HWFILTER.

Steve

c45p32

@stephenw10 Well holy crap batman, that worked. With all default settings:
Hardware Checksum Offloading: enable
Disable hardware TCP segmentation offload: disabled
Disable hardware large receive offload: disabled
and just setting ifconfig em0 -vlanhwfilter I was able to pull an IP address and surf the internet.

Jarhead

@c45p32
Are you saying this works on 2.6?

c45p32

@jarhead No, I couldn't get it to work on 2.6.0 (22.05) with only the driver and no script on igb0. My testing above was done on my back-up router with em0 and the lastest dev 2.7

You can get 2.6.0 (22.05) to work with the following instructions:
I've also attached my actual script that I keep in /root and I create a earlyshellcmd that points to /root/./vlanzero.sh
vlanzero.zip

Use this post (https://github.com/MonkWho/pfatt/issues/67#issuecomment-1043358822) to install the driver for igb0.

Reboot

Download vlan script from here: [vlanzero.sh.tar.gz] (https://forum.netgate.com/assets/uploads/files/1614644825333-vlanzero.sh.tar.gz) and unzip it.

Edit vlanzero.sh and change ONT_IF='xx0' to your WAN assignment, usually igb0. You can find this under Interfaces > Assignments

In the same file change ONT_ETHER_ADDR='xx:xx:xx:xx:xx:xx' to your mac address for your WAN. You can usually see this under Status > Interfaces Under the WAN interface heading. Save and close the file.

Goto Diagnostics > Command Prompt and Upload the vlanzero.sh file you just edited.

ssh into pfSense

type the following commands:
mv /tmp/vlanzero.sh /root/vlanzero.sh
chmod +x vlanzero.sh
./vlanzero.sh

Go to Interface > Assignments and change your WAN to ngeth0 click save and apply

Make sure you have the package shellcmd installed. Go to Services > Shellcmd

Click "Add" for Command put "/root/./vlanzero.sh" without quotes, shellcmd type set it to "earlyshellcmd", and then name it what ever you want. Save

Reboot and you should be working.

Note you need to either release your IP using the Frontier box, or just unplug your ONT for a few min. I usually just unplug easier.

stephenw10

Yeah, it won't work in 2.6/22.05 because it doesn't have the vlan0 bpf patch for dhclient. 2.7/22.11 does.

Jarhead

@stephenw10
So Shouldn't 2.7 "just work" without needing to do anything?

stephenw10

It would for any NIC except an e1000. At least any that I've tested.
For em/igb you need to disable vlan_hwfilter. Currently.

Steve

michaellacroix

@stephenw10

Great find!

stephenw10

What would confirm that as the source of confusion would be if the cases where two seemingly identical boxes had different behaviour actually had different NIC chips and only one supported vlan_hwfilter.

Steve

michaellacroix

@stephenw10

Hi Stephen, does netgate make any type of release notes for the daily snapshots? As they move along I'm curious as to what issues were addressed. Thanks