<SOLVED> Netgate SG-5100 igb interface will not work with DHCP for some devices

[bmeeks]

I have a really strange issue and wondered if anyone else has encountered something similar.

I have a Netgate SG-5100. It has been in use with my cable modem ISP (using an Arris modem in bridge mode) for several years. The cable modem was attached to the WAN port (igb0) of the SG-5100, and the WAN IP was set to DHCP. This worked reliably for years (and still does).

Yesterday I was finally able to get a shiny new fiber-to-the-home symmetrical Gigabit connection from a new ISP. The ONT provided is an Azores Networks WAG-D20: https://www.balticnetworks.com/products/azores-1x-10gbe-1x-2-5gbe-intel-based-xgspon-ont. It is also configured in bridge mode and issues a WAN address for my SG-5100 via DHCP.

But yesterday when I swapped the RJ45 cable on the SG-5100 WAN port from the Arris cable modem over to the ONT, my WAN port would not get a DHCP address. Packet captures on pfSense for the WAN port were not helpful at all as they simply showed pfSense repeatedly issuing DHCPREQUEST packets out the port and waiting for a DHCPOFFER that never came back. Tried rebooting the firewall and swapping the cable with no change in behavior. Even tried putting a small Netgear Gigabit switch between the SG-5100 WAN port and Azores Networks ONT, but it made no difference. But plugging the Arris cable modem's LAN output back into the SG-5100 WAN port (igb0) resulted in immediate connectivity with a DHCP-assigned address. Wore a small hole in the side of my head from scratching it so much .

Finally, in desperation, I plugged the ONT Ethernet cable into another open SG-5100 port just to see if it would talk. The port chosen was OPT2 (ix1). I configured that port for DHCP, saved it, and immediately that port received an IP address via DHCP from the ONT. So, I then moved the exact same working RJ45 cable coming from the ONT over to the WAN port (igb0). No IP address and no connectivity. Move it back to OPT2 (the ix1 port) and immediately get an IP. So, the igb0 port on the SG-5100 refuses to play well with the Azores Networks ONT, but works perfectly fine with the Arris cable modem. So, for now I've simply reassigned my WAN interface to use port ix1.

Has anyone seen something similar with the igb ports? They are using the Intel i210 chipset according to the info logged in dmesg.boot. The ix ports use the Intel X553 chipset.

[stephenw10]

Check for priority tagging on the DHCP replies. There is bug in e1000 that incorrectly handles VLAN0 even though dhclient can now correctly receive them in 23.01.

https://redmine.pfsense.org/issues/12821

Disabling VLAN Hardware offloading on igb0 should allow it to work.

Steve

[bmeeks]

@stephenw10 said in Netgate SG-5100 igb interface will not work with DHCP for some devices:

Check for priority tagging on the DHCP replies. There is bug in e1000 that incorrectly handles VLAN0 even though dhclient can now correctly receive them in 23.01.

https://redmine.pfsense.org/issues/12821

Disabling VLAN Hardware offloading on igb0 should allow it to work.

Steve

Ha! Thanks! I do recall that bug now. It was blowing my mind yesterday when something that should be so simple (swap a cable) did not work.

I will give this a test. I assume I need to put the disable command in a rc.conf.local file and perhaps also in System Tunables?

[stephenw10]

I would probably add it as an early shell cmd just so it's in the config.

[bmeeks]

@stephenw10 said in Netgate SG-5100 igb interface will not work with DHCP for some devices:

I would probably add it as an early shell cmd just so it's in the config.

Okay. Thanks for the tip.

[bmeeks]

To close the loop: disabling VLAN_HWFILTER on the igb port solved the issue.

Put this in Early Shell Command using the Shellcmd package:

/sbin/ifconfig igb0 -vlanhwfilter -vlanhwtso

I can now get a DHCP address on the igb0 WAN port from the Azores Networks WAG-20D ONT.