VLANs stop working after upgrading from 24.11 (for both 25.07.1 + 25.11)

petrt3522

@stephenw10 no, no lost traffic on 24.11

stephenw10

said in VLANs stop working after upgrading from 24.11 (for both 25.07.1 + 25.11):

If you boot back into the 24.11 BE do you still see the same dropped packets and errors at boot?

OK do you see those errors in the 24.11 boot logs? Compared to the boot errors in 25.11.1?

That latest pcap in on the WAN. I assume you also see the pings both ways on the internal interface?

Are you able to test 26.03-RC?

petrt3522

@stephenw10 None of those "chvf0" or mapping related errors in 24.11 boot.

Note, I upgrade 25.11 to 25.11.1 and recieved this odd error, but couldn't find a corresponding log?? h![ttps://imgur.com/GSCwbpE](image url)

Good of you ask, I tested now and pings+replies do show-up in the VLAN. ![https://imgur.com/bBQxqgA](image url)

Upgraded that environment to 26.03, but tests look to be the same.
WAN: h![ttps://imgur.com/lukPcAA](image url)
VLAN: ![https://imgur.com/UtIvOo2](image url)
"chvf0: cannot map registers." errors are listed.

stephenw10

Just to confirm though, you're not using that Chelsio NIC in this failing VLAN connection right?

petrt3522

@stephenw10 only using a 4 port Netextreme II

stephenw10

Hmm. This still feels like a config error somehow because there are many people using the bce driver with VLANs.

Though I can't see what it is.

Are you able to test a far more basic config? Something mostly default with just the VLANs added?

petrt3522

@stephenw10 I could try. However, I am not aware of a way to flush a config, AND from only 1 BE, so I can retain my 24.11 working production. I'm not sure how reimporting a config that I manually cleaned would go, that is also if I wouldn't miss anything and cause a mapping issue.

Any thoughts to accomplish?

stephenw10

You can clone the current BE (or use quick create) and boot into it. Then default the config in that new BE add back only the minimum config. You can always boot back into any other BE.

petrt3522

@stephenw10 Ok, upgraded that BE to 26.03.1 (had to do console pfsense-upgrade first,then GUI to 26.03.1, as it would only display "Previous...." branch).

Did factory reset:

Changed LAN to /24
VLAN 3988 created on BCE1 (LAN)
Assigned new interface to 3988 with /27 = 192.168.4.225
DHCP 3988 setup for .226-.253
Flushed test client from DHCP leases, and did /release on client.
New IP address assigned/accepted.
Created ANY ANY firewall rule under 3988

Ping to 3988 interface failed
Ping from 3988 to client failed
Ping from 3988 to 1.1.1.1 success
firewall logs showed no denys

https://imgur.com/a/0TtVuk8

??

stephenw10

Hmm, interesting. So the client does appear in the pfSense DHCP lease table with the correct MAC?

And I assume it must be in the ARP table?

And the client cannot ping either pfSense or 1.1.1.1?

Feels like it must be some hardware off-loading then. What does ifconfig -vm bce1 show?
Edit: Ah I see I asked that previously. Still feels relevant though. Some new off-load feature in the driver not playing nicely with that particular chip could do it.

petrt3522

working with GPT 5.6 today, I decided to also assign the VLAN as an Access port to bce2 to test a hardwired connection. Setup bridge0 and disabled IP on VLAN interface and switched to brdige0 & setup matching DHCP. Setup Any rule under bridge0 interface. All traffic passed!

Cloned live environment & upgraded. Setup Bridge0 the same by moving IP, DHCP, rules to it. This is run on 26.03.1

 ifconfig -vm bce1
bce1: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> met                                 ric 0 mtu 1500
        description: LAN
        options=800b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,LINKSTATE>
        capabilities=c01bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_                                 HWCSUM,TSO4,VLAN_HWTSO,LINKSTATE>
        ether 00:10:18:df:28:02
        inet 192.170.1.1 netmask 0xffff0000 broadcast 192.170.255.255
        inet 192.168.5.1 netmask 0xffffff00 broadcast 192.168.5.255
        inet 10.2.10.1 netmask 0xffffffff broadcast 10.2.10.1
        inet6 fe80::210:18ff:fedf:2802%bce1 prefixlen 64 scopeid 0x2
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        supported media:
                media autoselect mediaopt flowcontrol
                media autoselect
                media 1000baseT mediaopt full-duplex,master
                media 1000baseT mediaopt full-duplex
                media 1000baseT mediaopt master
                media 1000baseT
                media 100baseTX mediaopt full-duplex
                media 100baseTX
                media 10baseT/UTP mediaopt full-duplex
                media 10baseT/UTP
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        drivername: bce1

REVERTED back to VLAN interface, disabling Bridge0. Works!

STUMBLED ONTO ACTUAL ISSUE:
When connected via VLAN network, and inside pfSense GUI under DHCP Server~VLAN Scope, I enable "Enable Static ARP" traffic stopped immediately on Apply, when I was connected on the VLAN.
Re-connected to standard LAN (has "Enable Static ARP" disabled), disabled Static ARP and reconnected to VLAN, Traffic is then fully flowing!

stephenw10

Ah, yes, static ARP is almost always a bad idea IMO. If anything changes MAC address it just breaks and it difficult to troubleshoot.

I'm not sure why it would make any difference between 24.11 and 25.X though.