PfSense Filtering Bridge on Wrong Interfaces?
-
Obvious question is what rules do you have on LAN_WIFI?
-
Obvious question is what rules do you have on LAN_WIFI?
Pretty simple: both LAN_WIFI and LAN_WIRE have the same set of rules: "pass DNS", "pass NTP", and then "pass all with logging".
The LAN bridge interface just has a single "pass all with logging" rule.
Traffic from LAN_WIRE gets logged by the LAN_WIRE "pass all" rule (as expected), but traffic from LAN_WIFI gets logged by the LAN "pass all " rule (should be LAN_WIFI).
-
So I setup another test to simplify this a bit:
TEST1 (igb4) | |---TEST (Bridged: TEST1 + TEST2) | TEST2 (igb5)
With the following rules:
TEST1: Pass All IPv4 with Logging TEST2: Pass All IPv4 with Logging TEST: Deny All IPv4 with Logging
And the bridging system tunables still set to their default config (filter on member interfaces, not bridge interface).
Some behavior as before:
PING TEST2 -> TEST1 - Passed, Logs as TEST2 Pass All PING TEST1 -> TEST2 - Blocked, Logs as TEST Deny All
igb4 has been associated with the problem interface in both cases, so maybe I'll try again with a different interface in case it's a quirk of igb4.
-
So that's just the real interfaces, no VLANs or LAGG?
If you're pinging between the bridge members why is the traffic hitting the bridge interface at all? It should go directly.
Are the NIC MACs being correctly read?Steve
-
The ports are are igb physical NICs.
I'm pinging between a host connected to the TEST1 interface and a host connected to the TEST2 interface. Those pings should cross the bridge (since that's what a bridge does), but the filtering should happen at the TEST1 and TEST2 interfaces, not the bridge interface itself.
igb4: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500 options=503bb <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsum,tso4,tso6,vlan_hwfilter,vlan_hwtso>ether 00:90:0b:33:e2:42 inet6 fe80::290:bff:fe33:e242%igb4 prefixlen 64 scopeid 0x5 nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect (1000baseT <full-duplex>) status: active igb5: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500 options=403bb <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsum,tso4,tso6,vlan_hwtso>ether 00:90:0b:33:e2:43 inet6 fe80::290:bff:fe33:e243%igb5 prefixlen 64 scopeid 0x6 nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect (100baseTX <full-duplex>) status: active bridge0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500 ether 02:fe:4a:c8:9c:00 inet 192.168.1.10 netmask 0xffffff00 broadcast 192.168.1.255 nd6 options=1 <performnud>id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 member: igb5 flags=143 <learning,discover,autoedge,autoptp>ifmaxaddr 0 port 6 priority 128 path cost 55 member: igb4 flags=143 <learning,discover,autoedge,autoptp>ifmaxaddr 0 port 5 priority 128 path cost 55</learning,discover,autoedge,autoptp></learning,discover,autoedge,autoptp></performnud></up,broadcast,running,simplex,multicast></full-duplex></performnud,auto_linklocal></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsum,tso4,tso6,vlan_hwtso></up,broadcast,running,promisc,simplex,multicast></full-duplex></performnud,auto_linklocal></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsum,tso4,tso6,vlan_hwfilter,vlan_hwtso></up,broadcast,running,promisc,simplex,multicast>
-
Are those NICs not identical, they have consecutive MACs?
igb4 has VLAN_HWFILTER enabled and igb5 doesn't. No idea what that might be doing now you're not using VLANs.
If the bridge is working correctly I wouldn't expect to see any traffic hit the rules on the bridge interface because it's not being routed to some other interface. The traffic between the two hosts should remain on the bridge only. So why is it both hitting the bridge interface filters at all and not the member interface filters. :-\Steve
-
Are those NICs not identical, they have consecutive MACs?
They're constitutive NICs on the same mobo: it's pretty common for such nics to have consecutive MACs
igb4 has VLAN_HWFILTER enabled and igb5 doesn't. No idea what that might be doing now you're not using VLANs.
Not sure what's going on there. Maybe leftover from previous vlan setups on igb4…
If the bridge is working correctly I wouldn't expect to see any traffic hit the rules on the bridge interface because it's not being routed to some other interface. The traffic between the two hosts should remain on the bridge only. So why is it both hitting the bridge interface filters at all and not the member interface filters. :-\
Yep, that's the problem this thread is about. Why is traffic hitting the bridge interface filter instead of the member interface filter? Hence the title: "pfSense Filtering Bridge on Wrong Interfaces?" ::)
It works correctly in one direction, but not the other. pfSense has tunable options to toggle between filtering at the bridge interface or filtering at the member interfaces, but member interface filtering is the default and that's what I have setup. And I don't think it's ever suppose to change behavior based on direction: the system tunable controlling bridge vs member filtering should be consistent regardless of direction.
-
Yes, just thinking out loud there.
it's pretty common for such nics to have consecutive MACs
Exactly, which is why I thought they were probably identical yet are reporting different hardware options.
Do they report the same capabilities in 'ifconfig -m igb4'?
Steve
-
Do they report the same capabilities in 'ifconfig -m igb4'?
I believe the VLAN_HWFILTER difference was just because I still had a VLAN setup on the igb4 interface, it just wasn't assigned anywhere. I deleted the VLAN and rebooted and now it's gone:
[2.2-BETA]/root(1): ifconfig -m igb4 igb4: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500 options=403bb <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsum,tso4,tso6,vlan_hwtso>capabilities=507fb <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,polling,vlan_hwcsum,tso4,tso6,lro,vlan_hwfilter,vlan_hwtso>ether 00:90:0b:33:e2:42 inet6 fe80::290:bff:fe33:e242%igb4 prefixlen 64 scopeid 0x5 nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect (1000baseT <full-duplex>) status: active supported media: media autoselect media 1000baseT media 1000baseT mediaopt full-duplex media 100baseTX mediaopt full-duplex media 100baseTX media 10baseT/UTP mediaopt full-duplex media 10baseT/UTP</full-duplex></performnud,auto_linklocal></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,polling,vlan_hwcsum,tso4,tso6,lro,vlan_hwfilter,vlan_hwtso></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsum,tso4,tso6,vlan_hwtso></up,broadcast,running,promisc,simplex,multicast>
[2.2-BETA]/root(2): ifconfig -m igb5 igb5: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500 options=403bb <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsum,tso4,tso6,vlan_hwtso>capabilities=507fb <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,polling,vlan_hwcsum,tso4,tso6,lro,vlan_hwfilter,vlan_hwtso>ether 00:90:0b:33:e2:43 inet6 fe80::290:bff:fe33:e243%igb5 prefixlen 64 scopeid 0x6 nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect (100baseTX <full-duplex>) status: active supported media: media autoselect media 1000baseT media 1000baseT mediaopt full-duplex media 100baseTX mediaopt full-duplex media 100baseTX media 10baseT/UTP mediaopt full-duplex media 10baseT/UTP</full-duplex></performnud,auto_linklocal></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,polling,vlan_hwcsum,tso4,tso6,lro,vlan_hwfilter,vlan_hwtso></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsum,tso4,tso6,vlan_hwtso></up,broadcast,running,promisc,simplex,multicast>
Let me dig into it a bit more this evening. I may have stumbled on something but I need to poke at it some more first…
-
I played with this a bit more. It appears that maybe I'm just misunderstanding the intended bridging behavior. Here's how the system appears to be behaving now:
TEST1 (igb4)
|
|–- TEST_BRIDGE (TEST1 + TEST2)
|
TEST2 (igb5)TEST1: Pass All IPv4 with Logging
TEST2: Pass All IPv4 with Logging
TEST_BRIDGE: Pass All IPv4 with LoggingPING TEST1 -> TEST2 - Logs as TEST1 Pass All
PING TEST2 -> TEST1 - Logs as TEST2 Pass All
PING TEST1 -> Internet - Logs as TEST_BRIDGE Pass All
PING TEST2 -> Internet - Logs as TEST_BRIDGE Pass AllSo it appears as though cross-bridge traffic (e.g. TEST1 to TEST2) get handled by the rules for the ingress member port, whereas traffic from a bridge-port to an external device (e.g. the Internet) gets filtered by the bridge rules.
Is that how its suppose to work? I was assuming that everything would just be filtered by the member interface rules and that the bridge rules wouldn't come into play at all, but maybe that's not so?
-
That's certainly not how I understood it.
How do you have the sysctls set in the above setup?It's an interesting point you raise though. The bridge setup I have on my test box has no rules on the member interfaces. Whilst it's never given any trouble, clients on the bridged interfaces can access external addresses and the webgui, I can't be sure whether I ever tested connectivity between the bridged nics. :-
I'll test it when I get a chance.Steve
-
How do you have the sysctls set in the above setup?
I believe they're set to the default behavior of only filtering on the member interfaces:
net.link.bridge.pfil_member = 1
net.link.bridge.pfil_bridge = 0