Transparent Firewall/Filtering Bridge with VLAN Trunk Redux

[wdennis]

Hi all,

I have a scenario where I'd like to insert a control / monitoring point on a trunk link between two switches. I have done transparent (layer-2) firewalls before on a single access link (i.e., no VLAN tagging) and it works well. In this case, however, it's a 802.1q trunk link, with an arbitrary number of tagged VLANS, and one untagged ("native") vlan.

In searching the forum for like designs, I came across this post –
https://forum.pfsense.org/index.php/topic,64601.msg350130.html

It looks quite like what I'm trying to do, but the post (and the OP's solution) isn't quite clear to me...

I have a pfSense box with three (actually 4, but let's ignore that one for now) NICs; em0 is "LAN", em1 is "WAN", bge0 is "MGMT" (OPT1). The MGMT interface is used (only) for control of the pfSense box. LAN and WAN have been set up as address type "None" (i.e. no IPv4/v6 addressing) and joined together by a bridge "bridge0". My first question is, will the frame VLAN tagging be passed through the bridge "as is", or does FreeBSD need to have "VLAN interfaces" established for each tagged VLAN, and then these VLAN interfaces joined together in separate bridges, one per VLAN, for the tagging to stay intact on either side of the pfSense box? Also, if I can use only one bridge, can I filter on traffic by VLAN?

A futher question is, if I have to use VLAN int's on the pfSense box, does the untagged ("native") VLAN use the base LAN / WAN int's (em0 / em1) and then the VLAN ones get sub-int's like em0.2 (i.e for VLAN 2) and em0.3 (for VLAN 3) for instance? (Sorry, I'm used to Linux VLAN int's, not sure how FreeBSD represents these...) If so, I'd guess that I'd have to bridge the base LAN & WAN ints (em0 / em1), and then have another bridge for LAN-V2 & WAN-V2 (em0.2 / em1.2), etc?

I have attached diagrams showing the "one bridge for all VLANs" and "separate bridges for each VLAN" to illustrate each scenario.

Thanks for any clues provided...

• Will
  
  
  
  

[wdennis]

Anyone?? Someone must know the answer to this…

[wdennis]

Oh well, to the lab we go… I'll report back what I find out.

[wdennis]

OK, initial labbing results:

• pfSense Version: 2.1-RELEASE (amd64)
• 4 int's: LAN + WAN in a bridge "BR0" (bridge0), with another int PKTCAP (OPT2) set as the SPAN port; the 4th port MGMT (OPT1) is the int used to manage the box
• have a "north" switch off the WAN port, and a "south" switch off the LAN port (refer to diagram above); both are set for 802.1q tagging, with VLAN 1 (untagged frames), and VLAN 2 & 3 (tagged frames.) When the two switches were connected directly, all test traffic passed without problem on each VLAN.

a) No traffic is passing the L2 firewall in either direction… and I have rules in both the LAN and WAN tabs to allow all IPv4 and all IPv6 traffic.
b) added rules in the BR0 tab to allow for all IPv4 and IPv6, still no traffic passes...
c) If I use tcpdump and look at either the LAN (em0) or WAN (em1) ints, I see traffic from the LAN side, but not from the WAN side. This is especially weird on the WAN side tcpdump  :(
d) If I tcpdump the PKTCAP int, I do see traffic from *both sides... ???
e) I went ahead and added VLAN tagged sub-ints for both the LAN and WAN ints on reading this comment from JimP in another thread:
https://forum.pfsense.org/index.php/topic,64601.msg350315.html#msg350315
Here's the output from ifconfig:
[2.1-RELEASE][admin@test-inline-fw.nec-labs.com]/root(1): ifconfig | grep em
em0: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500
inet6 fe80::215:17ff:fe54:ab12%em0 prefixlen 64 scopeid 0x1
em1: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500
inet6 fe80::215:17ff:fe54:ab13%em1 prefixlen 64 scopeid 0x2
em0_vlan2: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
inet6 fe80::215:17ff:fe54:ab12%em0_vlan2 prefixlen 64 scopeid 0x9
vlan: 2 vlanpcp: 0 parent interface: em0
em0_vlan3: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
inet6 fe80::215:17ff:fe54:ab12%em0_vlan3 prefixlen 64 scopeid 0xa
vlan: 3 vlanpcp: 0 parent interface: em0
em1_vlan2: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
inet6 fe80::215:17ff:fe54:ab12%em1_vlan2 prefixlen 64 scopeid 0xb
vlan: 2 vlanpcp: 0 parent interface: em1
em1_vlan3: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
inet6 fe80::215:17ff:fe54:ab12%em1_vlan3 prefixlen 64 scopeid 0xc
vlan: 3 vlanpcp: 0 parent interface: em1
Traffic still not passing, and I see there aren't firewall rule tabs for the tagged sub-int's, not can I add them to bridges…

I've done a LOT of Google searching for the answer to this here, nothing seems particularly relevant, and what I have tried does not work to let traffic pass... So I'm about to give up here. I may try to see if $WORK will buck up for the commercial support option, but not sure that will fly... Bummer - everything I've tried to do so far with pfSense in the past has been a breeze and worked well, but this time I've seem to run into a brick wall. If anyone knows what else I might try, PLEASE chime in here... thanks.

</up,broadcast,running,simplex,multicast></up,broadcast,running,simplex,multicast></up,broadcast,running,simplex,multicast></up,broadcast,running,simplex,multicast></up,broadcast,running,promisc,simplex,multicast></up,broadcast,running,promisc,simplex,multicast>

[wdennis]

Kinda funny, this talking to myself here  :P

Anyways, installed openlldp, and figured out that I had cabled up the SPAN (PKTCAP) and the WAN links backwards… That helps explain c) and d) above ;)  (last thing I did on Friday before leaving - haste makes waste, folks...)

I fixed that, traffic still was not passing; I had also in the meantime went into System > Advanced > Firewall/NAT and checked the "Disable all packet filtering" box... In any case, when I rebooted the pfSense box (out of frustration more than anything else) when it came back up, what do you know -- traffic started passing!

So I did a bunch of traffic tests (ICMP pings) on all the VLANs, everything was working. The I went into System > Advanced > Firewall/NAT again and UNchecked the "Disable all packet filtering" box. Checked pfTop, I could see all the traffic between my test boxes, OSPF between the routers, etc. So far, so good... Then I went into Firewall > Rules and wrote a test rule to block incoming ICMP from a node that sits off the WAN int side. I applied the rule, and - the node can still ping! ?? I tried a few different firewall rules (all on ICMP traffic) and none seem to be working...

So giving up for the night here, but a few questions before I sign off...

1. Do I need the VLAN sub-ints on the LAN and WAN ports to ensure that tagged frames pass correctly thru the bridge, or is that not necessary?  (My original question, see top post)
2. Why wouldn't the ICMP block rule (block any ICMP in from <given_ip>) be working? (i.e. what should I look at?)

Oh well, at least I'm a little further down the road here.. Goodnight all.</given_ip>

[wdennis]

Figured it out… If another system on the LAN side is pinging the system I put the block in for, it sets up a dynamic ACL based on state that allows the blocked system's ping traffic in. So it seems that dynamic state-based ACLs trump a deny from a static ACL (rule)?

In any case, when the LAN system stopped pinging the blocked WAN system, and I later tried to have the blocked WAN system ping the LAN system, it did block the traffic, and I can see the blocks in the logs. Whew, I was losing my mind there...

If anyone knows the answer to the need for VLAN sub-ints to pass the tagged trunk traffic, that would be great to know (I could lab it and find out empirically, I guess, but if someone knows for sure, chime in!)

[Gio]

Hi

I am curious if you ever got an answer and working setup on this? I also have been considering setting up pfsense in transparent mode to do some traffic shaping and traffic control for a small datacenter setup with VLANs (all public IPs - segmented smaller blocks from a /25)