Transparent Firewall/Filtering Bridge with VLAN Trunk

btarrh · Jul 18, 2013, 4:59 PM

I am trying to duplicate Trendchiller and William Tarrh's Transparent Firewall/Filtering Bridge using VLAN's. I can successfully duplicate this setup while not utilizing VLAN's but my ultimate goal is to filter four (4) VLAN's on a single box. Having four pfSense servers doesn't seem logical when I can create a VLAN Trunk and use a single box. I would like to make this work successfully on a single VLAN before testing all the VLAN's at once. Any help would be great.

1. I start by successfully setting up my VLAN on the WAN and LAN from the initial CLI. I give my LAN and WAN different static IP's on the same subnet. I am able to successfully ping out
from the WAN and access the GUI from the LAN.
2. I enable the Filtering Bridge "System Tunables" -> net.link.bridge.pfil_bridge = 1
3. I go to "Assign Interfaces" -> Bridges -> and select my WAN and LAN. As soon as I select "Save" I am locked out and can no longer access the box from the WAN or LAN until I unplug
the LAN cable and am able to access the interface after about 15 seconds. /var/log/system.log will states that it can no longer talk with the default gateway.???

interface GigabitEthernet5/0/1
description rhph-347-i
switchport trunk encapsulation dot1q
switchport trunk native vlan 1005
switchport trunk allowed vlan 71,116,117,180
switchport mode trunk
power inline never
speed auto 10 100 1000
snmp trap mac-notification change added
snmp trap mac-notification change removed
no snmp trap link-status
no cdp enable
spanning-tree portfast trunk
end
!
interface GigabitEthernet3/0/36
description rhph-347-j
switchport trunk encapsulation dot1q
switchport trunk native vlan 1005
switchport trunk allowed vlan 960-963
switchport mode trunk
power inline never
snmp trap mac-notification change added
snmp trap mac-notification change removed
no snmp trap link-status
no cdp enable
spanning-tree portfast trunk
end

http://pfsense.trendchiller.com/transparent_firewall.pdf
http://people.pharmacy.purdue.edu/~tarrh/Transparent%20Firewall-Filtering%20Bridge%20-%20pfSense%202.0.2%20By%20William%20Tarrh.pdf

jimp · Jul 19, 2013, 3:41 PM

@btarrh:

1. I start by successfully setting up my VLAN on the WAN and LAN from the initial CLI. I give my LAN and WAN different static IP's on the same subnet. I am able to successfully ping out
from the WAN and access the GUI from the LAN.
2. I enable the Filtering Bridge "System Tunables" -> net.link.bridge.pfil_bridge = 1

That doc is old.

#1 - never do that. Only ONE interface on a bridge should have an IP.
#2 - don't do that either, it isn't doing what you think it's doing. The doc and the old advice are no longer relevant.

Use or put a third NIC in the box for OOB management. It's extremely tough to work with bridging if you are managing the firewall from an interface being bridged.

In the ideal scenario you'll have:

LAN VLAN A – Bridge A (IP address on the bridge) -- WAN VLAN A
LAN VLAN B -- Bridge B (IP address on the bridge) -- WAN VLAN B
LAN VLAN C -- Bridge C (IP address on the bridge) -- WAN VLAN C
LAN VLAN D -- Bridge D (IP address on the bridge) -- WAN VLAN D

WAN and LAN VLANs would have an interface type of "none" (so no IP address on them). Bridge interfaces would be assigned and have the IPs configured there.

Firewall rules would go on the WAN VLAN and LAN VLAN tabs for each individual VLAN.

If you want to filter on the bridge interface, disable bridge member filtering, enable bridge filtering, and put the firewall rules on the assigned bridge tabs instead.

Either way, unless you do all of that from a third interface unrelated to the VANs, you'll be in for a lot of avoidable pain.

btarrh · Jul 19, 2013, 7:28 PM

Jimp,

Thank you so much for your reply. I have been poking at this for weeks and reaching the end of my sanity (seriously). I'm setting the pfSense box as I type and will do some testing on the rules to be sure I've got your instructions correct.

Is it necessary to disable NAT for this to work properly? Do I need to modify NAT at all?

Thanks,

–Bill

jimp · Jul 19, 2013, 7:31 PM

You probably should disable NAT, at least such that it wouldn't match any of the traffic coming across the VLANS (if you have a non-bridge management interface it could still NAT out WAN if you need it to)

btarrh · Jul 20, 2013, 8:39 PM

Physical Interfaces
WAN - bce0
LAN - bce1
MANAGEMENT - em0

During the whole configuration the only port connected is the VLAN-WAN

1. I setup my VLAN's during the initial CLI configuration (WAN-VLAN assigned to bce0 and LAN-VLAN assigned to bece1).
2. I access the GUI with a DHCP address automatically assigned to my WAN (I create a temporary anti-lockout rule to allow ANY to WAN over HTTPS).
3. I assign my interfaces and then immediately configure my MANAGEMENT port with a static ip/24 (no default gateway) and create an allow to ANY rule.
4. I then login through the MANAGEMENT interface and begin my configuration.
5. I configure my WAN-VLAN and LAN-VLAN as NONE and save changes.
6. I then BRIDGE the WAN-VLAN to the LAN-VLAN and save changes.
7. I then configure the BRIDGE with a static IP/24 and its default gateway and save changes.
8. I then enable Manual Outbound NAT rule generation, delete all auto populated rules, apply and save changes.
9. I then disable bridge member filtering and enable bridge filtering and save changes.
10. finally I set my DNS and NTP servers.
11. At this point I always restart the box.

With ONLY the VLAN-WAN connected I can set rules on the BRIDGE to allow PING and SSH from ANY and can successful get a response and access the box from internal or external. As soon as I plug in the VLAN-LAN traffic will no longer pass through the BRIDGE and all PING responses and SSH connectivity is stopped/disconnected. When I unplug the VLAN-LAN traffic will start to flow in about 30 seconds and things return to normal.

If the VLAN-WAN is disconnected and I connect only the VLAN-LAN I see the same errors/issues as mentioned above, seems that the issue might be on the VLAN-LAN interface.? I wonder if I'm creating a loop somewhere and how to even detect it in the logs.

The log below is from /var/log/system.log, the start is when I plug in the LAN/Trusted and then unplug again, as you can see I am able to reconnect with SSH. If there is a more helpful log I should be looking at please let me know.

Jul 20 12:58:44 pfsense check_reload_status: Linkup starting bce1
Jul 20 12:58:44 pfsense kernel: bce1: link state changed to UP
Jul 20 12:58:44 pfsense kernel: bce1_vlan963: link state changed to UP
Jul 20 12:58:44 pfsense kernel: bce1_vlan962: link state changed to UP
Jul 20 12:58:44 pfsense kernel: bce1_vlan961: link state changed to UP
Jul 20 12:58:44 pfsense kernel: bce1_vlan960: link state changed to UP
Jul 20 12:58:44 pfsense check_reload_status: Linkup starting bce1_vlan963
Jul 20 12:58:44 pfsense check_reload_status: Linkup starting bce1_vlan962
Jul 20 12:58:44 pfsense check_reload_status: Linkup starting bce1_vlan961
Jul 20 12:58:44 pfsense check_reload_status: Linkup starting bce1_vlan960
Jul 20 12:58:46 pfsense php: : Hotplug event detected for opt4 but ignoring since interface is configured with static IP ()
Jul 20 12:58:46 pfsense check_reload_status: rc.newwanip starting bce1_vlan960
Jul 20 12:58:49 pfsense php: : rc.newwanip: Informational is starting bce1_vlan960.
Jul 20 12:58:49 pfsense php: : rc.newwanip: on (IP address: ) (interface: opt4) (real interface: bce1_vlan960).
Jul 20 12:58:49 pfsense php: : rc.newwanip: Failed to update opt4 IP, restarting…
Jul 20 12:58:49 pfsense check_reload_status: Configuring interface opt4
Jul 20 12:58:52 pfsense apinger: rrdtool respawning too fast, waiting 300s.
Jul 20 12:58:57 pfsense apinger: ALARM: BRIDGE71GW(128.210.71.1) *** down ***
Jul 20 12:59:07 pfsense check_reload_status: Reloading filter
Jul 20 12:59:14 pfsense check_reload_status: Linkup starting bce1
Jul 20 12:59:14 pfsense kernel: bce1: link state changed to DOWN
Jul 20 12:59:14 pfsense kernel: bce1_vlan963: link state changed to DOWN
Jul 20 12:59:14 pfsense kernel: bce1_vlan962: link state changed to DOWN
Jul 20 12:59:14 pfsense kernel: bce1_vlan961: link state changed to DOWN
Jul 20 12:59:14 pfsense kernel: bce1_vlan960: link state changed to DOWN
Jul 20 12:59:14 pfsense check_reload_status: Linkup starting bce1_vlan963
Jul 20 12:59:14 pfsense check_reload_status: Linkup starting bce1_vlan962
Jul 20 12:59:14 pfsense check_reload_status: Linkup starting bce1_vlan961
Jul 20 12:59:14 pfsense check_reload_status: Linkup starting bce1_vlan960
Jul 20 12:59:14 pfsense sshd[15098]: fatal: Write failed: Operation not permitted
Jul 20 12:59:14 pfsense sshd[15098]: fatal: Write failed: Operation not permitted
Jul 20 12:59:16 pfsense php: : Hotplug event detected for opt4 but ignoring since interface is configured with static IP ()
Jul 20 12:59:57 pfsense apinger: alarm canceled: BRIDGE71GW(128.210.71.1) *** down ***
Jul 20 13:00:07 pfsense check_reload_status: Reloading filter
Jul 20 13:00:18 pfsense ntpd_intres[15056]: host name not found: wppcendc01.central.purdue.lcl
Jul 20 13:00:28 pfsense sshd[46762]: Accepted keyboard-interactive/pam for admin from 128.210.116.96 port 31222 ssh2

btarrh · Jul 22, 2013, 2:31 PM

I found this topic over the weekend after searching "check_reload_status" from my log - http://forum.pfsense.org/index.php/topic,63052.0.html.
First I'll try disabling State Killing and run a test and then I'll try disabling Gateway Monitoring and report my results.

In your recommended setup you said: "If you want to filter on the bridge interface, disable bridge member filtering, enable bridge filtering, and put the firewall rules on the assigned bridge tabs instead.".

I noticed that even if I leave System Tunables default and don't disable-bridge member filtering and enable-bridge filtering my firewall rules still only work on the BRIDGE interface.

btarrh · Jul 22, 2013, 2:00 PM

I requested one of our campus network engineers to look at the logs on the switch and this message was generated every time I plug in the LAN/Trust: Received BPDU with inconsistent peer vlan id ## on GigabitEthernet3/0/36 VLAN###.

The engineer then made this recommendation:
"I wonder if we have an interface mixup. For instance, one of the messages says port 3/0/36, which is the 960-963 range on my end, is receiving traffic with a vlan tag of 960 - 963. Check that first. But if that doesn't work, try blocking BPDU's on the firewall."

According to this post in the forum (http://forum.pfsense.org/index.php?topic=9545.0) I can filter out the BPDU's with this method:
spanning-tree bpdufilter enable

btarrh · Jul 23, 2013, 3:19 PM

I am successfully using pfSense as a transparent filtering bridge on a VLAN trunk with four subnets.

Our two VLAN Trunks we on the same switch and according to http://forum.pfsense.org/index.php?topic=9799.0;prev_next=next this may cause a loop which was confirmed in the log from the switch posted above. I believe the loops could of been resolved in the Spanning Tree options on the bridge interface but that would of needed to be done by someone much smarter than myself. I requested our network engineer to add "spanning-tree bpdufilter enable" to both of our VLAN Trunks and that resolved the looping issue and ultimately allowed traffic to flow without the switch turning off the port because of the loop.

Core7 · Jan 22, 2023, 6:36 PM

Colleagues, good afternoon. After a long time of trying, I managed to get the vlan to work with the pfsense in bridge mode. the WAN and LAN work perfectly, however, when creating another bridge with the WAN interface and the physical LAN without VLAN, the previously created VLAN stops working, but when I remove the WAN interface from the bridge it works again. What could be causing the problem?