Bridging two interfaces WAN(vRouter) & LAN(LAN Router) w/ OPT1(MGMT)
-
I added any any allow rules in pf under
To_VYOS - WAN
To_3850 - LAN
OPT2 - BRIDGE0Status / System Logs / Firewall /Normal View
Shows no blocks since May 7 12:45:54
-
@juesor Do you really want any to connect to Opt2 or just source LAN.net to connect to Opt2.net?
-
Right now I want WAN and LAN to connect to each other over OPT2(Bridge).
Anything that lives in WAN and anything that lives in LAN should be able to bridge through pf and get to the other side.
-
Here is a quick drawing.
MGMT connects between pfsense and 3850 on a different vl810
-
Part of me is thinking that due to the mgmt interface and its gateway pf isn't bridging the interfaces.
But I cannot find any reason for this to not be working.
-
@juesor Have you looked at this: https://docs.netgate.com/pfsense/en/latest/bridges/index.html
From the above reference: "Also, in order for these functions to work, the IP address on the bridge must be the address used by clients as their gateway. These issues are discussed more in-depth in Bridging interoperability."
You seems close, keep working at it.
-
From that documentation, my setup is going to be a transparent firewall and it says i don't need to IP anything which is where we started.
Internal/external bridges connect a LAN to a WAN resulting in what is commonly called a “transparent firewall”.
Internal/External Bridges
An Internal/External type bridge, also known as a “transparent firewall”, is used to insert a firewall between two segments without altering the other devices. Most commonly this is used to bridge a WAN to an internal network so that the WAN subnet may be used “inside” the firewall, or internally between local segments as an in-line filter. Another common use is for devices behind the firewall to obtain IP addresses via DHCP from an upstream server on the WAN.In a transparent firewall configuration, the firewall does not receive the traffic directly or act as a gateway, it merely inspects the traffic as it passes through the firewall.
Note
Devices on the internal side of this bridge must continue to use the upstream gateway as their own gateway. Do not set any IP address on the firewall as a gateway for devices on a transparent bridge.
-
@juesor In my research, I came across this image which claim to need three NIC as yours, except your is virtual.
-
-
@juesor Yes, here is the PDF: http://users.ox.ac.uk/~clas0415/assets/Setting-up-pfSense-as-a-Stateful-Bridging-Firewall-with-commodity-hardware.pdf
-
Still no go.
Followed that pdf guide to the T and I also changed the bridge from WAN & LAN to WAN & OPT.
I'm wondering if anyone else has ever seen it when the bridge would basically not work.
When I packet capture on the interface facing the 3850.
13:58:30.484256 ARP, Request who-has 172.26.0.50 tell 172.26.0.49, length 46
13:58:30.484445 ARP, Request who-has 172.26.0.50 tell 172.26.0.49, length 46
13:58:35.490849 ARP, Request who-has 172.26.0.50 tell 172.26.0.49, length 46
13:58:35.491015 ARP, Request who-has 172.26.0.50 tell 172.26.0.49, length 46Great pings from the VLAN interface are looking for who has 172.26.0.50
Let's try that in reverse from VYOS trying to ping the 3850.
14:49:43.800853 IP 172.26.0.50 > 172.26.0.49: ICMP echo request, id 26267, seq 1, length 64
14:49:44.830652 IP 172.26.0.50 > 172.26.0.49: ICMP echo request, id 26267, seq 2, length 64
14:49:45.854536 IP 172.26.0.50 > 172.26.0.49: ICMP echo request, id 26267, seq 3, length 64
14:49:46.878667 IP 172.26.0.50 > 172.26.0.49: ICMP echo request, id 26267, seq 4, length 64
14:49:47.902641 IP 172.26.0.50 > 172.26.0.49: ICMP echo request, id 26267, seq 5, length 64
14:49:48.894546 ARP, Request who-has 172.26.0.49 tell 172.26.0.50, length 46
14:49:48.927016 IP 172.26.0.50 > 172.26.0.49: ICMP echo request, id 26267, seq 6, length 64
14:49:49.918516 ARP, Request who-has 172.26.0.49 tell 172.26.0.50, length 46
14:49:49.950569 IP 172.26.0.50 > 172.26.0.49: ICMP echo request, id 26267, seq 7, length 64
14:49:50.942582 ARP, Request who-has 172.26.0.49 tell 172.26.0.50, length 46
14:49:50.974568 IP 172.26.0.50 > 172.26.0.49: ICMP echo request, id 26267, seq 8, length 64Ok so the ICMP from VYOS is being seen.
Let's do the same but pinging from VYOS and capture on the 3850 interface.
14:51:08.286935 ARP, Request who-has 172.26.0.49 tell 172.26.0.50, length 46
14:51:08.286959 ARP, Request who-has 172.26.0.49 tell 172.26.0.50, length 46
14:51:09.310991 ARP, Request who-has 172.26.0.49 tell 172.26.0.50, length 46
14:51:09.311011 ARP, Request who-has 172.26.0.49 tell 172.26.0.50, length 46
14:51:10.335411 ARP, Request who-has 172.26.0.49 tell 172.26.0.50, length 46
14:51:10.335433 ARP, Request who-has 172.26.0.49 tell 172.26.0.50, length 46
14:51:11.358975 ARP, Request who-has 172.26.0.49 tell 172.26.0.50, length 46
14:51:11.358984 ARP, Request who-has 172.26.0.49 tell 172.26.0.50, length 46
14:51:12.383035 ARP, Request who-has 172.26.0.49 tell 172.26.0.50, length 46
14:51:12.383045 ARP, Request who-has 172.26.0.49 tell 172.26.0.50, length 46I don't see the ICMP but for some reason, that looks like it is working?
Ok getting into VYOS I can run a tcpdump
vyos@vyos:~$ monitor traffic interface eth2
tcpdump: verbose output suppressed, use -v or --v for full protocol decode
listening on eth2, link-type EN10MB (Ethernet), capture size 262144 bytesok sweet the tcpdump is running let's see if we see anything when pinging 172.26.0.50 from the 3850.
#ping 172.26.0.50
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 172.26.0.50, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)Crap.
Nothing in VYOS showing packets are received and failing
Now let's keep running the packet capture on VYOS and capture on the pfsense. starting at the to_3850 and then to_vyos.
CRAP!!!
The to_3850 shows NOTHING
The to_VYOS shows NOTHING
What is going on here?¿?¿
-
First of all, you must examine your Layer2 Topology and ensure you won't create any Layer2 Loop with this Setup. If all Uplinks of the ESXi are connected to the same Layer2 Segment proceed with cautions. Spanning-Tree won't save you here, cause STP-BPDUs are filtered on the ESXi per default.
You need to allow Forged transmits and Promiscuous Mode on the used Port Groups for the bridged interfaces. This is a necessary configuration on the Virtualization Platform to make this setup work. After Changing these settings you have to either reboot the firewall or disconnect and reconnect the interfaces to allocated a new virtual port with the updated settings.Better to configure MAC-Learning for these Port-Groups. This Option is available on Distributed Switch since version 6.6.0 through API Calls.
-
@artes
Thanks. I set promiscuous mode on the vlan's within ESXi.
It still isn't working.
So i stood up a new lab.
3 vlans in esxi
910
911
9124 vm's
v1 and v2 are windows 10 machines to simulate traffic over the bridge
w1 is the admin lan connection
And pf is in the middleFor 1 brief second, I get a good ping during a reboot of the pfsense server.
Now it makes me think that there is something filtering this traffic in pfsense but without anything in the logs and only arp requests found in a packet capture.
What could i look for to figure this out?
-
@juesor Found this very recent link from Broadcom I want to share, maybe it can help: https://techdocs.broadcom.com/us/en/symantec-security-software/web-and-network-security/proxysg/6-7/Overview_ISG_SGW_VA/ISG_SWG_VA_before_you_begin/ISG_SWG_VA_create_a_virtual_switch.html
-
That's just outlining vSwitchs in ESXi.
I'm curious about the 1 good ping during the reboot. It's almost as if the firewall was down and came up and then started blocking the flow.
-
@juesor I shared because of this statement:
However, if you have one good ping, you'll need to find what's blocking ... note I said a firewall rule in my first response.
-
That doesn't explain that when I do "pfctl -d" pings still fail.
I wouldn't expect bridging the interfaces had anything to do with the firewall service.
And the fact that the FW log doesn't show any block's.
And the fact that packet capture only shows ARP from both sides but not ICMP received.
-
@juesor said in Bridging two interfaces WAN(vRouter) & LAN(LAN Router) w/ OPT1(MGMT):
And the fact that packet capture only shows ARP from both sides but not ICMP received.
I believe by default the firewall doesn't accept ICMP on WAN. So, one would need to add a rule Action : Pass , Interface : WAN , Protocol : ICMP , Source Type : Any, and Destination : WAN address. Although you're only using pfSense as a bridge, you still need to pass traffic from WAN to the bridge. So, that's why you'll have one good ping at the NIC but as soon as it reach the firewall, it gets shutdown.
-
-
@juesor said in Bridging two interfaces WAN(vRouter) & LAN(LAN Router) w/ OPT1(MGMT):
But there is no address on the wan interface so WAN address throws an !
Did all resources you have looked at as guide had WAN with an IP address? I'll visit the spiceworks and Lawrence sources again to confirm. Okay, from the spicework source note WAN has an IP.
Also notice from the Lawrence video, WAN has IP ... it's just the bridge (transparent) that doesn't have an IP.
