Bridging two interfaces WAN(vRouter) & LAN(LAN Router) w/ OPT1(MGMT)

[A Former User]

@juesor

First of all, you must examine your Layer2 Topology and ensure you won't create any Layer2 Loop with this Setup. If all Uplinks of the ESXi are connected to the same Layer2 Segment proceed with cautions. Spanning-Tree won't save you here, cause STP-BPDUs are filtered on the ESXi per default.

Spoiler

You need to allow Forged transmits and Promiscuous Mode on the used Port Groups for the bridged interfaces. This is a necessary configuration on the Virtualization Platform to make this setup work. After Changing these settings you have to either reboot the firewall or disconnect and reconnect the interfaces to allocated a new virtual port with the updated settings.

Better to configure MAC-Learning for these Port-Groups. This Option is available on Distributed Switch since version 6.6.0 through API Calls.

[juesor]

@artes

Thanks. I set promiscuous mode on the vlan's within ESXi.

It still isn't working.

So i stood up a new lab.

3 vlans in esxi

910
911
912

4 vm's

v1 and v2 are windows 10 machines to simulate traffic over the bridge
w1 is the admin lan connection
And pf is in the middle

For 1 brief second, I get a good ping during a reboot of the pfsense server.

Now it makes me think that there is something filtering this traffic in pfsense but without anything in the logs and only arp requests found in a packet capture.

What could i look for to figure this out?

[NollipfSense]

@juesor Found this very recent link from Broadcom I want to share, maybe it can help: https://techdocs.broadcom.com/us/en/symantec-security-software/web-and-network-security/proxysg/6-7/Overview_ISG_SGW_VA/ISG_SWG_VA_before_you_begin/ISG_SWG_VA_create_a_virtual_switch.html

[juesor]

@nollipfsense

That's just outlining vSwitchs in ESXi.

I'm curious about the 1 good ping during the reboot. It's almost as if the firewall was down and came up and then started blocking the flow.

[NollipfSense]

@juesor I shared because of this statement:

However, if you have one good ping, you'll need to find what's blocking ... note I said a firewall rule in my first response.

[juesor]

@nollipfsense

That doesn't explain that when I do "pfctl -d" pings still fail.

I wouldn't expect bridging the interfaces had anything to do with the firewall service.

And the fact that the FW log doesn't show any block's.

And the fact that packet capture only shows ARP from both sides but not ICMP received.

[NollipfSense]

@juesor said in Bridging two interfaces WAN(vRouter) & LAN(LAN Router) w/ OPT1(MGMT):

And the fact that packet capture only shows ARP from both sides but not ICMP received.

I believe by default the firewall doesn't accept ICMP on WAN. So, one would need to add a rule Action : Pass , Interface : WAN , Protocol : ICMP , Source Type : Any, and Destination : WAN address. Although you're only using pfSense as a bridge, you still need to pass traffic from WAN to the bridge. So, that's why you'll have one good ping at the NIC but as soon as it reach the firewall, it gets shutdown.

[juesor]

@nollipfsense

But there is no address on the wan interface so WAN address throws an !

[NollipfSense]

@juesor said in Bridging two interfaces WAN(vRouter) & LAN(LAN Router) w/ OPT1(MGMT):

@nollipfsense

But there is no address on the wan interface so WAN address throws an !

Did all resources you have looked at as guide had WAN with an IP address? I'll visit the spiceworks and Lawrence sources again to confirm. Okay, from the spicework source note WAN has an IP.

Also notice from the Lawrence video, WAN has IP ... it's just the bridge (transparent) that doesn't have an IP.

[juesor]

@nollipfsense

Yea thats for bridging and using WAN as the gateway for traffic.

In my case i'm just using pf to filter traffic between two locations.

The whole transparent bridge i shouldn't need to IP anything in pfsense outside of my mgmt interface.

I moved the two VM's to a separate vlan 913 and they can ping each other.

Which shows connectivity. Now i'll move them back to the separate vlans 910 & 911 which are part of the bridge 910 = OPT1(vmx2) and 911 = WAN(vmx0)

[juesor]

@juesor

Ok so when i moved it back you can see that 2 pings worked 2 failed.

PF shouldn't be this willy nilly about things.

[juesor]

@juesor

Ok this is the kick in the pants.

If every vm is on the same host it works. i can ping across the bridge.

vmotion kept moving stuff around.

Now I don't know how this is going to work by moving back to vyos, pfsense, and my 3850 as the 3850 is physical but we will see once i wrap my head around this.

I was running on the cli of pfsense "tcpdump -i bridge0 host 192.168.1.1" and it started working like a Christmas miracle.

[NollipfSense]

@juesor said in Bridging two interfaces WAN(vRouter) & LAN(LAN Router) w/ OPT1(MGMT):

I was running on the cli of pfsense "tcpdump -i bridge0 host 192.168.1.1" and it started working like a Christmas miracle.

Congratulations, your persistence paid off!

[juesor]

@nollipfsense

One down side is I downgraded to 2.4.4 and it worked i put this on my production setup and it's not working.

I confirmed all my settings were exactly the same as the 2.4.4.

There is no traffic on the bridge interface but the ICMP is found on the vmx0 and vmx2 interfaces.