Pfsense cannot port forward to Layer 3 switch
-
I have a Pfsense (2.7.2) connected to a Brocade 6450 for testing, final platform will be Brocade 7250 stack, but should be pretty similar.
The Brocade is setup to handle 90% of the inter-vlan routing so that traffic does not have to go back between 2 buildings to hit a 1GB link to the pfsense and come back to the switch. I already have full pings between all subnets using a Transit VLAN and static routes on pfsense, DHCP coming from Windows server on all VLANs using ip-helper. Pfsense can ping any device on any VLAN, and they all have working internet.
Problem is I cannot port forward from the pfsense wan to the remote subnet on the Brocade. This is on a lab system, not production, I tested just using RDP, RDP works fine inter-vlan between any hosts on any vlan, but it will not work for port forward from WAN. My diagram has an example showing a web server.
Note - this is not double NAT, no ACL on Brocade, so this should be fine.
- Core Switch = 10.0.x.2 on all VLANs
- VLAN1 10.1.1.1/24 (Router igb1 untagged only)
- VLAN77 = Transit 10.77.0.0/29 (Router igb2 tagged)
- VLAN10 = Servers 10.0.10.0/24
- VLAN20 = Data 10.0.20.0/23 (untag all ports)
- VLAN30 = Data Wifi 10.0.30.0/24
- Inter-VLAN Routing 10+20+30
- VLAN40 = GuestWifi 10.0.40.0/24
All clients get core 10.0.x.2 as default gateway. Pfsense has static routes for all VLANs set to 10.77.0.2.
Brocade conf: https://pastebin.com/6DvMFAq9
-
@totalimpact
RDP also needs UDP. -
@viragomann
Been doing this professionally for 20+ years - never forwarded UDP for RDP, but to humor you-
Still no. I believe Outbound NAT needs to be disabled for the Transit network, which cannot happen it Automatic or Hybrid NAT are on, so I setup manual Outbound NAT mappings on the WAN port for all those subnets, but still no.
PCAP on pfsense shows packets leaving on the transit network interface, but the Brocade does not seem to receive them, its kind of noisey on the pcap and I'm trying to get it all, so hard to see.
-
@totalimpact said in Pfsense cannot port forward to Layer 3 switch:
I believe Outbound NAT needs to be disabled for the Transit network,
Yes, but pfSense applies NAT only to interfaces, which has an upstream gateway stated in the settings.
You should not have a gateway on the transit network interface.Apart from this, it's possible to disable the Outbound NAT for certain destinations in hybrid mode with "no-NAT" rules.
PCAP on pfsense shows packets leaving on the transit network interface, but the Brocade does not seem to receive them
Sure? Did you do a PCAP on the Brocade as well to verify?
Due to the states I'd suspect, that you have an asymmetric routing issue, which would mean, that the request packet is forwarded properly to the server, but response packets take another path.
Check the routing tables on all involved devices to investigate this.
BTW: For the Transit you stated 10.77.0.0/29 above, but on the Brocade you have configured 10.77.0.0/25. But I don't think, that this matters here. It would just affect broadcast traffic.
-
@viragomann said in Pfsense cannot port forward to Layer 3 switch:
asymmetric routing issue
You cannot do that - having static routes requires a gateway on the Transit network. To remove the gateway you would need to first delete the static routes, then pfsense wont know how to find them.
I definitely agree it is an asymmetric routing issue, just not sure where to address it.
pcap on Brocade is jumping thru some hoops. I ran one, but I am not setup to analyze the entire thing, I can only get small clumps at a time, in which I have not seen anything referencing my traffic. I will get it setup to grab the entire pcap so I can be sure.
-
@totalimpact said in Pfsense cannot port forward to Layer 3 switch:
having static routes requires a gateway on the Transit network.
Not on the interface - you create a gateway to the IP on the transit network, but you don't actually put that gateway on the interface of pfsense on the transit.. Or pfsense thinks a wan interface and creates an outbound nat on it.
You create the gateway in the routing gateway section not on the specific interface.
