Transparent firewall with multiple subnets asymmetric routing issue
-
Hello, we are having an issue where TCP traffic get's dropped between servers "behind" a transparent PFSense firewall.
We have two completely different /24 of public IP addresses, let's call them A.A.A.0/24 and B.B.B.0/24.
The pfSense is setup as a transparent firewall, with the gateways (A.A.A.1 and B.B.B.1) of these public IP spaces residing on a router connected to the physical WAN port and all devices using public IP's connected to the physical LAN port (bridged as OPT1).
PFSense itself has an IP A.A.A.250 on OPT1 for management and unter system/routing/gateway A.A.A.1 as default gateway.All devices of A.A.A.0/24 and B.B.B.0/24 can communicate to and from the internet (if the rules allow it) and A.A.A.20 can ping B.B.B.20. The problem is when A.A.A.20 tries to open a TCP connection (like SSH) to B.B.B.20.
My preferred solution would be to tell PFsense that it can route traffic from A.A.A.20 to B.B.B.20 without ever hitting the gateway on the WAN side and I tried this by adding a static route for B.B.B.0/24 under System/Routing/Static Routes and giving it a gateway of Null4-127.0.0.1 but that didn't do anything. I also tried to simple add a second gateway B.B.B.1 on OPT1 to System/Routing/Gateways but this didn't change anything either.
A.A.A.0/24 and B.B.B.0/24 are connected over a single physical cable to the router on the carrier side, but I can not even get the traffic to go out of PFSense to the router and then come back in ...if I do the same with two servers directly connected to the edge switch, the two networks can communicate, this means the router is definitely not the issue here and it seems PFSense does not like to handle TCP traffic this way either.
The only thing that works at the moment is if I assign the server A.A.A.20 an additional IP B.B.B.21 and the server B.B.B.20 and IP A.A.A.21 ...obviously they can then communicate directly and without ever hitting the firewall at all. Unfortunately this is not practical since we would essentially cut our usable IP's in half.
Does anybody have an idea to get the two /24's to talk to each other? -Thanks much!