Need assistance with simple inter LAN routing
-
This is a stupid problem to have but still, it is a problem nonetheless. I'd appreciate any assistance. Here is a simplified description of a network I am working on:
WAN3 –-----+
WAN1 ---- PFSENSE FW1 ---- WAN2
LAN1 --------+
So pfSense FW1 has three WAN type connections (WAN1 and WAN2 is in load balanced config).
LAN1 is 192.168.0.1/24
On the same LAN1 segment, there is another pfSense FW2:
LAN1 ------ PFSENSE FW2 --------- LAN2
LAN1 has IP 192.168.0.2
LAN2 is 192.168.1.1/24
On LAN2 I have a PC2, IP 192.168.1.100.
On LAN1 there is a PC1, IP 192.168.0.30. It listens on several ports, 1433, 443 etc.
So to make FW2 a router essentially, I set up LAN1 and LAN2 on FW2 with no gateways. There are no custom routes as I do not see why that would be needed.
On FW1 I did add a static route and an additional gateway GW2 (GW1 is for WAN - unrelated to this discussion). The GW2 has IP 192.168.0.2 (i.e. pointing to FW2's LAN1 IP). The route has network 192.168.1.0/24 and GW2.
On FW1 I added on LAN1 interface a firewall rule allow all from source LAN1 net to 192.168.1.0/24.
On FW2 I added on LAN1 interface a firewall rule allow any to any. On FW2 LAN2 I added the same rule.
The idea, as I understand routing, is that if a PC1 connected to the same network segment on LAN1 than FW1 is located, the traffic would flow as follows:
PC1 tries to connect to PC2 port 1433. It sends a TCP SYN packet to the default gateway as PC2 is on a different subnet than PC1. Default GW is FW1 LAN1 IP, so packet goes to 192.168.0.1. FW1 sees that it is destined for PC2 on 192.168.1.100, and the static route I added tells it that subnet 192.168.1.0/24 is reachable via gateway 192.168.0.2 (FW2 LAN1 IP). FW2 gets the packet, and knows that subnet 192.168.1.0/24 is on LAN2, so it sends it off to that network adapter. The SYN packet arrives at PC2. PC2 replies with TCP SYN/ACK, the packet travels to FW2 LAN2 IP (as this is the default GW on PC2), FW2 knows PC1 is on the LAN1 interface and forwards the packet onwards. The switches figure out how to get the reply packet to PC1. PC1 sends a ACK via the same path as before and the connection is established. For me, this is partially working - I can connect but the connection is frequently interrupted).
The opposite - PC2 to PC1 - is NOT working. I can ping PC1 from PC2. I can traceroute from PC2 to PC1. But TCP packets do not work. Specifically, the SYN arrives at PC1 from PC2, PC2 replies with SYN,ACK which arrives at FW1 but it never reaches FW2.
To help myself, I'll trace the flow as I understand it:
PC2 tries to connect to PC1 port 1433. It sends a TCP SYN packet to the default gateway as PC1 is on a different subnet than PC2. Default GW is FW2 LAN2 IP, so packet goes to 192.168.1.1. FW2 sees that it is destined for PC1 on 192.168.0.30, and sends it off to interface LAN1. PC1 receives the SYN packet due to the switches finding the path (same subnet). PC1 replies with TCP SYN/ACK, the packet travels to the default gateway as PC2 is connected on a different subnet than PC1. Default GW is FW1 LAN1 IP, so packet goes to 192.168.0.1. FW1 sees that it is destined for PC2 on 192.168.1.100, and the static route I added tells it that the subnet 192.168.1.0/24 is reachable via gateway 192.168.0.2 (FW2 LAN1 IP). [Here the packet gets lost as a tcpdump shows the packet never arrives at FW2]. I would have expected the packet to arrive at FW2 LAN1, then gets routed to FW2 LAN2 to PC2.
My guess is that since pfSense is a stateful firewall, even though I have an allow any to any rule, it still needs to keep its state table intact. So for a TCP handshake, each FW needs to see the SYN, SYN/ACK and ACK packets to know the connection is established. Since only one part of this traffic flows through the firewall and the rest is delivered straight via the switches, the firewall drops the packet.
My questions are: Is this assumption above correct? And how do I fix this? How should I set up a simple setup like this so that I can access PC1 from PC2 and vice versa?
Is the solution as simple as adding a static route to FW2, for subnet 192.168.0.0/24, with gateway FW1 LAN1 (192.168.0.1)? This way the packets are forced to flow through both firewalls?
-
It is the same thing again and again and again and again.
Don't put hosts on the same network used for router-to-router transit.
You have an asymmetric routing problem due to poor network design.
-
It is the same thing again and again and again and again.
Don't put hosts on the same network used for router-to-router transit.
You have an asymmetric routing problem due to poor network design.
It is not the same thing again and again for me… First time I ran into this issue, so some advice would be much appreciated. If I am not to place hosts on this LAN segment where the two routers are connected, how do I then configure an (existing) network where I already have:
WAN --- FW1 --- LAN1
And then need to have an additional subnet LAN2, where some devices on LAN2 can communicate with some devices on LAN1, and some devices on LAN1 being able to communicate with some devices on LAN2, but the rest being isolated from one another? I cannot fiddle with LAN1. How would I add LAN2 to this network assuming I cannot add additional NICs to FW1 and that VLAN is not supported?
-
How would I add LAN2 to this network assuming I cannot add additional NICs to FW1 and that VLAN is not supported?
In a couple of words, "you don't."
Network expansion sometimes requires investment in some gear. If a small, managed switch that does dot1q wasn't $30 on amazon your scenario might carry more weight.
Yes there are some checkboxes that might help you work around an inherently-flawed design but I will leave that to others.
Hint: see System > Advanced, Firewall & NAT, Static route filtering
-
Thanks - this is helpful. So you recommend for the configuration I want, either have no hosts on the router to router segment (transit network), or make use of VLANs or additional NICs in the firewall to have one router as opposed to two?
-
Yes.
