Static route vs setting gateway in rules



  • Hello,

    I have the following setup:

    LAN 10.6.0.0/16 <–-> Local pfSense <---> Internet <---> Remote pfSense <---> LAN 192.168.2.0/24

    IP of LAN interface of local pfSense: 10.6.0.253/16
    IP of LAN interface of remote pfSense: 192.168.2.1/24

    There is an IPSec VPN between the 2 pfSense boxes.

    Now, we are testing a WAN optimisation (WO) solution, which is installed on a PC with IP 10.6.0.250 locally and another instance is installed on the remote LAN with IP 192.168.2.5. Both WO devices have a single NIC and operate in what is called "server mode".

    A UDP tunnel is built between the 2 WO devices inside the IPSec VPN tunnel.

    Traffic is redirected to these devices and the packets are either optimised or not. If optimised, they go into the UDP tunnel, otherwise they are sent to the firewall LAN interface.

    There is an FTP server on the remote LAN with IP 192.168.2.21, having its gateway set to the IP of the WO device there, ie, 192.168.2.5.
    I have an FTP client locally on 10.6.1.180.

    I need to make all traffic going to the remote LAN go to the local WO device first.

    I believe there are a few ways to do this:
    1. I can set the gateway of all local machines to be 10.6.0.250 (IP of local WO device), but then I will not be able to control traffic on the firewall. I don't want this.
    2. In "System -> Routing -> Gateways", I can create a GATEWAY with IP=10.6.0.250 (IP of local WO device) on the LAN interface of the local pfSense, then put a rule on the LAN interface with SRC=10.6.0.0/16, DEST=192.168.2.0/24, then set the GATEWAY=10.6.0.250 (IP of local WO device).
    3. I can create the GATEWAY as in the 2nd option above, then I create a static route with DESTINATION NETWORK=192.168.2.0/24 and set the GATEWAY=10.6.0.250 (IP of local WO device).

    As I said, I don't want to use method 1.
    I've tried method 2, but the FTP transfers start, stop and then restart, without completing at all. I noticed a few packets being blocked by the local pfSense.
    I've tried method 3 and everything is working fine.

    My questions:
    1. Is there any other method to achieve what I explained above?
    2. When an incoming packet on the LAN interface matches an existing static route on pfSense, do the rules on the interface apply?
    3. What is the order of processing: are static routes processed before the rules on the interface?
    4. What is the difference between methods 2 & 3 in terms of states created in the state table?
    5. With method 3 (static routing), I have lost a bit of control on the matching packets, for example, I cannot deny a particular IP address access to the remote network. Is there a way to regain that control?

    Any help is appreciated.

    Thanks.



  • I am surprised that method 3 works. I would have thought that with method 2 and 3 the WO device would have trouble setting up the UDP tunnel. 10.6.0.250 would try to connect to 192.168.2.5 - that tunnel setup packet would get to pfSense, which would route it back to 10.6.0.250, the WO device itself. The packet would be dropped because the tunnel is not available - catch-22.
    You want pfSense to do ordinary routing for traffic from 10.6.0.250 to 192.168.2.5 - the tunnel packets themselves - so put an ordinary pass rule for those on LAN. Then follow with the method 2 policy route which will apply to the rest of the LAN traffic arriving to pfSense.
    Q2 and 3 - rules and routes are quite separate things. The rules are processed, if the packet matches a pass rule with no special gateway selected, then pf lets it through to the ordinary routing table, which has any static routes, routes known by VPN settings, local networks… If the packet matches a pass rule with a gateway specified, then the packet goes to that gateway, ignoring the ordinary routing table.
    Q5 - after reading the above, you will see that you can use ordinary block rules to stop anything you like, then the packets are stopped at the packet filter and never get to either way of routing.
    The problem I see for you is "If optimised, they go into the UDP tunnel, otherwise they are sent to the firewall LAN interface" - the "otherwise" - the source IP of these packets will, presumably, be the IP of the LAN client that originated them. So pfSense will see the packet again, apply the same rules, and route it back to WO - a loop. How can pfSense know that this is 2nd time it has seen the packet, so to process it differently this time???



  • Thanks for your reply.

    I made 2 mistakes in my post:
    1. In method 2, "DEST=192.168.2.0/24" is wrong. I actually have "DEST=192.168.2.21/32". The latter is the IP address of the FTP server to which I connect for testing.
    2. In method 3, "DESTINATION NETWORK=192.168.2.0/24" is also wrong. It actually is "DESTINATION NETWORK=192.168.2.21/32".

    Apologies for these mistakes. I'm currently testing with traffic going to that FTP server.
    If successful, I'll have to send all traffic to the WO device.

    Also, I forgot to mention that I have a rule on the LAN interface, which is as follows:
    PASS protocol=UDP src=10.6.0.250 destination=192.168.2.5

    Regarding your statement: "the source IP of these packets will, presumably, be the IP of the LAN client that originated them", as far as I'm aware, the IP address of all non-optimised (and optimised) packets = IP address of WO device, i.e., 10.6.0.250.
    That's what makes it complicated.

    Thanks again for your reply.



  • I believe I have found the cause of the problem. I needed to add the following rules on both pfSense boxes:

    Local pfSense:
    1. PASS protocol=GRE src=10.6.0.250 dest=PUBLIC_IP_OF_REMOTE_WO_DEVICE
    2. PASS protocol=IP src=10.6.0.0/16 dest=192.168.6.0/24 gateway=10.6.0.250

    Remote pfSense:
    1. PASS protocol=GRE src=192.168.6.190 dest=PUBLIC_IP_OF_LOCAL_WO_DEVICE
    2. PASS protocol=IP src=192.168.6.0/24 dest=10.6.0.0/16 gateway=192.168.6.190



  • Well, it turns out I was wrong again :-(.

    After resetting the states, it seems that PBR by setting a gateway on a rule does not work well in this situation.
    However, I confirm that when I use a static route, everything works fine.

    What is the difference?

    Any help please?


Log in to reply