Asymetric Routing



  • Hi,

    We have a failover multiwan connection with 2 internet connections.  PFsense is version 2.0.0, but is being upgraded to 2.0.1 tomorrow.

    Connection 1 is a Comcast cable modem, this is the Tier 1 connection and all of our office traffic should route over Comcast as long as it's up.

    Connection 2 is an AT&T DSL line that is set as Tier 2 and is used when Comcast dies (Failover works and has worked live in production twice so far):

    This PFSense box runs OpenVPN Server for remote access to our office.  I have 2 OpenVPN servers configured.  The one on the Comcast connection is the primary OpenVPN, the one on the AT&T Circuit is supposed to be the backup.  The idea is that if Comcast dies, clients will move to the AT&T Circuit after a few minutes of downtime.  I have rules set on the Comcast interface that route the Comcast OpenVPN traffic to the Comcast gateway, and rules for the AT&T traffic to go to the AT&T gateway.

    However…

    When I try to connect to the AT&T VPN, tcpdump sees the incoming OpenVPN packets on the AT&T interface.  The replies are routed out the Comcast interface to the Comcast gateway instead of out the AT&T interface to the AT&T gateway.  Asymetric routing anyone?

    I'm not sure if this is a bug or a configuration problem.

    This is the pf rule that that is created for the Comcast interface (X.X.X.X = Comcast Gateway Y.Y.Y.Y = OpenVPN Listen IP):

    pass in quick on em0 route-to (em0 X.X.X.X) inet proto udp from any to Y.Y.Y.Y port = 1195 keep state label "USER_RULE: OpenVPN"

    This is the pf rule that is created for the AT&T gateway as seen with pfctl -s rules (Z.Z.Z.Z = AT&T gateway):

    pass in quick on em2 route-to (em2 Z.Z.Z.Z) inet proto udp from any to any port = 1196 keep state label "USER_RULE"

    Thanks,

    Dan



  • @dferris93:

    I have rules set on the Comcast interface that route the Comcast OpenVPN traffic to the Comcast gateway, and rules for the AT&T traffic to go to the AT&T gateway.

    Sounds like you have two issues here. One, sounds like you're specifying a gateway on WAN rules, which is wrong. That's forcing OpenVPN traffic back up to your router on each connection. Don't specify a gateway on WAN rules.

    Two, not enough info there to tell for sure but it sounds like you have two OpenVPN servers with the same tunnel network. Can't do that, use different networks for each.



  • I should take a screenshot, but my 2 OpenVPN servers don't have the same tunnel network.  One is set to 10.55.0.58/24 and the other is 10.55.0.59/24.

    I will set the gateway to * on the WAN interfaces and see what happens.



  • Thanks for the response, I got it to work by removing the gateway from the rules and setting the gateway in the Interface config.  I see that the rule is not a reply-to instead of route-to.

    Dan



  • Dan,
    Care to go into more detail as to how you got it working.

    Is there any way to get the reply traffic route via a specific gateway without using the routing table.
    The reason I'm asking is that I would like the return traffic out of an interface to use a gateway group.
    I've found that the 'gateway' field in the firewall rules only apply to traffic generated on that side. Any return traffic that goes through will always use the routes in the routing table and not the rule.

    Any thoughts?

    -E



  • @eytanes:

    Is there any way to get the reply traffic route via a specific gateway without using the routing table.
    The reason I'm asking is that I would like the return traffic out of an interface to use a gateway group.
    I've found that the 'gateway' field in the firewall rules only apply to traffic generated on that side. Any return traffic that goes through will always use the routes in the routing table and not the rule.

    That's a much different scenario than this one, the reply-to is automatically added to WAN rules which takes care of that. The exception being where you have multiple routers on the same interface, then reply-to is only set for the one chosen as the gateway on that interface. Disabling reply-to is at times a work around for that.

    Please start a new thread with a description of what you're trying to do for further feedback.


Locked