Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    [SOLVED] Policy-Based Routing Not Consistently Going Out the Specified Gateway

    Scheduled Pinned Locked Moved OpenVPN
    42 Posts 4 Posters 9.9k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • luckman212L
      luckman212 LAYER 8
      last edited by

      Are your OpenVPN clients set to "Don't pull routes" ? (they should be…)

      1 Reply Last reply Reply Quote 0
      • F
        Finger79
        last edited by

        Here's some of the LAN rules (farther down the rule list) for most of the traffic.  For example, the "Allow Web Traffic" rule sends all 80/443 traffic out the VPN gateway.

        lan_pass_rules.png
        lan_pass_rules.png_thumb

        1 Reply Last reply Reply Quote 0
        • F
          Finger79
          last edited by

          @luckman212:

          Are your OpenVPN clients set to "Don't pull routes" ? (they should be…)

          I've played with that setting in the past and had undesirable results.  Similar to the guy in this thread, my real IP started leaking instead of going out the VPN.

          I'll try it again.

          1 Reply Last reply Reply Quote 0
          • luckman212L
            luckman212 LAYER 8
            last edited by

            I think you definitely need route-nopull to be checked. If your DNS is "leaking" then you need to adjust the DNS settings on your Tablet so the lookups policy route via the tunnel. If you're using pfSense as the DNS server for the tablet, this isn't going to work since Unbound or dnsmasq upstream queries will be sourced from the firewall itself. The simple fix for that is to manually set a DNS server on the tablet and make sure your policy route traps that traffic (udp/53).

            1 Reply Last reply Reply Quote 0
            • F
              Finger79
              last edited by

              No, not DNS leaking, all traffic leaking such as through 80/443.  Even though I have a "NO_WAN_EGRESS" policy-based filtering setup, it doesn't seem to work when I don't pull routes from the VPN provider.  (Very not cool!)

              I'm getting inconsistent results right now on my laptop on various "What is my IP?" sites.

              1)  Real WAN IP address (undesirable)
              2)  VPN connection 1 (expected)
              3)  VPN connection 2 (expected)

              My DNS does not appear to be leaking at all, which is good.  All DNS through unbound goes out through the VPN, as configured.

              1 Reply Last reply Reply Quote 0
              • luckman212L
                luckman212 LAYER 8
                last edited by

                Ah ok sorry I saw "leaking" and jumped to conclusion you were talking about DNS since that is usually what people refer to when talking about leaks.

                Did you try the pfctl commands from a few posts ago? And you double checked your outbound NAT rules?

                1 Reply Last reply Reply Quote 0
                • F
                  Finger79
                  last edited by

                  I reverted back to having both OpenVPN client connections pulling routes.  Having them not pull routes was 100 times more undesirable since it exposed my entire LAN through the normal gateway and the VPN gateway randomly.

                  1 Reply Last reply Reply Quote 0
                  • F
                    Finger79
                    last edited by

                    @luckman212:

                    Ah ok sorry I saw "leaking" and jumped to conclusion you were talking about DNS since that is usually what people refer to when talking about leaks.

                    Did you try the pfctl commands from a few posts ago? And you double checked your outbound NAT rules?

                    Not sure what to look for in the outbound NAT rules.
                    LAN to WAN
                    LAN to VPN1
                    LAN to VPN2

                    I don't have a NAT rule just for the .103 exception… should be included in the above 3 rules.

                    I did the pfctl command and got inconsistent results.  Sometimes it would go out the VPN and other times it would go out the WAN.

                    1 Reply Last reply Reply Quote 0
                    • DerelictD
                      Derelict LAYER 8 Netgate
                      last edited by

                      You people who want to take a complicated setup like policy routing multiple openvpn connections then blame the software when it doesn't work yet obviously have no real grasp on what really needs to happen to make it work simply floor me.

                      In order for tagging and matching along the NO_WAN_EGRESS vein to work, EVERY packet that should go over the VPN must be tagged.

                      That is going to be a crap shoot without enabling "don't pull routes."

                      You are going to have to know exactly how to structure your rules in either case.

                      Two choices:

                      Enable "don't pull routes" and policy route VPN traffic

                      Don't enable "don't pull routes" and policy route clear internet traffic.

                      With multiple VPN providers, not enabling "don't pull routes" is going to be very complicated because they will both want to enable the 0/1 and 128/1 rules.

                      Chattanooga, Tennessee, USA
                      A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                      DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                      Do Not Chat For Help! NO_WAN_EGRESS(TM)

                      1 Reply Last reply Reply Quote 0
                      • luckman212L
                        luckman212 LAYER 8
                        last edited by

                        Anything in your OpenVPN logs?

                        My brain started working again & I remembered that you are trying to put your default LAN behind the OpenVPN tunnel and just make an exception for the Tablet.  That's the reverse of what I do (I have an alias for devices that I want "hidden" and everything else just uses default routing).  So in your case you should probably set the default route of your pfSense is set to the VPN gateway, otherwise your DNS traffic will leak – or, you can set DHCP to hand out an upstream DNS server to LAN clients e.g. 8.8.8.8, etc.

                        As to why you are getting inconsistent results, I am at a bit of a loss. Maybe this needs a fresh set of eyes. Anyone else got any ideas?

                        1 Reply Last reply Reply Quote 0
                        • F
                          Finger79
                          last edited by

                          @Derelict:

                          You people who want to take a complicated setup like policy routing multiple openvpn connections then blame the software when it doesn't work yet obviously have no real grasp on what really needs to happen to make it work simply floor me.

                          In order for tagging and matching along the NO_WAN_EGRESS vein to work, EVERY packet that should go over the VPN must be tagged.

                          That is going to be a crap shoot without enabling "don't pull routes."

                          I think you're misunderstanding.  The policy-based filtering floating rule works perfectly (matches the previously tagged packets).  That's not what this thread is about.

                          @Derelict:

                          In order for tagging and matching along the NO_WAN_EGRESS vein to work, EVERY packet that should go over the VPN must be tagged.

                          Yep, no issues here.  Tagging is working as expected.

                          @Derelict:

                          You people who want to take a complicated setup like policy routing multiple openvpn connections

                          The VPN connections are in one gateway group.  The policy-route is set to send all traffic out the VPN gateway.  That's working perfectly.

                          The only thing that's not consistently working is the policy-route for one device on the LAN to go out the normal WAN gateway.

                          1 Reply Last reply Reply Quote 0
                          • DerelictD
                            Derelict LAYER 8 Netgate
                            last edited by

                            OK where is that rule in relation to all your other rules? You have yet to show that.

                            I was more commenting on the nonsense like this:

                            No, not DNS leaking, all traffic leaking such as through 80/443.  Even though I have a "NO_WAN_EGRESS" policy-based filtering setup, it doesn't seem to work when I don't pull routes from the VPN provider.  (Very not cool!)

                            It works perfectly when configured correctly.

                            Chattanooga, Tennessee, USA
                            A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                            DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                            Do Not Chat For Help! NO_WAN_EGRESS(TM)

                            1 Reply Last reply Reply Quote 0
                            • F
                              Finger79
                              last edited by

                              @Derelict:

                              OK where is that rule in relation to all your other rules? You have yet to show that.

                              I was more commenting on the nonsense like this:

                              No, not DNS leaking, all traffic leaking such as through 80/443.  Even though I have a "NO_WAN_EGRESS" policy-based filtering setup, it doesn't seem to work when I don't pull routes from the VPN provider.  (Very not cool!)

                              It works perfectly when configured correctly.

                              Screenshots of that posted earlier.  The "Allow Web Traffic" rule sets the policy-based filtering tag "NO_WAN_EGRESS" and also sets the policy-based routing gateway to the VPN_Gateway.

                              The only time that traffic leaked out the naked WAN was when I told both client VPN connections to not pull routes.  Then I got inconsistent results:  some traffic went out the VPN, and other traffic went out the WAN.  It was random.

                              1 Reply Last reply Reply Quote 0
                              • F
                                Finger79
                                last edited by

                                @luckman212:

                                So in your case you should probably set the default route of your pfSense is set to the VPN gateway, otherwise your DNS traffic will leak – or, you can set DHCP to hand out an upstream DNS server to LAN clients e.g. 8.8.8.8, etc.

                                DNS seems to work perfectly.  Unbound sends all traffic through the VPN tunnels and never out the naked WAN.  (That interface is unchecked.)

                                1 Reply Last reply Reply Quote 0
                                • DerelictD
                                  Derelict LAYER 8 Netgate
                                  last edited by

                                  You are all over the place. that rule routes traffic to PIA for, presumably, destinations 80 and 443.

                                  All other traffic will go out the default gateway (or the OpenVPN connection that happens to have been able to set the 0.0.0.0/1 and 128.0.0.0/1 rules, which as you found in the other thread, will be the first OpenVPN connection without "don't pull routes" set that connects. The other one will receive errors when trying to set those routes.)

                                  You are indicating there is a problem with some other host that is unable to go out WAN. Where is that rule in relation to all the other rules?

                                  Not blocking out things that really don't matter might help people help you.

                                  Chattanooga, Tennessee, USA
                                  A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                                  DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                                  Do Not Chat For Help! NO_WAN_EGRESS(TM)

                                  1 Reply Last reply Reply Quote 0
                                  • F
                                    Finger79
                                    last edited by

                                    @Derelict:

                                    You are indicating there is a problem with some other host that is unable to go out WAN.

                                    Negative.  It's not unable to go out WAN.  It just doesn't do it consistently.

                                    @Derelict:

                                    Where is that rule in relation to all the other rules?

                                    Answered earlier:
                                    @Finger79:

                                    Here's some of the LAN rules (farther down the rule list) for most of the traffic.  For example, the "Allow Web Traffic" rule sends all 80/443 traffic out the VPN gateway.

                                    The .103 tablet exception rule matches first.  It just doesn't consistently send traffic out the WAN.  Sometimes it does, other times it doesn't.  But the rule always fires.

                                    1 Reply Last reply Reply Quote 0
                                    • DerelictD
                                      Derelict LAYER 8 Netgate
                                      last edited by

                                      Is WANGW flapping?

                                      System > Logs, Gateways

                                      Chattanooga, Tennessee, USA
                                      A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                                      DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                                      Do Not Chat For Help! NO_WAN_EGRESS(TM)

                                      1 Reply Last reply Reply Quote 0
                                      • F
                                        Finger79
                                        last edited by

                                        @Derelict:

                                        Is WANGW flapping?

                                        System > Logs, Gateways

                                        I don't see it mentioned anywhere in the Gateway logs.

                                        I had to turn off WANGW gateway monitoring (meaning it's always considered "Up").  The dpinger pings may have pissed it off.  I'll turn it back on and see if that helps.

                                        1 Reply Last reply Reply Quote 0
                                        • DerelictD
                                          Derelict LAYER 8 Netgate
                                          last edited by

                                          Right. If that was happening when you were seeing "random" routing then the same principles that make "NO_WAN_EGRESS" necessary would apply equally to WANGW if it was flagged as down. In that case you would need "NO_VPN_EGRESS."

                                          Chattanooga, Tennessee, USA
                                          A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                                          DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                                          Do Not Chat For Help! NO_WAN_EGRESS(TM)

                                          1 Reply Last reply Reply Quote 0
                                          • F
                                            Finger79
                                            last edited by

                                            I get that. Fortunately, gateway flapping appears to not be the reason behind this.

                                            When I do a fresh reboot of pfSense, the tablet traffic consistently goes out the WAN, as expected.  Some time later (or some event later), it decides to go out the VPN only, even though the rule still fires that specifies WANGW.  It's just ignoring the gateway in the rule but still logging the rule as having fired.  Weird.

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.