Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Loadbalancing + OpenVPN (Policy based routing); incoming connections not working

    Scheduled Pinned Locked Moved Routing and Multi WAN
    7 Posts 2 Posters 624 Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • V
      Volans
      last edited by

      Hi guys,

      I read just about every thread regarding Multi-WAN-OpenVPN-Loadbalancing but was not able to find a solution to my problem.

      I have several networks with clients behind an OpenVPN-Server (Debian + OpenVPN 2.3.4). Now I want to setup site-to-site OpenVPN connections from pfSense routers with more than one WAN connection. There are many tutorials for this on the internet like: https://nguvu.org/pfsense/pfsense-multi-vpn-wan/

      In short: it's creating one OpenVPN client for every WAN connection, creating an interface for every OpenVPN client, creating a gateway for every OpenVPN interface, creating a gateway group out of those gateways with the same tier and creating a firewall filter rule (with the gateway group specified) for traffic that should go to the networks behind the OpenVPN server. What I understand is that then I only have to create outbound NAT rules.. so that the sender address of the packets is the OpenVPN client address and the reply comes the same way the request was made (correct if I'm wrong).

      For loadbalancing to function I have to tell my OpenVPN clients that they shouldn't pull routes from the server (I did that in the client config), because we don't want to use the routing table.. we want the firewall filter rule to direct the traffic into the right gateway group. I found out that because of this I shouldn't use monitoring IPs in the networks behind the OpenVPN server, because pfSense then adds a static route to the routing table. For this I added two or more virtual interfaces on the OpenVPN server just for monitoring IPs.

      So much for that.. outgoing loadbalancing through multiple OpenVPN connection is working! :)

      And now the problem: incoming traffic through the VPN tunnels reaches the client, the client replies, the packet goes back to the pfSense.. but this time the pfSense doesn't care about the firewall filter rule where I said that the packet should go out through the OpenVPN gateway group.  :( I tested that with tcpdump on all hops.

      Maybe I forgot something or I just don't understand what I'm doing. I hope somebody can point me in the right direction! :)

      1 Reply Last reply Reply Quote 0
      • DerelictD
        Derelict LAYER 8 Netgate
        last edited by

        Policy routing is for connections being established into the interface the policy routing is on. (ie from a LAN client into a LAN interface with the policy routing rules)

        If you want connections coming in from OpenVPN from arbitrary locations (locations for which there isn't a kernel route for the reply traffic with that OpenVPN server as the gateway) then you need an assigned interface on the OpenVPN instance in question.

        You also need to be sure that the inbound connection is not matched by the rules on the OpenVPN tab, but is matched by rules on the assigned interface tab. That will get reply-to functioning for the reply traffic.

        I cannot recommend a pfSense Gold subscription, with video lectures (Hangouts) from the guys who literally wrote the book on pfSense, enough. There is one called Advanced OpenVPN Topics (or something close to that) that covers all of this and more. And that is but one of dozens.

        If that is too steep the aforementioned book is < $25. See my sig.

        Chattanooga, Tennessee, USA
        A comprehensive network diagram is worth 10,000 words and 15 conference calls.
        DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
        Do Not Chat For Help! NO_WAN_EGRESS(TM)

        1 Reply Last reply Reply Quote 0
        • V
          Volans
          last edited by

          Hi Derelict,

          thank you for your answer and for your recommendation. I thought about that too, but this month there's no money for that. I've got a book about pfSense already called "The definitive Guide", but theres only one section about that topic: "11.3.3. Local Services and Multi-WAN".

          There it says: "In the case of traffic initiated on the Internet destined for an OPT WAN interface, pfSense automatically uses pf's reply-to directive in all WAN and OPT WAN rules, which ensures the reply traffic is routed back out the correct WAN interface."

          I don't really feel this "automatically"-thing.. but maybe it's because of OpenVPN in this case? I don't know.

          I tried to follow your instructions: There is an assigned interface on both OpenVPN instances, because I needed them for creating a gateway group, right? Then I checked the OpenVPN tab under firewall rules: there was no rule. Then I went to the tabs for the OpenVPN assigned interfaces and created one rule on both, where I matched the incoming traffic with an alias for those networks behind the OpenVPN server. But I don't know how to further setup this rule. Should I specify a gateway in this rule? The interface itself? Or the gateway group? I tried both.. but it didn't work.

          1 Reply Last reply Reply Quote 0
          • V
            Volans
            last edited by

            I've now purchased the Gold Subscription and there it says under "Assigning OpenVPN Interfaces" that this enables several beneficial changes like: "Adds reply-to to rules on the VPN interface tab to help with return routing". But there is no auto created rule on that OpenVPN interface tab.. and I can't find any further infos about that.

            Then there is a section "OpenVPN Site-to-Site with Multi-WAN and OSPF" but this a extra package is required on both sites.. and the OpenVPN server is not a pfSense, it's Debian system.

            1 Reply Last reply Reply Quote 0
            • DerelictD
              Derelict LAYER 8 Netgate
              last edited by

              In order to get reply-to for connections coming into pfSense through OpenVPN, the incoming traffic MUST NOT match on the OpenVPN tab. If it does, that will prevail and that is where the state will be created and, being an interface group, not an interface, there will be no reply-to.

              The traffic MUST NOT be matched by the OpenVPN tab and MUST be passed by a rule on the assigned interface tab.

              How traffic is load balanced for connections coming into pfSense is determined by the other side.

              Chattanooga, Tennessee, USA
              A comprehensive network diagram is worth 10,000 words and 15 conference calls.
              DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
              Do Not Chat For Help! NO_WAN_EGRESS(TM)

              1 Reply Last reply Reply Quote 0
              • V
                Volans
                last edited by

                Thank you for your patience. I found the source you were talking about: it's in the Hangout archive in the video "Advanced OpenVPN Concepts" at 22:07.

                What I understood: the incoming traffic first hits the "OpenVPN" tab and then goes through the interface tabs of the corresponding OpenVPN instance. The reply-to will be added by the first rule that matches. If it matches a rule on the "OpenVPN" tab, then it gets a reply-to for the default gateway. If it matches a rule on the corresponding OpenVPN interface tab, then it gets a reply-to for this interface (exactly what we want).

                The thing that wasn't clear for me was that EVERY rule will do that, no matter how it is configured. So I checked "Floating rules" and the "OpenVPN" tab, deleted all rules there and created a "pass any to any" rule on every OpenVPN instance interface tab.

                Problem solved!  ;)

                1 Reply Last reply Reply Quote 0
                • DerelictD
                  Derelict LAYER 8 Netgate
                  last edited by

                  Just for clarity, rules that match the OpenVPN tab do not get reply-to at all so the replies are routed according to the routing table. That usually means they go out the default gateway.

                  Rules matching the assigned interface tab (which means they weren't matched by the OpenVPN tab or processing would have stopped there) get reply-to on the states.

                  Glad it's working.

                  Chattanooga, Tennessee, USA
                  A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                  DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                  Do Not Chat For Help! NO_WAN_EGRESS(TM)

                  1 Reply Last reply Reply Quote 0
                  • First post
                    Last post
                  Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.