Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Big problem with setting up second OpenVPN client connection

    2.2 Snapshot Feedback and Problems - RETIRED
    3
    8
    2.1k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • S
      sense678
      last edited by

      Hi,

      Happy new year to everybody!

      I was trying to get a second VPN client running today, to the same provider just a different server.
      So what I did is to create a new LAN (VPNLAN2) because this server should only be accessible from a special subnet.
      Then I created a client based on my first OpenVPN client, I just changed the server in it.

      After that I mapped ovpnc2 client to VPN_WAN2. I then went into routing an copied the gateway from VPN_WAN1 to VPN_WAN2 just changing the names.
      I then did an outbound rule base on VPN_WAN1 and changed the values accordingly to the VPNLAN2 subnet and VPN_WAN2 values.

      But i don't get this to work. The VPN provider shows the second connections in it's statistics so both connections are running from my home to them.
      Also the dynamic IP's in the Gateway section look correct. I can see nothing special in the openVPN logs. IN the dashboard widget both connections show as up, as I can also see in my VPN providers page.

      But on the dashboard the VPN_WAN2_GW is always shown as down and I am getting constantly these errors  in system logs :
      Jan 1 09:15:05 lighttpd[22961]: (network_openssl.c.118) SSL: 5 -1 1 Operation not permitted
      Jan 1 09:15:05 lighttpd[22961]: (connections.c.619) connection closed: write failed on fd 20
      Jan 1 09:15:05 lighttpd[22961]: (connections.c.1692) SSL (error): 5 -1 1 Operation not permitted
      Jan 1 09:15:05 lighttpd[22961]: (network_openssl.c.118) SSL: 5 -1 1 Operation not permitted
      Jan 1 09:15:05 lighttpd[22961]: (connections.c.619) connection closed: write failed on fd 17
      Jan 1 09:15:05 lighttpd[22961]: (connections.c.1692) SSL (error): 5 -1 1 Operation not permitted
      Jan 1 09:15:06 lighttpd[22961]: (network_openssl.c.118) SSL: 5 -1 1 Operation not permitted
      Jan 1 09:15:06 lighttpd[22961]: (connections.c.619) connection closed: write failed on fd 22
      Jan 1 09:15:06 lighttpd[22961]: (connections.c.1692) SSL (error): 5 -1 1 Operation not permitted
      Jan 1 09:15:07 lighttpd[22961]: (network_openssl.c.118) SSL: 5 -1 1 Operation not permitted
      Jan 1 09:15:07 lighttpd[22961]: (connections.c.619) connection closed: write failed on fd 25
      Jan 1 09:15:07 lighttpd[22961]: (connections.c.1692) SSL (error): 5 -1 1 Operation not permitted
      Jan 1 09:15:22 lighttpd[22961]: (connections.c.1692) SSL (error): 5 -1 1 Operation not permitted
      Jan 1 09:15:22 lighttpd[22961]: (connections.c.1692) SSL (error): 5 -1 1 Operation not permitted
      Jan 1 09:15:22 lighttpd[22961]: (connections.c.1692) SSL (error): 5 -1 1 Operation not permitted
      Jan 1 09:15:43 lighttpd[22961]: (network_openssl.c.118) SSL: 5 -1 1 Operation not permitted
      Jan 1 09:15:43 lighttpd[22961]: (connections.c.619) connection closed: write failed on fd 33
      Jan 1 09:15:43 lighttpd[22961]: (connections.c.1692) SSL (error): 5 -1 1 Operation not permitted
      Jan 1 09:15:44 kernel: ovpnc2: link state changed to UP
      Jan 1 09:15:44 check_reload_status: rc.newwanip starting ovpnc2
      Jan 1 09:15:44 lighttpd[22961]: (network_openssl.c.118) SSL: 5 -1 1 Operation not permitted
      Jan 1 09:15:44 lighttpd[22961]: (connections.c.619) connection closed: write failed on fd 18
      Jan 1 09:15:44 lighttpd[22961]: (connections.c.1692) SSL (error): 5 -1 1 Operation not permitted
      Jan 1 09:15:45 php-fpm[11678]: /rc.newwanip: rc.newwanip: Info: starting on ovpnc2.
      Jan 1 09:16:06 lighttpd[22961]: (connections.c.1692) SSL (error): 5 -1 1 Operation not permitted
      Jan 1 09:16:06 lighttpd[22961]: (connections.c.1692) SSL (error): 5 -1 1 Operation not permitted
      Jan 1 09:16:06 lighttpd[22961]: (connections.c.1692) SSL (error): 5 -1 1 Operation not permitted
      Jan 1 09:16:07 lighttpd[22961]: (network_openssl.c.118) SSL: 5 -1 1 Operation not permitted
      Jan 1 09:16:07 lighttpd[22961]: (connections.c.619) connection closed: write failed on fd 15
      Jan 1 09:16:07 lighttpd[22961]: (connections.c.1692) SSL (error): 5 -1 1 Operation not permitted
      Jan 1 09:16:10 kernel: ovpnc2: link state changed to UP
      Jan 1 09:16:10 check_reload_status: rc.newwanip starting ovpnc2
      Jan 1 09:16:11 php-fpm[2125]: /rc.newwanip: rc.newwanip: Info: starting on ovpnc2.
      Jan 1 09:16:33 lighttpd[22961]: (connections.c.1692) SSL (error): 5 -1 1 Operation not permitted
      Jan 1 09:16:33 lighttpd[22961]: (connections.c.1692) SSL (error): 5 -1 1 Operation not permitted
      Jan 1 09:16:33 lighttpd[22961]: (connections.c.1692) SSL (error): 5 -1 1 Operation not permitted
      Jan 1 09:16:34 lighttpd[22961]: (connections.c.1692) SSL (error): 5 -1 1 Operation not permitted
      Jan 1 09:16:40 lighttpd[22961]: (connections.c.1692) SSL (error): 5 -1 1 Operation not permitted
      Jan 1 09:16:40 lighttpd[22961]: (connections.c.1692) SSL (error): 5 -1 1 Operation not permitted
      Jan 1 09:16:40 lighttpd[22961]: (connections.c.1692) SSL (error): 5 -1 1 Operation not permitted
      Jan 1 09:16:40 lighttpd[22961]: (connections.c.1692) SSL (error): 5 -1 1 Operation not permitted
      Jan 1 09:16:41 lighttpd[22961]: (connections.c.1692) SSL (error): 5 -1 1 Operation not permitted
      Jan 1 09:16:41 lighttpd[22961]: (connections.c.1692) SSL (error): 5 -1 1 Operation not permitted

      This makes me crazy, a reboot didn't help at all. i tried this a few days before and it seemed that is was running on this older snapshot but can't tell it 100% because it was just a quick test.

      1 Reply Last reply Reply Quote 0
      • S
        sense678
        last edited by

        If i disable the VPN_WAN2 interface, remove the GW and disable the NAT rule, so there is only one VPN_GW1 left and a NAT for VPN_GW1, everything runs smooth, no error messages.
        And in the dashboard both openvpn clients show up as connected, also in the VPN service.

        But of course I can only use VPN1 then because there is no GW and NAT for VPN2.

        Sh…

        1 Reply Last reply Reply Quote 0
        • S
          sense678
          last edited by

          So, I was finally able to fix this, but only because my VPN provider luckily provides more than one connect solution. This seems to be really a bug in pfsense itself.

          As I said I had a perfectly working OpenVPN client, connected thorugh port 443 UDP to the VPN server A.

          So I had ovpnc1 assigned to a interface (VPNWAN1), created a gateway for it (VPNWAN1_GW) with dynamic address and a NAT rule for my VPNLAN1 net to VPNWAN1 address. So far so good.

          Then I created the second client which connected to a different server also on Port 443 UDP -> VPN server B, because I wanted a different LAN net (VPNLAN2) to connect to a different VPN server.

          So I had ovpnc2 assigned to a interface (VPNWAN2) and checked the connection on the dashboard, everything looked fine, then I checked the connection on my VPN provider side, everything looked fine, I had 2 connections to two different servers.
          Then I created a gateway for it (VPNWAN2_GW) with dynamic address and a NAT rule for my VPNLAN2 net to VPNWAN2 address -> AND THIS MESSED UP EVERYTHING:

          You have to add that if you connect to my VPN provider through UDP 443 you get a dynamic IP from the server in the 10.4.0.0/16 range.

          So VPNWAN1_GW had a gateway 10.4.A.B and
          VPNWAN2_GE had a gateway 10.4.C.D

          It seems that pfsense cannot handle that at all. As I said the OpenVPN logs looked good but the system logs created the errors mentioned in my first post constantly, the routing went crazy, everytime you updated the dashboard the gateways were blinking (one time down, then up, then down, then up) and in the firewall logs you could see that the pfsense was trying to route VPNLAN1 through my normal LAN/WAN which was blocked then because it found no working gateway anymore.

          So what did I do?
          I deleted the VPNWAN2, VPNWAN2_GW and the NAT rule for VPNLAN2 to VPNWAN2 to "calm down" pfsense in the first step.

          Then I checked at my provider that I can also connect through UDP port 80 to it.

          So I changed the setup of the second client to UDP port 80, checked the connections on the dashboard and at my provider, looked good, but that was also ok with the other solution.
          The difference when connection through UDP 80 is that you get a dynamic address for the gateway in the 10.6.0.0/16 range.

          So I reassigned ovpnc2 to a interface VPNWAN2, created a new VPNWAN2_GW and a new NAT rule.

          After restarting the client VPNWAN1_GW had still 10.4.A.B and
          VPNWAN2_GW then got an IP 10.6.C.D

          And then everything works like a charm, no SSL errors, no "routing gone wild" and everything works perfectly.

          But from my point of view this should also work like I wanted it in my first setup because the GW addresses received by the servers are different, it should be no problem to use UDP port 443 to two different servers, so maybe the devs should have a look a that!!!

          For me it's ok working that way for the moment.

          1 Reply Last reply Reply Quote 0
          • dotdashD
            dotdash
            last edited by

            If your interfaces were getting a /16 subnet mask, then both your gateways were reachable by both interfaces. That is not a bug.
            You might be able to hack around it with a secondary FIB, but I haven't played with that feature yet, and I don't think it's in the GUI yet.

            EDIT: Just booted up my 2.2 test kit, and it looks like the kernel isn't compiled for multiple fibs. So nevermind on that part.

            1 Reply Last reply Reply Quote 0
            • S
              sense678
              last edited by

              What do I get wrong here? The question is not if they are able to be reached by each other, the question is why do they do that?
              For what do I assign gateways then if they do what they want?

              Cheers…

              1 Reply Last reply Reply Quote 0
              • dotdashD
                dotdash
                last edited by

                If both gateways are reachable by both interfaces the routing is going to go crazy. You are not supposed to do that. I would point to your vpn provider as doing it wrong by giving you a /16 mask. In FreeBSD, you can work around it by starting the second vpn instance pointed to a secondary routing table, but this feature is not currently offered in pfSense. The way you are doing it seems perfectly fine also, as you are getting non-conflicting /16's.
                My point is that unless you are using advanced equipment with multiple routing tables (virtual routers, or whatever they call them), your configuration is not valid. I would not call this a bug, but the lack of an advanced feature. It should be doable in pfSense 2.2x, but probably requires a good deal of work, which is why I would expect it has not been added.

                1 Reply Last reply Reply Quote 0
                • C
                  cmb
                  last edited by

                  You mentioned adding gateways, was that via some means other than assigning the OpenVPN interfaces and letting their dynamic gateways be automatically created? Gateways should never be added for OpenVPN, the dynamic gateways that get added suffice.

                  1 Reply Last reply Reply Quote 0
                  • S
                    sense678
                    last edited by

                    @dotdash : ok, thanks for your comments, I am running it now that way (with the other UDP port getting a 10.6.x.y) gateway and it seems it's working without any flaws. It can only be a /16 mask from the provider because the problems occured with GW1 having 10.4.A.B and GW2 having 10.4.C.D so it has to be a /16 mask otherwise with a /24 mask there wouldn't have been these conflicts.

                    @cmb : No I didn't add the gateways for the VPN clients on my own, but I somehow "renamed" the automatic ones (through adding a gateway based on the automatic created gateway (+ button on the gateway), entered an alternative monitor IP and shortened the name, then saving) -> they then had a shorter name and the "automatic" ones disappeared automatically.  But I don't see a mistake in doing it like this because I did that from the beginning on and it's working without any flaws. I did the copying workaround because you can't rename the automatic created ones.

                    So I removed the BUG from headline, thanks!

                    1 Reply Last reply Reply Quote 0
                    • First post
                      Last post