Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    v21.02 Broke all ExpressVPN Gateways

    Scheduled Pinned Locked Moved OpenVPN
    44 Posts 8 Posters 8.6k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • T
      thaddeusf
      last edited by

      I have done another completely clean install and this time it is the 2.5 pfsense community edition on a ProtectLI.

      The bottom line is that I can create an ExpressVPN interface but the Gateway to the interface if marked 'Offline, Packetloss: 100%"

      A monitor IP is listed and is the same as the Gateway IP.

      If I turn Gateway monitoring off (system/routing/gateways) in the configuration of the ExpressVPN interface, the entire firewall stops passing traffic.

      This is not precisely the same behavior I saw on the SG-3100, but it's close. The 3100 would start passing traffic with gateway monitoring off, but the performance would tank.

      This is pretty obviously a bug. Is there additional information (logs, etc) I can provide?

      B 1 Reply Last reply Reply Quote 0
      • B
        bcruze @thaddeusf
        last edited by

        https://openvpn.net/community-downloads/

        Under important notices states it pretty clear

        The service providers are at fault not Netgate

        T 1 Reply Last reply Reply Quote 1
        • T
          thaddeusf @bcruze
          last edited by

          @bcruze
          The only thing I am complaining about is that Netgate is not making 2.4.5 available for the SG-3100 when I specifically requested it.

          It was my own damn fault for rashly upgrading it.

          What could possibly be a valid reason not making it available?

          1 Reply Last reply Reply Quote 0
          • B
            bod
            last edited by

            Well looks like I found the solution with regards to Private Internet Access. A user has found that using the config below will work. I have even turned gateway monitoring on and multiple service restarts and it works everytime.

            https://www.privateinternetaccess.com/forum/discussion/38389/aes-256-gcm-with-asus-wrt-merlin-386-1-final-on-pia#latest

            Force AES-256-GCM
            Put in "NCP-disable" in Custom Options.

            Will let you know tomorrow if it's all still going well.

            B T 2 Replies Last reply Reply Quote 0
            • B
              bcruze @bod
              last edited by

              @bod said in v21.02 Broke all ExpressVPN Gateways:

              Well looks like I found the solution with regards to Private Internet Access. A user has found that using the config below will work. I have even turned gateway monitoring on and multiple service restarts and it works everytime.

              https://www.privateinternetaccess.com/forum/discussion/38389/aes-256-gcm-with-asus-wrt-merlin-386-1-final-on-pia#latest

              Force AES-256-GCM
              Put in "NCP-disable" in Custom Options.

              Will let you know tomorrow if it's all still going well.

              its been posted a few times it will work. then speeds will go to a grinding halt. my previous provider would also work for a while. but if I restarted the tunnel. or rebooted the router it stopped working again. please let us know either way

              1 Reply Last reply Reply Quote 0
              • T
                thaddeusf @bod
                last edited by

                @bod

                With my configuration, I see:

                • GCM only does not successfully connect.
                • If I include 'ncp-disable' in the custom, it seems to stop OpenVPN at load (which is confusing, but I've not had time to research)

                Really interested to hear your results and compare.

                I'm going to try to switch to L2TP/IPsec until this can be resolved.

                ALSO, Netgate finally sent me the link to 2.4.5 for the SG-3100. While continuing research on the ProtectLI, I'm going to repair the ARM appliance.

                1 Reply Last reply Reply Quote 0
                • T
                  thaddeusf
                  last edited by thaddeusf

                  I've made some progress today.

                  • I've been able to configure an L2TP/IPSec interface (with gateways) to the ExpressVPN server. It work as without any problem, but the performance is significantly slower not to mention with compromised security.

                  • I've been able to disable Gateway Monitoring on the OpenVPN to ExpressVPN interface. Apparently my previous attempts which proved unstable were caused by the interfaces being load balanced.

                  I've opened a ticket with ExpressVPN as well. Hopefully we'll make some progress.

                  1 Reply Last reply Reply Quote 0
                  • B
                    bod
                    last edited by

                    Spoke too soon. The Watchdog service (Action) broke it.

                    No issues with throughput on AES-256-GCM. Still getting sustained 40mbps.

                    1 Reply Last reply Reply Quote 0
                    • T
                      thaddeusf
                      last edited by

                      This post is deleted!
                      1 Reply Last reply Reply Quote 0
                      • G
                        guido666
                        last edited by guido666

                        I'm having a similar/related issue. After upgrading to 2.5, my traffic does not get routed out the VPN gateway. It hits the proper firewall rule, and gets passed, but it just doesn't go out the VPN gateway and instead goes out the WAN gateway.

                        The VPN gateway (in Status) shows as "Offline, Packetloss: 100%".

                        If I disable Gateway monitoring (System > Routing > Gateways) for the VPN gateway, the traffic instantly starts going through the VPN gateway correctly again (matching the same firewall rule).

                        If related, I also get the 3 errors mentioned previously.
                        Mar 11 01:03:28 openvpn 49069 Options error: option 'route' cannot be used in this context ([PUSH-OPTIONS])
                        Mar 11 01:03:28 openvpn 49069 Options error: option 'dhcp-option' cannot be used in this context ([PUSH-OPTIONS])
                        Mar 11 01:03:28 openvpn 49069 Options error: option 'redirect-gateway' cannot be used in this context ([PUSH-OPTIONS])

                        GertjanG 1 Reply Last reply Reply Quote 0
                        • GertjanG
                          Gertjan @guido666
                          last edited by

                          @guido666

                          Looks like you confirmed - see above - https://redmine.pfsense.org/issues/11448

                          There is a patch already.
                          Tried (try) it ?!!

                          No "help me" PM's please. Use the forum, the community will thank you.
                          Edit : and where are the logs ??

                          G 1 Reply Last reply Reply Quote 1
                          • G
                            guido666 @Gertjan
                            last edited by

                            @gertjan

                            Thanks! Patch applied, and traffic seems to be routing through the VPN correctly, even with Disable Gateway Monitoring unchecked.

                            I've set the patch to auto-load. Once 2.5.1 comes out, and presumably includes the fix, will I need to disable it, and remove the patch at that point? I've never used this patch system before.

                            GertjanG T 2 Replies Last reply Reply Quote 0
                            • GertjanG
                              Gertjan @guido666
                              last edited by Gertjan

                              @guido666 said in v21.02 Broke all ExpressVPN Gateways:

                              and presumably includes the fix, will I need to disable it, and remove the patch at that point? I've never used this patch system before.

                              The patcher checks the files it needs to patch, and if it can find specific lines it will change these for new lines, the patch.

                              When you upgrade, and nothing was changed in the file(s) you patched, the patch still auto applies.
                              If the patch is now included in the update, it can't find the same specific lines any more, as the source has changed.
                              The patcher will try - and fail, which is ok, as the patch is needed any more / already there.

                              It's a close "to set it and forget it system".

                              No "help me" PM's please. Use the forum, the community will thank you.
                              Edit : and where are the logs ??

                              G 1 Reply Last reply Reply Quote 1
                              • G
                                guido666 @Gertjan
                                last edited by

                                @gertjan Thanks for your help.

                                1 Reply Last reply Reply Quote 0
                                • T
                                  thaddeusf @guido666
                                  last edited by

                                  @guido666

                                  This patch did not fix the problem for me.

                                  Will you send your custom settings?

                                  Thanks.

                                  G 1 Reply Last reply Reply Quote 0
                                  • G
                                    guido666 @thaddeusf
                                    last edited by

                                    @thaddeusf

                                    Which settings are you looking for?

                                    T 1 Reply Last reply Reply Quote 0
                                    • T
                                      thaddeusf @guido666
                                      last edited by

                                      @guido666
                                      The original custom options from ExpressVPN are:

                                      fast-io;persist-key;persist-tun;remote-random;pull;comp-lzo;tls-client;verify-x509-name Server name-prefix;remote-cert-tls server;key-direction 1;route-method exe;route-delay 2;tun-mtu 1500;fragment 1300;mssfix 1450;verb 3;sndbuf 524288;rcvbuf 524288

                                      Have you changed these?

                                      Also, have you changed it from AES-256-CBC?

                                      Thanks

                                      G 2 Replies Last reply Reply Quote 0
                                      • G
                                        guido666 @thaddeusf
                                        last edited by

                                        @thaddeusf No, I did not change them from default.

                                        Here's what I have...

                                        fast-io;persist-key;persist-tun;remote-random;pull;comp-lzo;tls-client;verify-x509-name Server name-prefix;ns-cert-type server;key-direction 1;route-method exe;route-delay 2;tun-mtu 1500;fragment 1300;mssfix 1450;verb 3;sndbuf 524288;rcvbuf 524288

                                        @gertjan The patch was working for me, however, today I had to reboot the router and now can not get traffic through again. The patch was set to auto-run at start up, and I also tried reverting and applying it again, to no avail.

                                        I'm getting these errors in the OpenVPN logs again...

                                        Mar 18 15:09:03 openvpn 38456 Options error: option 'route' cannot be used in this context ([PUSH-OPTIONS])
                                        Mar 18 15:09:03 openvpn 38456 Options error: option 'dhcp-option' cannot be used in this context ([PUSH-OPTIONS])
                                        Mar 18 15:09:03 openvpn 38456 Options error: option 'redirect-gateway' cannot be used in this context ([PUSH-OPTIONS])

                                        Full log of one connection attempt...

                                        Mar 18 15:10:10 openvpn 38456 MANAGEMENT: Client disconnected
                                        Mar 18 15:10:10 openvpn 38456 MANAGEMENT: CMD 'status 2'
                                        Mar 18 15:10:10 openvpn 38456 MANAGEMENT: CMD 'state 1'
                                        Mar 18 15:10:10 openvpn 38456 MANAGEMENT: Client connected from /var/etc/openvpn/client21/sock
                                        Mar 18 15:10:07 openvpn 38456 MANAGEMENT: Client disconnected
                                        Mar 18 15:10:07 openvpn 38456 MANAGEMENT: CMD 'status 2'
                                        Mar 18 15:10:07 openvpn 38456 MANAGEMENT: CMD 'state 1'
                                        Mar 18 15:10:07 openvpn 38456 MANAGEMENT: Client connected from /var/etc/openvpn/client21/sock
                                        Mar 18 15:10:07 openvpn 38456 MANAGEMENT: Client disconnected
                                        Mar 18 15:10:07 openvpn 38456 MANAGEMENT: CMD 'status 2'
                                        Mar 18 15:10:07 openvpn 38456 MANAGEMENT: CMD 'state 1'
                                        Mar 18 15:10:07 openvpn 38456 MANAGEMENT: Client connected from /var/etc/openvpn/client21/sock
                                        Mar 18 15:09:05 openvpn 38456 Initialization Sequence Completed
                                        Mar 18 15:09:03 openvpn 38456 /usr/local/sbin/ovpn-linkup ovpnc21 1500 1629 <REDACTED> <REDACTED> init
                                        Mar 18 15:09:03 openvpn 38456 /sbin/ifconfig ovpnc21 <REDACTED> <REDACTED> mtu 1500 netmask 255.255.255.255 up
                                        Mar 18 15:09:03 openvpn 38456 TUN/TAP device /dev/tun21 opened
                                        Mar 18 15:09:03 openvpn 38456 TUN/TAP device ovpnc21 exists previously, keep at program end
                                        Mar 18 15:09:03 openvpn 38456 Incoming Data Channel: Using 512 bit message hash 'SHA512' for HMAC authentication
                                        Mar 18 15:09:03 openvpn 38456 Incoming Data Channel: Cipher 'AES-256-CBC' initialized with 256 bit key
                                        Mar 18 15:09:03 openvpn 38456 Outgoing Data Channel: Using 512 bit message hash 'SHA512' for HMAC authentication
                                        Mar 18 15:09:03 openvpn 38456 Outgoing Data Channel: Cipher 'AES-256-CBC' initialized with 256 bit key
                                        Mar 18 15:09:03 openvpn 38456 Using peer cipher 'AES-256-CBC'
                                        Mar 18 15:09:03 openvpn 38456 OPTIONS IMPORT: adjusting link_mtu to 1629
                                        Mar 18 15:09:03 openvpn 38456 OPTIONS IMPORT: peer-id set
                                        Mar 18 15:09:03 openvpn 38456 OPTIONS IMPORT: --ifconfig/up options modified
                                        Mar 18 15:09:03 openvpn 38456 OPTIONS IMPORT: compression parms modified
                                        Mar 18 15:09:03 openvpn 38456 OPTIONS IMPORT: timers and/or timeouts modified
                                        Mar 18 15:09:03 openvpn 38456 Options error: option 'route' cannot be used in this context ([PUSH-OPTIONS])
                                        Mar 18 15:09:03 openvpn 38456 Options error: option 'dhcp-option' cannot be used in this context ([PUSH-OPTIONS])
                                        Mar 18 15:09:03 openvpn 38456 Options error: option 'redirect-gateway' cannot be used in this context ([PUSH-OPTIONS])
                                        Mar 18 15:09:03 openvpn 38456 PUSH: Received control message: 'PUSH_REPLY,redirect-gateway def1,dhcp-option DNS <REDACTED>,comp-lzo no,route <REDACTED>,topology net30,ping 10,ping-restart 60,ifconfig 10.145.0.82 10.145.0.81,peer-id 19'
                                        Mar 18 15:09:03 openvpn 38456 SENT CONTROL [Server-4042-1a]: 'PUSH_REQUEST' (status=1)
                                        Mar 18 15:09:02 openvpn 38456 [Server-4042-1a] Peer Connection Initiated with [AF_INET]<REDACTED>:1195
                                        Mar 18 15:09:02 openvpn 38456 Control Channel: TLSv1.3, cipher TLSv1.3 TLS_AES_256_GCM_SHA384, 2048 bit RSA
                                        Mar 18 15:09:02 openvpn 38456 VERIFY OK: depth=0, C=VG, ST=BVI, O=MyVPN, OU=MyVPN, CN=Server-4042-1a, emailAddress=support@<REDACTED>.com
                                        Mar 18 15:09:02 openvpn 38456 VERIFY X509NAME OK: C=VG, ST=BVI, O=MyVPN, OU=MyVPN, CN=Server-4042-1a, emailAddress=support@<REDACTED>.com
                                        Mar 18 15:09:02 openvpn 38456 VERIFY OK: nsCertType=SERVER
                                        Mar 18 15:09:02 openvpn 38456 VERIFY OK: depth=1, C=VG, ST=BVI, O=MyVPN, OU=MyVPN, CN=MyVPN CA, emailAddress=support@<REDACTED>.com
                                        Mar 18 15:09:02 openvpn 38456 VERIFY WARNING: depth=1, unable to get certificate CRL: C=VG, ST=BVI, O=MyVPN, OU=MyVPN, CN=MyVPN CA, emailAddress=support@<REDACTED>.com
                                        Mar 18 15:09:02 openvpn 38456 VERIFY WARNING: depth=0, unable to get certificate CRL: C=VG, ST=BVI, O=MyVPN, OU=MyVPN, CN=Server-4042-1a, emailAddress=support@<REDACTED>.com
                                        Mar 18 15:09:02 openvpn 38456 WARNING: this configuration may cache passwords in memory -- use the auth-nocache option to prevent this
                                        Mar 18 15:09:02 openvpn 38456 TLS: Initial packet from [AF_INET]<REDACTED>:1195, sid=66de6eb3 9b1e2179
                                        Mar 18 15:09:02 openvpn 38456 UDPv4 link remote: [AF_INET]<REDACTED>:1195
                                        Mar 18 15:09:02 openvpn 38456 UDPv4 link local (bound): [AF_INET]<REDACTED>:0
                                        Mar 18 15:09:02 openvpn 38456 Socket Buffers: R=[42080->524288] S=[57344->524288]
                                        Mar 18 15:09:02 openvpn 38456 TCP/UDP: Preserving recently used remote address: [AF_INET]<REDACTED>:1195
                                        Mar 18 15:09:02 openvpn 38456 Incoming Control Channel Authentication: Using 512 bit message hash 'SHA512' for HMAC authentication
                                        Mar 18 15:09:02 openvpn 38456 Outgoing Control Channel Authentication: Using 512 bit message hash 'SHA512' for HMAC authentication
                                        Mar 18 15:09:02 openvpn 38456 WARNING: experimental option --capath /var/etc/openvpn/client21/ca
                                        Mar 18 15:09:02 openvpn 38456 NOTE: the current --script-security setting may allow this configuration to call user-defined scripts
                                        Mar 18 15:09:02 openvpn 38456 WARNING: --ns-cert-type is DEPRECATED. Use --remote-cert-tls instead.
                                        Mar 18 15:09:02 openvpn 38456 MANAGEMENT: unix domain socket listening on /var/etc/openvpn/client21/sock
                                        Mar 18 15:09:02 openvpn 38326 library versions: OpenSSL 1.1.1i-freebsd 8 Dec 2020, LZO 2.10
                                        Mar 18 15:09:02 openvpn 38326 OpenVPN 2.5.0 amd64-portbld-freebsd12.2 [SSL (OpenSSL)] [LZO] [LZ4] [MH/RECVDA] [AEAD] built on Feb 5 2021
                                        Mar 18 15:09:02 openvpn 38326 WARNING: file '/var/etc/openvpn/client21/up' is group or others accessible
                                        Mar 18 15:09:02 openvpn 38326 WARNING: Compression for receiving enabled. Compression has been used in the past to break encryption. Sent packets are not compressed unless "allow-compression yes" is also set.
                                        Mar 18 15:09:02 openvpn 38326 WARNING: Compression for receiving enabled. Compression has been used in the past to break encryption. Sent packets are not compressed unless "allow-compression yes" is also set.

                                        1 Reply Last reply Reply Quote 0
                                        • B
                                          bod
                                          last edited by bod

                                          As I understand it... the patch is for the "No Pull" option only. OpenVPN>Tunnel Settings>Don't Pull Routes.

                                          If you only want one device to access the tunnel then apply patch and ensure the box is ticked. Personally, I would never want that option disabled. I would rather specify using firewall rules which device was routed to the VPN.

                                          I notice in both your configs you have PULL enabled?

                                          Sorry if I'm mis-understanding this - I'm no expert on PfSense.

                                          G 1 Reply Last reply Reply Quote 0
                                          • G
                                            guido666 @bod
                                            last edited by

                                            @bod That's how ExpressVPN specified it. This has all been working fine for years for me, and now, after upgrading to v2.5 and without changing any other settings, it's completely broken. The gateway shows down, with 100% packet loss. Something internally isn't getting data through the tunnel.

                                            I even upgraded to the v2.5.1-RC, to see if that contained more fixes similar to the patch mentioned above that might restore operation, but it doesn't. I'm no OpenVPN guru, so I'm hoping someone who is might have an idea. Thanks!

                                            B 1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.