v21.02 Broke all ExpressVPN Gateways
-
@bod That's how ExpressVPN specified it. This has all been working fine for years for me, and now, after upgrading to v2.5 and without changing any other settings, it's completely broken. The gateway shows down, with 100% packet loss. Something internally isn't getting data through the tunnel.
I even upgraded to the v2.5.1-RC, to see if that contained more fixes similar to the patch mentioned above that might restore operation, but it doesn't. I'm no OpenVPN guru, so I'm hoping someone who is might have an idea. Thanks!
-
@guido666 said in v21.02 Broke all ExpressVPN Gateways:
@bod That's how ExpressVPN specified it. This has all been working fine for years for me, and now, after upgrading to v2.5 and without changing any other settings, it's completely broken. The gateway shows down, with 100% packet loss. Something internally isn't getting data through the tunnel.
I even upgraded to the v2.5.1-RC, to see if that contained more fixes similar to the patch mentioned above that might restore operation, but it doesn't. I'm no OpenVPN guru, so I'm hoping someone who is might have an idea. Thanks!
https://openvpn.net/community-downloads/
IMPORTANT NOTICES
BF-CBC CIPHER IS NO LONGER THE DEFAULT
Cipher handling for the data channel cipher has been significantly changed between OpenVPN 2.3/2.4 and v2.5, most notably there are no “default cipher BF-CBC” anymore because it is no longer considered a reasonable default. BF-CBC is still available, but it needs to be explicitly configured now.
For connections between OpenVPN 2.4 and v2.5 clients and servers, both ends will be able to negotiate a better cipher than BF-CBC. By default they will select one of the AES-GCM ciphers, but this can be influenced using the –data-ciphers setting.
Connections between OpenVPN 2.3 and v2.5 that have no –cipher setting in the config (= defaulting to BF-CBC and not being negotiation-capable) must be updated. Unless BF-CBC is included in –data-ciphers or there is a “–cipher BF-CBC” in the OpenVPN 2.5 config, a v2.5 client or server will refuse to talk to a v2.3 server or client, because it has no common data channel cipher and negotiating a cipher is not possible. Generally, we recommend upgrading such setups to OpenVPN 2.4 or v2.5. If upgrading is not possible we recommend adding data-ciphers AES-256-GCM:AES-128-GCM:AES-128-CBC (for v2.5+) or cipher AES-128-CBC (v2.4.x and older) to the configuration of all clients and servers.
If you really need to use an unsupported OpenVPN 2.3 (or even older) release and need to stay on BF-CBC (not recommended), the OpenVPN 2.5 based client will need a config file change to re-enable BF-CBC. But be warned that BF-CBC and other related weak ciphers will be removed in coming OpenVPN major releases.
For full details see the”Data channel cipher negotiation” section on the man page.
CONNECTIVITY TO SOME VPN SERVICE PROVIDER MAY BREAK
Connecting with an OpenVPN 2.5 client to at least one commercial VPN service that
implemented their own cipher negotiation method that always reports back that it is using BF-CBC to the client is broken in v2.5. This has always caused warning about mismatch ciphers. We have been in contact with some service providers and they are looking into it. This is not something the OpenVPN community can fix. If your commercial VPN does not work with a v2.5 client, complain to the VPN service provider. -
@bcruze You can tell from something in my logs that this is the problem? Can you help me understand what's going on?
-
@guido666 said in v21.02 Broke all ExpressVPN Gateways:
@bcruze You can tell from something in my logs that this is the problem? Can you help me understand what's going on?
i was having the SAME exact issue with 2 other providers. i wrote them both and told them my scenario, i had just updated to openvpn 2.5 (Pfsense 21.02)and now the tunnels do not work. i also asked them if any of their servers had open vpn2.5. one gave me 5 servers... they all immediately connected and worked even after restarting the tunnel or rebooting the Pfense router.
less that 24 hours after that email this was posted on their Blog:We have upgraded all our OpenVPN servers to OpenVPN 2.5
4 February 2021. ALL of my original servers worked after that!the other provider i canceled the rest of my plan as they never replied back
i encourage you to write your provider and see if any of their servers are running the latest openvpn
-
@thaddeusf I think I may have come across something. I reinstalled back to pfSense 2.4.5-P1, and restored my config from 2.5.0, because I had made some other changes I was hoping to try to keep. OpenVPN didn't work. I started looking at logs, and noticed some "cipher" errors.
Long story short: I think that during the upgrade 2.4.5-P1 -> 2.5.0, pfSense changed the cipher settings for all the VPN connections!
From the ExpressVPN guide (google it... these forums won't let me post a link):
- Encryption Algorithm: In the text editor you opened earlier, look for the word “cipher.” Select the algorithm shown after “cipher” in the dropdown menu. For example: AES-256-CBC.
- Enable NCP: Uncheck this box.
- NCP Algorithms: Leave blank.
Looking at my post-2.5.0 upgrade config XML, it contains these for all my OpenVPN client entries...
...
<data_ciphers_fallback>AES-256-CBC</data_ciphers_fallback>
...
<data_ciphers>AES-128-GCM,AES-256-CBC</data_ciphers>
<ncp_enable>enabled</ncp_enable>
...So NCP is enabled now? Why? My config backups pre-2.5.0 are missing the <crypto>AES-256-CBC</crypto> line altogether. Looking at my old config backups, the v19.1 backup (17Oct2020) has the <crypto>AES-256-CBC line, but the v21.4 backup (3Mar2021) does not.
When I open the entries in pfSense's however, it's incorrectly showing as AES-128-CBC in the box, and yes, NCP is enabled.
If I manually set the Encyption Algorithm back to AES-256-CBC, and uncheck Enable Negotiable Cryptographic Parameters again, then it connects again. I'm able to pass traffic, it seems.
Strangely, even though traffic seems to be flowing through the VPN gateway (e.g. "What's my IP?" shows the VPN external IP), the Status > Gateways still shows "Offline - 100% loss".
-
I just tried re-upgrading to 2.5.0, using the fixed VPN crypto settings, and it again changes the crypto settings to something else (that would not work). After fixing them, the VPN tunnel connects again, but I still can't pass traffic. Working on that now.
Also, it keeps disabling the hardware crypto offloading, too.
-
@guido666 said in v21.02 Broke all ExpressVPN Gateways:
I just tried re-upgrading to 2.5.0, using the fixed VPN crypto settings, and it again changes the crypto settings to something else (that would not work). After fixing them, the VPN tunnel connects again, but I still can't pass traffic. Working on that now.
Also, it keeps disabling the hardware crypto offloading, too.
i have started to have some weird things going on.
i have been manually stopping tunnels, making a change or two and clicking start and its doing the no data thing again.but try this... when do data is flowing. login to Pfense click status > then filter reload > reload filter. after 15 seconds data starts flowing for me!
-
i have started to have some weird things going on. i can't decided to stick with GCM or Cha1305 ultimately
i have been manually stopping tunnels, making a change or two (saving)and clicking start and its doing the no data thing again.
but try this... when no data is flowing. login to Pfense click status > then filter reload > reload filter. after 15 seconds data starts flowing for me!i will also note under system > misc > skip rules when gateway down IS checked
-
I posted a small "Expr*ssVPN adventure here :
Has anyone found solution for ExpressVPN? -
Yep I read that, great work!
I no longer can test this myself as I no longer have the subscription to azirevpn where I was having the same issue as this entire thread, was just curious if it helped anyone