pfsense 2.5.2: ExpressVPN connection working by gateway has 100% packet loss

amdreallyfast

Status:

Status -> OpenVPN shows that the connection is up.
Status -> System Logs -> OpenVPN shows no errors.
Status -> Gateways shows the ExpressVPN gateway (automatically created as part of following ExpressVPN's setup instructions) having 100% packet loss.
Note: The EXPRESSVPNSANFRANCISCO_VPNV4 gateway was automatically created and I can't edit it (that I know of).
Also Note: Blocked off my public IP address.

Status -> System Logs -> Gateway shows no errors, but it does show many "alarm latency" messages that look like this:

send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr 68.100.112.1 bind_addr 68.100.115.149 identifier "WAN_DHCP "
send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr fe80::36f8:e7ff:fe58:e419%re1 bind_addr fe80::3af7:cdff:fec0:2bdd%re1 identifier "WAN_DHCP6 "
send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr 10.169.1.161 bind_addr 10.169.1.162 identifier "EXPRESSVPNSANFRANCISCO_VPNV4 "

Note: The IP address shown in the screenshot (10.169.1.161) is not my IP address; it's not even in my subnet. Don't know where that's coming from.

Things I've tried/looked at:

Following the ExpressVPN directions for three different server configurations:
b. Los Angeles
c. Washington DC
c. San Francisco (latest tried, and the one that's currently configured)
Changing the compression and encryption algorithms of OpenVPN. When I do, the status of the VPN goes down, and when I change the settings back to what they were in the instructions, the VPN status goes back in the green. This tells me that my OpenVPN configuration is correct (I think) and that my problem is elsewhere (I just don't know where).
Forum topic: expressvpn-interface-is-up-but-gateway-is-down
Solution: "In the section where you configure the EXPRESSVPN interface, DO NOT set the IPv4 Configuration as DHCP, set it as NONE."
That option is not available on my ExpressVPN interface. See screenshot.

Note: The post was from 2017, and the pfsense GUI may have changed since then. The next thing I tried explicitly called out not setting "IPv4/IPv6 Configuration", so this may be an outdated solution.
Forum topic: has-anyone-found-solution-for-expressvpn
I gathered that the community concluded that there was a tiny setting somewhere that wasn't configured quite right, and so no single answer was settled on. A very thorough reply by Gertjan shows his entire setup and has a decent walkthrough, but somehow he was able to create the ExpressVPN interface and have the gateway immediately online. Mine's not doing that, so this post didn't help.
Forum topic: v2-5-broke-expressvpn-interface-to-gateway-monitoring
This is the exact problem that I'm having, but discussion looks incomplete and didn't arrive at a conclusion.
Forum topic: express-vpn-received-control-message-auth_failed
This doesn't seem to be my problem, but it had some thorough posts with settings, so I thought that I could gather something from it. Didn't find anything new.
Forum topic: openvpn-connectivity-to-expressvpn-in-v-2-5-2
This is exactly my problem and is post from this year, but the OP never replied that their problem was solved, and the final post doesn't mention a final configuration. The last configuration in the post shows 4 custom options, but I don't know if those are the only 4 custom options or just 4 that they added. The post does mention the need for OpenVPN 2.5.2 if you're running pfsense 2.5.2. I checked the version ("openvpn --version") and found that I was OpenVPN 2.5.2, so that's not a problem either.

Here is my configuration:

System -> Cert. Manager -> CAs -> Added according to ExpressVPN's instructions.
System -> Cert. Manager -> Certificates -> Added according to ExpressVPN's instructions.
VPN -> OpenVPN -> Clients -> Added according to ExpressVPN's instructions.
1. General Information
  1. Disabled - <unchecked>
  2. Server mode - "Peer to Peer ( SSL/TLS )"
  3. Protocol - "UDP on IPv4 only"
  4. Device mode tun - "Layer 3 Tunnel Mode'
  5. Interface - "WAN"
  6. Local port - <blank>
  7. Server host or address - "usa-sanfrancisco-ca-version-2.expressnetw.com" (copy-pasted from ExpressVPN's San Fransisco configuration file)
  8. Server port - "1195" (copy-pasted from ExpressVPN's San Fransisco configuration file)
  9. Proxy host or address - <blank>
  10. Proxy port - <blank>
  11. Proxy Authentication - "none"
  12. Description - "ExpressVPN San Francisco"
2. User Authentication Settings
  1. Username - <copy-pasted from ExpressVPN>
  2. Password - <copy-pasted from ExpressVPN>
  3. Authentication Retry - <unchecked>
3. Cryptographic Settings
  1. TLS Configuration - <checked>
  2. TLS Key - <(copy-pasted from ExpressVPN's San Fransisco configuration file)
  3. TLS Key Usage Mode - "TLS Authentication"
  4. TLS keydir direction - "Use default direction"
  5. Peer Certificate Authority - "ExpressVPN"
  6. Peer Certificate Revocation list - "No Certificate Revocation Lists defined. One may be created here: System > Cert. Manager > Certificate Revocation"
  7. Client Certificate - "ExpressVPN Cert (In Use)" (the one I created earlier)
  8. Data Encryption Negotiation - <checked>
  9. Data Encryption Algorithms
    1. AES-256-GCM (default)
    2. AES-128-GCM (default)
    3. CHACHA20-POLY1305 (default)
    4. AES-256-CBC (added according to ExpressVPN's instructions)
  10. Fallback Data Encryption Algorithm - "AES-256-CBC (256 bit key, 128 bit block)"
  11. Auth digest algorithm - "SHA512 (512-bit)"
  12. Hardware Crypto - "No Hardware Crypto Acceleration"
4. Tunnel Settings
  1. IPv4 Tunnel Network - <blank>
  2. IPv6 Tunnel Network - <blank>
  3. IPv4 Remote network(s) - <blank>
  4. IPv6 Remote network(s) - <blank>
  5. Limit outgoing bandwidth - <blank>
  6. Allow Compression - "Decompress incoming, do not compress outgoing (Asymmetric)" (required in order to select compression level)
  7. Compression - "Adaptive LZO Compression [Legacy style, comp-lzo adaptive]" (required by ExpressVPN instructions)
  8. Topology - <unchecked>
  9. Type-of-Service - <checked>
  10. Don't pull routes - <unchecked>
  11. Don't add/remove routes - <unchecked>
  12. Pull DNS - <unchecked>
5. Ping settings
  1. Inactive - "0"
  2. Ping method - "keepalive -- Use keepalive helper to define ping configuration"
  3. Interval - "10"
  4. Timeout - "60"
6. Advanced Configuration
  1. Custom options - "fast-io;persist-key;persist-tun;remote-random;pull;comp-lzo;tls-client;verify-x509-name Server name-prefix;remote-cert-tls server;key-direction 1;route-method exe;route-delay 2;tun-mtu 1500;fragment 1300;mssfix 1450;verb 3;sndbuf 524288;rcvbuf 524288" (copy-pasted from ExpressVPN's instructions)
  2. UDP Fast I/O - "Use fast I/O operations with UDP writes to tun/tap. Experimental."
  3. Exit Notify - "Retry 1x"
  4. Send/Receive Buffer - "512 KiB"
  5. Gateway creation - "IPv4 only"
  6. Verbosity level - 10
Interfaces -> ExpressVPNSanFrancisco (ovpnc1)
1. General Configuration
  1. Enable - <checked>
  2. Description - "ExpressVPNSanFrancisco"
  3. IPv4/IPv6 Configuration - "This interface type does not support manual address configuration on this page."
  4. MTU - <blank>
  5. MSS - <blank>
2. Reserved Networks
  1. Block private networks and loopback addresses - <unchecked>
  2. Block bogon networks - <unchecked>
Firewall -> Aliases -> Added
1. Properties
  1. Name - "LocalSubnets" (based on a tutorial)
  2. Description - "Home network"
  3. Type - "Network(s)"
2. Network(s)
  1. Network or FQDN - 10.62.33.0/24 (private IP that I made up)
Firewall -> NAT -> Outbound
1. Set according to ExpressVPN instructions
Firewall -> Rules -> LAN
1. Set according to ExpressVPN instructions

From reading the forums, it looks like ExpressVPN is working with pfsense 2.5.2 for at least some of you, so I know it's possible. Any ideas what I missed/messed up? Need more info?

lovan6

@amdreallyfast

The instruction you gathered was from Pfsense 2.4.5 Expressvpn.

Expressvpn does not have any specific instructions for 2.5.2.

I still used Pfsense 2.4.5 P1 and never bother to upgrade. Why fix it if ain't broke. It's just me.

I also have a 100 percent packet loss on my Gateway on Pfsense 2.4.5 P1 but never bother about it since I have a stable and fast connection.

I am 8500 miles away connected to Washington DC server. a latency of 200 ms but never get bother by latency since I am not a gamer and just use Epxressvpn for the purpose of geolocation and online buying.

If you ask Expressvpn support for help, It is no use since they don't know what they are talking about.

They talked like they are reading a script and worse use Google your concerns.

I can not blame Pfsense. if Expressvpn does not listen to their customers.

Gertjan

@lovan6 said in pfsense 2.5.2: ExpressVPN connection working by gateway has 100% packet loss:

The instruction you gathered was from Pfsense 2.4.5 Expressvpn.
Expressvpn does not have any specific instructions for 2.5.2.

Very true.
The instructions were made when OpenVPN 2.4.x was used : on the pfSense side, and probably Expr*ssVPN also.
But time passes, and pages like this are most often not maintained, or they don't include de xxxx exceptions possible, like other versions used.

I've posted somewhere in the past a setup ( for me ) using Expr*ss OpenVPN - today, using pfSense (it used OpenVPN 2.5.2 - see below).

It works : I can ping.

@lovan6 said in pfsense 2.5.2: ExpressVPN connection working by gateway has 100% packet loss:

I also have a 100 percent packet loss on my Gateway on Pfsense 2.4.5 P1

Ping something that actually replies to ping ^^

The interface doesn't need any rules, it is an outgoing interface, like WAN ("fewer rules is better").

@lovan6 said in pfsense 2.5.2: ExpressVPN connection working by gateway has 100% packet loss:

If you ask Expressvpn support for help, It is no use since they don't know what they are talking about.

They do, I guess (never contacted them).
They really try to make your live - and, I admit, their lives, easier by saying : use OUR app.
The app is Windows, Mac, and a boat load of Linux based 'bianaries'.
They will communicate an opvn client connection file if you insists on doing it the manual way, but, as usual, then you're on your own. "Experts don't need advise or howtos".

@amdreallyfast said in pfsense 2.5.2: ExpressVPN connection working by gateway has 100% packet loss:

Forum topic: openvpn-connectivity-to-expressvpn-in-v-2-5-2
This is exactly my problem and is post from this year, but the OP never replied that their problem was solved, and the final post doesn't mention a final configuration. The last configuration in the post shows 4 custom options, but I don't know if those are the only 4 custom options or just 4 that they added. The post does mention the need for OpenVPN 2.5.2 if you're running pfsense 2.5.2. I checked the version ("openvpn --version") and found that I was OpenVPN 2.5.2, so that's not a problem either.

This topic : https://forum.netgate.com/topic/167357/openvpn-connectivity-to-expressvpn-in-v-2-5-2/7?_=1640953374177

Start with an empty 'custom options' block.
The connection won't (probably) work and or the openvpn client log file will be full with warnings and errors.

First : get the 'opvn' file from Expr*ssVPN. Use this file to fill in the GUI settings. Save.
Now, direction 'console' or even better : SFTP and look at the generated opvn file, it's here :
/var/etc/openvpn/clientx (x is probably '1').
You'll find the pfSense OpenVPN client 'config.opvn' there.

Compare the original 'Expr*ssVPN' file you've downloaded with this 'pfSense' OpenVPN client config.opvn' file.

I had to add these missing :

verify-x509-name Server name-prefix;
remote-cert-tls server;
route-delay 2;
tun-mtu 1500;
fragment 1300;
mssfix 1450;
auth-nocache;

Now my config.opvn file looks like this :

dev ovpnc2
verb 3
dev-type tun
dev-node /dev/tun2
writepid /var/run/openvpn_client2.pid
#user nobody
#group nobody
script-security 3
daemon
keepalive 10 60
ping-timer-rem
persist-tun
persist-key
proto udp4
auth SHA512
up /usr/local/sbin/ovpn-linkup
down /usr/local/sbin/ovpn-linkdown
local 192.168.10.3
tls-client
client
lport 0
management /var/etc/openvpn/client2/sock unix
remote 45.91.22.2 1195 udp4
auth-user-pass /var/etc/openvpn/client2/up
capath /var/etc/openvpn/client2/ca
cert /var/etc/openvpn/client2/cert 
key /var/etc/openvpn/client2/key 
tls-auth /var/etc/openvpn/client2/tls-auth 1
data-ciphers AES-256-CBC:AES-256-CBC
data-ciphers-fallback AES-256-CBC
allow-compression asym
comp-lzo yes
resolv-retry infinite
fast-io
sndbuf 524288
rcvbuf 524288
verify-x509-name Server name-prefix

remote-cert-tls server

route-delay 2

tun-mtu 1500

fragment 1300

mssfix 1450

auth-nocache

"45.91.22.2" is the IP of the Expr*sVPN server - in Paris, I think.
"192.138.10.3" is my current WAN IP.

All this rocket science works because there is something that tells me all ik ok : the OpenVPN client itself.
Here it is : see the log example in the thread mentioned above.

@amdreallyfast said in pfsense 2.5.2: ExpressVPN connection working by gateway has 100% packet loss:

The post does mention the need for OpenVPN 2.5.2 if you're running pfsense 2.5.2.

As said :

With pfSense 2.5.2 you use OpenVPN 2.5.x :

[2.5.2-RELEASE][admin@pfsense.mypfsense.net]/root: openvpn --version
OpenVPN 2.5.2 amd64-portbld-freebsd12.2 [SSL (OpenSSL)] [LZO] [LZ4] [MH/RECVDA] [AEAD] built on Jun 24 2021
library versions: OpenSSL 1.1.1k-freebsd 25 Mar 2021, LZO 2.10
Originally developed by James Yonan
Copyright (C) 2002-2018 OpenVPN Inc sales@openvpn.net
....

as pfSense also uses (= comes with) "OpenSSL 1.1.1k" etc
Nothing has to be installed by the pfSense admin.

@amdreallyfast said in pfsense 2.5.2: ExpressVPN connection working by gateway has 100% packet loss:

but the OP never replied that their problem was solved

That's 'normal'.
When it works, the person asking the initial question doesn't reply pack with "Ok".
Only "No Ok" will produce feedback.

Btw : I'm using pfSense 2.5.2.
I'm using the same Expr*sVPN service.
Only the settings differ (user name, password).

The settings I mentioned work for me.
I used the doc from here https://openvpn.net/community-downloads/ - the OpenVPN 2.5.2 The "21 april 2021" release notes.

@amdreallyfast said in pfsense 2.5.2: ExpressVPN connection working by gateway has 100% packet loss:

Note: The IP address shown in the screenshot (10.169.1.161) is not my IP address;

It's the OpenVPN client gateway IP (not the final VPN WAN IP), handled by the Expr*ssVPN server. It's just a tunnel IP.

Also : I'm connected, but I'm not (policy) routing right now.
With the pfSense ping test I selected the EXP*SSVPN interface, and fired of some pings, that worked.
Actually using the connection needs some more work.
Two choices :
Declare the EXPESSVPN interface as the default outgoing IPv4 "WAN" interface.
Or do policy routing, see for example : How To Setup pfsense OpenVPN Policy Routing With Kill Switch Using A Privacy VPN

amdreallyfast

@gertjan

I've posted somewhere in the past a setup ( for me ) using Expr*ss OpenVPN - today, using pfSense (it used OpenVPN 2.5.2 - see below).

It works : I can ping.

I could ping too, and the gateway remained at 100% packet loss. I tried three servers (LA, San Francisco, DC), and all three VPN connections were up, all three could successfully ping google.com (and a couple others that I tried for good measure) through the Diagnostics -> Ping tool, and all three gateways also showed 100% packet loss. I don't know how that is possible. How is it possible to have 100% packet loss to a gateway but still being to ping an outside source through that connection?

Now, direction 'console' or even better : SFTP and look at the generated opvn file

I don't understand this sentence. Are you asking me to ssh into the router and run sftp to...do something? Please explain.

I've never ssh'd into this router before and it looks like it needs special setup. I tried ssh admin@192.168.1.1 (still using defaults except password until I can get this working), but the connection timed out, so ssh isn't an option until I get help fixing that.

Is it possible to look at my router's opvn file through the GUI? I haven't figured out how to do that yet either.

All this rocket science works because there is something that tells me all ik ok : the OpenVPN client itself.

Here it is : see the log example in the thread mentioned above.

Which log example? There are two in that thread.

amdreallyfast

@lovan6

I still used Pfsense 2.4.5 P1 and never bother to upgrade. Why fix it if ain't broke. It's just me.

I tried reinstalling from scratch, this time using 2.4.5, followed the configuration exactly, and had the exact same problem. I did it three times with servers in LA, San Francisco, and DC. All three of them show "up" in Status -> OpenVPN -> Clients, all of them could ping google.com through the Diagnostics -> Ping tool, and yet all of them also had 100% packet loss and none of their connections would work. I checked the system logs and all their connections were timing out and retrying every few minutes, always with the same result.

If 2.4.5 works for you but not for me, then I did something wrong and it's probably really small; I just don't know what I did wrong and need something to compare against. Do you know if it's possible for you to export your entire configuration in a single file and share it ((sans sensitive information like private keys and passwords) so that I can compare against my setup and see what I did wrong?

lovan6

@amdreallyfast

Send me a PM and I will send you my ExpressVpn config.

There are also several configurations for DNS resolver, Aliases, Firewall rules, Nat including ExpressVPN media streamer DNS.

Do a clean slate. There are several configurations not offered on ExpressVPN instructions. Particularly user destined for VPN only. By default ExpressVPN instructions, users are all connected to VPN.

As I said setup Pfsense 2.4.5 P1 first. Let me know how it works if you plan to upgrade to 2.5.2. Since I don't have the time if anything goes wrong with 2.5.2. Wife will start complaining if her VPN does not work.

FYI I used a different Pfsense setup. It is an Intel Kabylake processor and an Intel i350 T4 V2 server NIC. M.2 sata formatted to ZFS. Fiber optic connection, modem in bridge mode.

Gertjan

@amdreallyfast said in pfsense 2.5.2: ExpressVPN connection working by gateway has 100% packet loss:

I don't understand this sentence. Are you asking me to ssh into the router and run sftp to...do something? Please explain.

Use SSH if your not close to pfSense. Or use the console access.
IMHO : it's a shot in the dark to make a VPN connection to a server work just using the GUI.
I need to see what happens.
Comparing the file as you can download from Expr*ssVPN and the one actually created by pfSense is an important step. So, yeah, the good old command line access comes in handy here.

SFTP is FTP over SSH. Using SFTP permits you to browse the files on pfSense like you can browser files on your W10 using Explorer.

@amdreallyfast said in pfsense 2.5.2: ExpressVPN connection working by gateway has 100% packet loss:

I've never ssh'd into this router before and it looks like it needs special setup. I tried ssh admin@192.168.1.1 (still using defaults except password until I can get this working), but the connection timed out, so ssh isn't an option until I get help fixing that.

Check "Enable Secure shell" and Save.
No special other setup is needed.

If you use Windows OS, install world's most famous SSH application : Putty.

@amdreallyfast said in pfsense 2.5.2: ExpressVPN connection working by gateway has 100% packet loss:

Which log example?

The one that starts with

The connection logs (in reverse order) :

2021-10-22 12:39:45.217054+02:00 openvpn 7206 Initialization Sequence Completed

amdreallyfast

@gertjan
Working on it. After following your directions in the prior post (and after learning about sftp), I was able to find the config files, but there is no .ovpn file. There's are several other files:

I pieced together the files into something resembling the ExpressVPN .ovpn file that I got from their website, and then I used Notepad++'s Compare plugin to look at the differences. The following differences mostly come from the client1.conf file, who's contents are pasted at the top of the text on the left side of the compare:

There are a lot more items in the .conf file than at the top of the ExpressVPN-provided .ovpn file, and for the ones that are similar (near the bottom of the screenshot), there are a few that are different. It looks like the custom options specified by ExpressVPN's PfSense setup instructions are not identical to the options specified in the individual .ovpn files. Lovely.

On a hunch, I tried taking all those options from .ovpn file, splicing them together with semicolons, and then copy-pasted that chunk into the text box in VPN -> OpenVPN -> Clients -> <SF client> -> Advanced Configuration -> Custom options. And the VPN wouldn't even connect. That means that the custom options that ExpressVPN specifies in their .ovpn files are not correct. Dang it.

So which configuration is correct? Are either correct?

Then I started altering items one by one to slowly try to match the custom options specified in the ExpressVPN PfSense setup instructions, and eventually I got Status -> OpenVPN to show that it was online again, but Status -> Gateways still shows the San Francisco VPN gateway offline with 100% packet loss.

Curiously, the VPN connection succeeded even though I didn't have the exact same custom options as the ExpressVPN PfSense setup instructions. I was aiming for that, but it connected prior to reaching there. The following custom options on the right (minus the struck-through ones) seemed to work. Notice that dev tun, nobind, and auth-user-pass would all cause Status -> OpenVPN to show a failed connection.

It looks like the other differences in the screenshot don't matter for the VPN connection itself. Do you have any ideas why?

The gateway is still offline.

Would you send me your config file so that I can compare?

Gertjan

@amdreallyfast said in pfsense 2.5.2: ExpressVPN connection working by gateway has 100% packet loss:

but there is no .ovpn file.

Strange.

Another view : from WINSCP :

The is a folder in /var/etc/openvpn called client1
so the path is /var/etc/openvpn/client1/ here are the files for the first openvpn client - there must be a .opvn file.

@amdreallyfast said in pfsense 2.5.2: ExpressVPN connection working by gateway has 100% packet loss:

Would you send me your config file so that I can compare?

First thing tomorrow morning. Within 6 hours ;)

Gertjan

could you send me your config file so that I can compare?

Done.

See 2 forum messages back, it's al ready there.

edit !
Wait !

Check my config from above :
It says :

comp-lzo yes

as per instructions from Expr*ssVPN.

I reloaded the config, mine, for Paris, and yours, for San Franciso.
Now it says :

comp-lzo no

This is a very recent config change on their side.
I changed the option in the pfSense GUI for :

This way, my config shows :

comp-lzo no

And take note : this is 'bad' as the "comp-lzo" command is now deprecated (in openvpn 2.5.2).
It's probably better to select :

which introduced a new :

allow-compression asym

in the local config file.
This proves that Expr**ssvpn is still based on 2.4.x style of configuration.

To be clear : with

comp-lzo no

I could ping google.com just fine (it connected).

amdreallyfast

@gertjan

The lack of a client1 folder and the lack of an .ovpn file may be due to me using 2.4.5. Maybe. I'm intentionally using the older version because ExpressVPN's instructions are explicitly stated to be for 2.4.5, and I'm trying to avoid rocking the boat too much from the original instructions (at least until I get this working). And I say "maybe" because ExpressVPN was apparently able to create .ovpn files with 2.4.5. I just don't know how. Did you have to install a specific package in order to export the config file?

amdreallyfast

@gertjan

See 2 forum messages back, it's already there.

Are you sure? Two messages (of yours) back I see this post, and I'm not seeing a config file. Would you take a screenshot of the message (and link it if it's another forum post)?

amdreallyfast

@gertjan

I could ping google.com just fine (it connected).

Same. The gateway is offline with 100% packet loss, and yet the Diagnostics -> Ping tool could use that VPN client to ping. This tells me that the gateway being offline with 100% packet loss is not an issue that can be checked with the Ping tool. But how can I diagnose it then?

Gertjan

@amdreallyfast said in pfsense 2.5.2: ExpressVPN connection working by gateway has 100% packet loss:

And I say "maybe" because ExpressVPN was apparently able to create .ovpn files

Expr*ssVP always exported 'ovpn' files.
These are just 'plain text' files, filled with process command line options that the process openvpn understands.
pfSense 2.5.2 uses the same format and extension 'internally'.
That is because pfSense is just an OS (sort of), with a nice GUI, and for the rest it uses the same processes (binary programs) as all the other MAC based / Window based / FreeBSD based / Lines based systems.

Dono what pfSense 2.4.5 did, that's to far in the past for me.

@amdreallyfast said in pfsense 2.5.2: ExpressVPN connection working by gateway has 100% packet loss:

Did you have to install a specific package in order to export the config file?

Exporting what to what ?
To export a 'setup' (config opvn file + binaries) or just the opvn file from an OpenVPN Access server (server !!) then you need this

package.
See the pfSense manual or one of a thousands of OpenVPN video's, book, whatever.

@amdreallyfast said in pfsense 2.5.2: ExpressVPN connection working by gateway has 100% packet loss:

Are you sure? Two messages (of yours) back I see this post, and I'm not seeing a config file. Would you take a screenshot of the message (and link it if it's another forum post)?

Here :

That an image from this thread.
You can't see it ?

@amdreallyfast said in pfsense 2.5.2: ExpressVPN connection working by gateway has 100% packet loss:

This tells me that the gateway being offline with 100% packet loss is not an issue that can be checked with the Ping tool. But how can I diagnose it then?

Because some one - in your case 10.173.0.137 - doesn't answer to 'ping' this doesn't mean the gateway is offline.
What this implies is :
The path to host 10.173.0.137 doesn't exist
or
It doesn't reply to ping.
or ....

When I empty this field :

the pfSense montoring system will auto choose an IP, like your "10.173.0.137".

And I see the same thing : no ping replies. So it says 'offline'. Or, like you, I can ping 'google' just fine.
That why I fill in a IP that I "have /own / use" and that is close to the Exprss VPN exit WAN IP : I have a dedicated server near by. Or you could use what everybody does : use 8.8.8.8 as a monitoring IP.
And your ExpressVPN is now (nope, this should be : "looks like") on-line.

amdreallyfast

@gertjan

Expr*ssVP always exported 'ovpn' files.

...Then why can't I? This is weird. I previously mentioned that I've tried SF, LA, and DC, so I tried turning on all their VPN clients and interfaces, wondering if that might cause it to create a client1 folder to separate the three clients. Nope. My ... folder now looks like this. I don't know why OpenVPN is not creating .ovpn files. Any ideas?

That an image from this thread.
You can't see it ?

Oh. That's your file? I was looking for a file attachment (as opposed to copy-pasted values), but now I have questions about this (see next question).

I had to add these missing :
<several lines of custom options>
Now my config.opvn file looks like this :
<more lines of custom options>

Which one is your file? You said that you had to add some values, and then you say, "my config.ovpn file looks like this" and you show a list of custom options that does not include the ones that you just said you had to add. Now I'm confused about what your file looks like. Are you sure that's the entirety of the .ovpn file and that the file does not include the items that you had just added?

When I empty this field :
<screenshot of Monitor IP>
the pfSense montoring system will auto choose an IP, like your "10.173.0.137".
And I see the same thing : no ping replies. So it says 'offline'.

I don't think that we're seeing the same thing. The Monitor IP field is blank for all three of my VPN client configurations (SF, LA, DC), and all three of them do show ping replies while the gateway is offline. Please explain. Are you saying that I should leave Monitor IP blank or that I should not leave it blank?

Gertjan

@amdreallyfast said in pfsense 2.5.2: ExpressVPN connection working by gateway has 100% packet loss:

Then why can't I?

Because you strugle with the wrong questions ;)

There is no such thing as exporting a "OpenVPN client" config file.

A OpenVPN server installation you have set up, benefits from the pfSense "openvpn-client-export" package. The file you exports comes from the OpenVPN (server parameters) and are used by the future OpenVPN clients, so they can access the (your) OpenVPN server for remote access.

Your subject is all about the OpenVPN client set up.
The OpenVPN server - the provider, in this case - is ExpressVPN.
So it's Express VPN that shares a 'config file' with you, what I call the opvn config file. You downloaded it.
This file has to be used as a guide line to fill in the pfSense GUI client page.
Currently, pfSense does not offer a no way to 'import' the file you got from ExpessVPN.

Now you understand (bitter) why I want you to look at the openvpn client config file that pfSense GUI has created with it's GUI.
This file has to resemble as close as possible as the file you obtained from ExpressVPN.

The
Server
Client specific overrides
Wizard
Client Export
Shared key Export

are related to the OpenVPN server.

The OpenVPN client has only one page : the one that's underlined in my image.

An important aspect is also : the GUI can only handle a subset of the possible options or settings. A GUI is useful for enforcing 'illegal' option combinations.

Got it ?! ;)

Side note : it would be nice to have the possibility to import a server side generated file, so you can set up the client easely.
But there is a thing : settings are version specific. They change all the time.
"openvpn" as a program has close to one thousand settings (options and possible combination of options combined - it's close to impossible to write a GUI to handle them all.

Big (server type) apps, like apache2, nginx, postfix and bind (named) can not be set up using a GUI.
You have to fall back to the classic 'confg files' as the shear number of combinations of settings is just daunting.

Btw : the last part is totally IMHO.

amdreallyfast

@gertjan

Because you strugle with the wrong questions ;)

The curse of learning. Thanks for bearing with me through this. Books and tutorials can't correct me when I understand a term badly, but you can. Thanks :).

Now you understand (bitter) why I want you to look at the openvpn client config file that pfSense GUI has created with it's GUI.
This file has to resemble as close as possible as the file you obtained from ExpressVPN.

Alright, so how do I get "the openvpn client config file that pfSense GUI has created with it's GUI"? You walked me through sftp, but that showed that I didn't have a .ovpn file at all. Is there a way to get this .ovpn file through the GUI?

It sounds like you're looking for the Client Specific Overrides. Are you sure? Mine are blank. I have done anything here.

Gertjan

@amdreallyfast said in pfsense 2.5.2: ExpressVPN connection working by gateway has 100% packet loss:

It sounds like you're looking for the Client Specific Overrides

A point for you !

I meant :

All the other menu options are 'server' related.

edit :
I restarted reading you first post (see above).
I decided to enter exactly what you've mentioned.
One exception : I removed my password.

https://www.test-domaine.fr/VPN-Client.png

Compare this huge image with your settings.
Double check for differences.
( edit : I found a difference : my verbosity (log) level is 3 - level 10 is far to detailed, create hundreds of entries a second. Leaving it 10 or 3 has no other side effect )

With these settings :

I monitored 8.8.8.8 to check the VPN-Client connection - RTT is a bit high as I'm based in France, St Jose in the US is not really close.

All I had to to to make the connection works, was switching from :

"automatic mode" to "hybrid mode" and save.
I'm now connected the entire company over Expr*ssVPN ...... I wasn't expecting this, as I wanted to focus on making the connection work first, and then go for the outbound natting.

I didn't even add a 'policy' firewall rule on my LAN interface to make the connection work.
It's not activated :

So, first things first : Make your connection work.

Btw : this is the file the OpenVPN client is using right now :

/var/etc/openvpn/client2/config.ovpn

dev ovpnc2
verb 3
dev-type tun
dev-node /dev/tun2
writepid /var/run/openvpn_client2.pid
#user nobody
#group nobody
script-security 3
daemon
keepalive 10 60
ping-timer-rem
persist-tun
persist-key
proto udp4
auth SHA512
up /usr/local/sbin/ovpn-linkup
down /usr/local/sbin/ovpn-linkdown
local 192.168.10.3
tls-client
client
lport 0
management /var/etc/openvpn/client2/sock unix
remote usa-sanfrancisco-ca-version-2.expressnetw.com 1195 udp4
auth-user-pass /var/etc/openvpn/client2/up
auth-retry nointeract
capath /var/etc/openvpn/client2/ca
cert /var/etc/openvpn/client2/cert 
key /var/etc/openvpn/client2/key 
tls-auth /var/etc/openvpn/client2/tls-auth 1
data-ciphers AES-256-GCM:AES-128-GCM:CHACHA20-POLY1305:AES-256-CBC
data-ciphers-fallback AES-256-CBC
allow-compression asym
comp-lzo adaptive
resolv-retry infinite
fast-io
sndbuf 524288
rcvbuf 524288
fast-io

persist-key

persist-tun

remote-random

pull

comp-lzo

tls-client

verify-x509-name Server name-prefix

remote-cert-tls server

key-direction 1

route-method exe

route-delay 2

tun-mtu 1500

fragment 1300

mssfix 1450

verb 3

sndbuf 524288

rcvbuf 524288

You can clearly see de double spaces lines at the bottom, some of them are present twice in the config file.
This is an issue for later, because, with these settings, it works.

I'm using :

amdreallyfast

@gertjan

That's a real big screenshot. Kudos on spending the time to piece that monstrosity together.

Noticed some differences:

The ExpressVPN setup instructions for 2.4.5 say to leave NCP (Negotiable Cryptographic Parameters) blank, but in your 2.5.2 configuration you have it checked (now called "Data Encryption Negotiation").
The ExpressVPN setup instructions for 2.4.5 say to check "Don't pull routes", but you haven't.

But I followed along anyway and replicated your settings. And the gateways are still offline. I haven't adjusted the firewall rules yet. I'm still trying to get these gateways online.

Note: Prior to making these screenshots, I shortened the names of the VPN clients and their interfaces, so they'll appear a little different than in prior screenshots, but they're still the same ones.

VPN clients online
Gateways offline
Using ExpressVPN's IP address checker:
Interface settings
Gateway settings
Client settings

amdreallyfast

@gertjan

At this point I'm thinking that the only way forward is for a network expert who's familiar with PfSense and OpenVPN to sit down at my computer and poke around until they can figure out what's going on. Whatever my problem is, it's not normal. I'm not a network expert, and I don't know how to diagnose this. I've replicate other peoples' working settings, and it isn't working on my end. Here's some more diagnostic info; does it give you any ideas about where to start diagnosing next?

I can tell from a basic Wireshark scan of my network traffic that TCP failure and retransmission is not uncommon, and PfSense's Status -> System Logs -> Gateways shows me that communication with the gateways is frequently timing out. All gateways are timing out, including the WAN, which is what I'm currently using to get online and talk with you.

These are all from this morning:
It's rather frequent that I try to load a web page, wait several seconds, the page fails to load and the browser says that it can't find the page, and then it automatically retries and succeeds (sometimes needs to retry twice). I don't know what is causing this frequent failure, or if there's anything I can do about it. I've got a 100Mbps internet connection over cable that is usually stable and reliable once connection is actually established (ex: a long download or a big YouTube video doesn't fail once it's started). It shouldn't take 5-10sec to load a google search. The failures (when they happen) only occur when initially loading a page. If it were a random hardware failure (circuitry going bad, loose cable, etc.), I'd expect random disconnects during downloads and streaming large video, but I'm not seeing that.

I've mentioned before that when I use the ExpressVPN desktop app to make a VPN connection, all those connection issues for my PC disappear. Web pages load quickly 100% of the time, no timeouts, no lost connections. That makes me wonder if there's some sort of packet monitoring and sniffing going on after the traffic leaves my network that either slows the transmission enough to time out or stops it entirely. Why would making an encrypted tunnel to ExpressVPN's servers magically make the disconnects go away?
My connection is this: Cable -> modem -> router (mini PC running PfSense 2.4.5) -> TP-Link Archer A7 (wifi router/switch configured as access point only) -> PC.

Here's the TP-Link Archer A7 set in AP mode
The TP-Link Archer A7's DHCP Server is set to Auto, but the DHCP client list is empty, so I'm guessing that it was smart enough to not try to be a DHCP server when in AP mode. Should I explicitly turn that off?

Does this info give you any ideas of where I can start diagnosing next?

Any ideas of network diagnostic tools that can help me track down where the traffic is disappearing and why?