Looking for ideas on troubleshooting an OpenVPN file transfer speed problem.

BFost

I would appreciate any ideas you have for my troubleshooting of an OpenVPN transfer speed issue.

I use a Netgate SG-5100 running pfSense Plus 23.05
The OpenVPN server is on a Netgate SG-5100 firewall in Florida. The ISP is Frontier business fiber 1000mbps up and down.
VPN users are in Florida and Illinois, only 5 total, pretty light usage. They typically get a file transfer speed of 64 to 120 mbps. This is satisfactory to me, but we have one user in Illinois on Xfinity who regularly struggles with an abysmal 8 mbps. They are on a 500 mbps down / 10 mbps up home connection. I would be happy if all the users were getting what I’m getting which is 120 mbps, Florida Spectrum Cable internet.

Running ping and tracert to a local IP address while on the tunnel, the Illinois user is getting 60-70ms latency which seems totally fine to me. In Florida I’m getting 30ms latency.

Could this be some kind of Xfinity ISP throttling?

Would I benefit from different OpenVPN settings?

Detailed Info about my setup
OpenVPN Protocol UDP4
Port 1194
Mode : Remote Access SSL/TLS + User Auth
Data Ciphers : AES-128-GCM, AES-128-CBC
Digest : SHA256
D-H Params : 2048 bits
CPU Type Intel(R) Atom(TM) CPU C3558 @ 2.20GHz
4 CPUs
AES-NI CPU Crypto: Yes (active)
IPsec-MB Crypto: Yes (inactive)
QAT Crypto: Yes (inactive)

My reading of the CPU usage from "Diagnostics / System Activity" is that while transferring files one core is about 80% usage.

Here is the configuration file from an OpenVPN client :
dev tun
persist-tun
persist-key
data-ciphers AES-128-GCM:AES-128-CBC
data-ciphers-fallback AES-128-CBC
auth SHA256
tls-client
client
resolv-retry infinite
remote “my public static IPv4 address” 1194 udp4
nobind
verify-x509-name "MyVPNname" name
auth-user-pass
pkcs12 NetGate-UDP4-1194-myuser.p12
tls-auth NetGate-UDP4-1194-myuser-tls.key 1
remote-cert-tls server
explicit-exit-notify

Thank you!

planedrop

Well I think the first question here is what are you using as the file transfer protocol? OpenVPN of course just establishes the connection, it's not handling the actual transfer of the file, that's another protocol like SMB or NFS.

Some protocols are VERY (like SMB) latency dependent so that might be what you are running into.

I'd also maybe consider trying a WireGuard VPN setup to see if that behaves better, it's a little more management overhead but generally performs better and is more reliable than OpenVPN, it's also got less internal overhead and can help with latency a bit.

michmoor

@planedrop spot on.
Could be whatever protocol being used or an MTU issue.

BFost

@planedrop Thanks for your reply. This is SMB, moving files from a Synology SMB network share to a Windows VPN client computer using Windows File Explorer.

michmoor

@BFost SMB is notoriously slow on VPN due to the inefficiencies in the protocol and latency involved over a WAN.
If possible i would leverage OneDrive or Gdrive which get around using smb otherwise you just have to deal with the poor performance.

Look over this video
https://www.youtube.com/watch?v=LnDRZbTQv9I

planedrop

@BFost Yeah as @michmoor mentioned SMB is not good when it comes to high latency connections. Even with only roughly 30ms ping over my WireGuard tunnel I can still see some bad hiccups due to latency, it's just not a very efficient protocol for latent connections.

Something like a public cloud service would work better, or if you really want to self host there are some other options that would be faster. IIRC NFS performs better with high latency connections, but ideally something like S3 would be better, or just NextCloud.

BFost

@michmoor Great video, Lawrence Systems is awesome. We turned on FTP services on this Synology file server to do an FTP test over the VPN. Speeds were actually the same or worse than SMB, so something else is going on here. I'm going to set up a Wireguard test and try that, and start looking into a cloud hosted solution. Thanks for the info.

michmoor

@BFost Ehhhh could be a different problem tho. Switching to FTP now we concern ourselves with TCP. If TCP throughput is low then it could be due to send or receiver buffers, windowing and latency. So its not a true apples to apples test.
What is the RTT between the sites?
Are you seeing any packet loss while on VPN? Extended ping tests could show this. Try pinging at different sizes with the DF bit set to figure out the MTU.

We can switch to a different VPN protocol and test so the results there could eliminate a theory.

johnpoz

@BFost said in Looking for ideas on troubleshooting an OpenVPN file transfer speed problem.:

is getting 60-70ms latency which seems totally fine to me

You understand with that latency, your 8mbps is right in the ball part for a window size of 64k.. So you really need to look what is going on.

$math.jpg$

I take it they are downloading, and not uploading - because upload they have a max of 10 per their isp anyway..

Are they on wifi.. We have lots of users report bad vpn performance - they were just on a shit wifi connection. If they plugged in a wire, no issue with their performance.