22.05 and NordVPN tunneling
-
@stephenw10 Sorry, I guess I don't know what you mean by the load....other that 20-30% on each cpu.
Yes, I can get 800+ mb/s without the VPN connected.
Yes, it could be the far end, but I have tried different servers on NordVPN and I get very similar results.
-
I mean if I test an OpenVPN tunnel on a 4100 here and look at the activity I see:
last pid: 6977; load averages: 0.64, 0.30, 0.16 up 71+01:47:45 20:28:48 539 threads: 4 running, 522 sleeping, 13 waiting CPU 0: 36.6% user, 0.0% nice, 43.3% system, 0.0% interrupt, 20.1% idle CPU 1: 12.2% user, 0.0% nice, 30.3% system, 0.0% interrupt, 57.5% idle Mem: 33M Active, 261M Inact, 472M Wired, 3033M Free ARC: 237M Total, 92M MFU, 138M MRU, 692K Anon, 1027K Header, 5445K Other 133M Compressed, 249M Uncompressed, 1.87:1 Ratio Swap: 1024M Total, 1024M Free Message from syslogd@4100-2 at Oct 5 20:24:41 ...C TIME WCPU COMMAND 21459pm[80344]: /i 96 0 17M 7468K CPU0 0 46:25 93.43% /usr/local/sbin/openvpn --config /var/etc/openvpn/client1/config.ovpn 11 root 155 ki31 0B 32K RUN 1 1684.1 59.69% [idle{idle: cpu1}] 11 root 155 ki31 0B 32K RUN 0 1685.2 18.61% [idle{idle: cpu0}] 96097 root 26 0 15M 6668K select 0 0:02 12.32% iperf3 -c 10.2.4.1 -t 60 0 root -76 - 0B 544K - 1 15:31 7.75% [kernel{if_io_tqg_1}] 0 root -76 - 0B 544K - 0 4:25 1.24% [kernel{if_io_tqg_0}] 74666 root 52 0 133M 58M accept 0 0:17 0.38% php-fpm: pool nginx (php-fpm) 0 root -92 - 0B 544K - 0 242:49 0.32% [kernel{dummynet}] 0 root -76 - 0B 544K - 1 165:14 0.18% [kernel{if_config_tqg_0}] 20 root -16 - 0B 16K - 0 12:35 0.17% [rand_harvestq] 71350 root 20 0 15M 5528K CPU1 1 0:00 0.12% top -HaSPSo you can see OpenVPN using most of the CPU. iperf itself is using quite a bit there because I'm testing from the firewall directly.
Try testing a nordvpn client that isn't running on the firewall and see what sort of throughput you can get to the same server. It may be that's the best you can get and we are chasing the unobtainable.
Steve
-
@stephenw10 said in 22.05 and NordVPN tunneling:
Ok, here is what when speed testing with the vpn running on the 4100...top -HaSP
last pid: 21582; load averages: 0.27, 0.20, 0.13 up 35+04:02:56 14:10:41 603 threads: 4 running, 569 sleeping, 30 waiting CPU 0: 15.7% user, 0.0% nice, 34.1% system, 0.0% interrupt, 50.2% idle CPU 1: 23.1% user, 0.0% nice, 34.9% system, 0.0% interrupt, 42.0% idle Mem: 38M Active, 185M Inact, 473M Wired, 3117M Free ARC: 207M Total, 29M MFU, 170M MRU, 432K Anon, 918K Header, 6906K Other 84M Compressed, 212M Uncompressed, 2.53:1 Ratio PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND 13920 root 85 0 17M 7480K CPU0 0 124:02 85.90% /usr/local/sbin/openvpn --config /var/etc/openvpn/client6/config.ovpn 11 root 155 ki31 0B 32K RUN 0 823.4H 50.90% [idle{idle: cpu0}] 11 root 155 ki31 0B 32K RUN 1 829.0H 42.11% [idle{idle: cpu1}] 0 root -76 - 0B 528K - 0 102:18 16.04% [kernel{if_io_tqg_0}] 0 root -76 - 0B 528K - 1 114:40 2.98% [kernel{if_io_tqg_1}] 73000 root 20 0 12M 3480K bpf 1 5:38 0.85% /usr/local/sbin/filterlog -i pflog0 -p /var/run/filterlog.pid 25057 root 20 0 11M 2640K select 1 4:47 0.42% /usr/sbin/syslogd -s -c -c -l /var/dhcpd/var/run/log -P /var/run/syslog.pid -f /etc/s 12 root -60 - 0B 480K WAIT 1 155:42 0.33% [intr{swi4: clock (0)}] 20 root -16 - 0B 16K - 0 25:18 0.17% [rand_harvestq] 23700 root 20 0 14M 4856K CPU1 1 0:01 0.12% top -HaSP 0 root -76 - 0B 528K - 0 31:00 0.06% [kernel{if_config_tqg_0}] 19 root -16 - 0B 16K pftm 1 19:02 0.06% [pf purge] 21929 root 20 0 11M 2184K select 1 0:54 0.01% /usr/sbin/powerd -b hadp -a hadp -n hadp 5420 dhcpd 20 0 23M 12M select 1 0:42 0.01% /usr/local/sbin/dhcpd -user dhcpd -group _dhcp -chroot /var/dhcpd -cf /etc/dhcpd.conf -
Ok, well neither CPU core is anywhere near 100% there so it's not CPU limiting.
I would have to assume it's either limited at the server side or as a result of something in the route.
So check what throughput you can see using a NordVPN client running off the firewall.
What VPN client settings are you using in pfSense currently?
Steve
-
@stephenw10
So, if I use UDP, it cuts the speed way down. If I use Nord Lynx, it helps a LOT (testing with the same server). Is it possible to setup Nordlynx on pfsense? -
Nord Lynx is mostly Wireguard. So...maybe! I've never tried it and it may require some extra bits pfSense cannot currently do.
OpenVPN over TCP is almost always significantly slower than UDP. So if you're seeing the opposite that's suspicious. What settings have you tried? What were you using for the results you have stated here?
Steve
-
@stephenw10
I'm sorry I meant UDP vs Nord Lynx was slower vs faster. I can't get TCP to work, so no worry there. -
Ok, well it looks like Nord Lynx can be made to work in pfSense. You just need to install their client in something else and extract the keys first because for some reason they won't give them to you directly.
Or use one of the other VPN providers that do support Wireguard dircetly.
But you still haven't said what client settings you're using in OpenVPN so you might just have very slow encryption. Or no fastio. Or too smaller buffers.
Steve
-
@stephenw10
Sorry about not getting the client config to you...dev tun persist-tun persist-key ncp-ciphers AES-256-GCM:AES-128-GCM:AES-256-CBC cipher AES-256-CBC auth SHA256 tls-client client resolv-retry infinite remote xxx.xxx.xxx.xxx 1194 tcp-client nobind auth-user-pass remote-cert-tls server -
Ok, so you have AES-128-GCM available as a cipher via ncp which should be the fastest available. Can you see if Nord is negotiating that in the connection logs?
Since you're using UDP I would enable FastIO and increase the send/recv buffers to 512K. Both of those should give you an increase in throughput.
Steve
-
@stephenw10
The fast io option was already checked in the pfsense client GUI. The buffer was set to "default", whatever that is and I updated to 512k.On a separate note...regarding wireguard...I found another page to setup the wireguard because the reddit post ws limited....and it is overwhelming. I get the interface to show as "up", but wasn't routing traffic even though I set the pass option in the new interface and set a lan firewall entry to use that gateway.
-
Do you see recent handshakes in the Wireguard status?
The interface status can be misleading there.
Steve
-
@stephenw10
I had a spot where I put the private IP that the provider gave me in the wrong spot, corrected now. It appears to be connected (handshake shows green/recent), now I just have to figure out why I'm not routing...maybe a DNS issue, maybe routing through the firewall rules. Not sure yet...getting....so.....close.....thank...you.....
-
Probably something in the crypto-routing that is generated by the allowed subnets.
Also remember that Wireguard doesn't add any routing for you so you must add that manually if you need it. Though you're probably using policy routing here.
Steve
