pfSense and OpenVPN speeds

SteveITS

@Rico
This has IPSec numbers: https://www.netgate.com/products/appliances/

This has a chart at the bottom for TNSR but shows pfSense on a SG-5100: https://www.netgate.com/blog/choosing-the-right-netgate-appliance.html

Rico

Yes...but this thread is about OpenVPN @pfSense.
Impossible to relate anything for OpenVPN with IPsec numbers...

-Rico

stephenw10

I expect to see over 100Mbps on the 3100 if you are using a CESA supported cipher, which AES-CBC should be.

I would also expect to see over 125Mbps on the 1100 using AES-GCM.

There are many variables etc!

Steve

Rico

Well I don‘t care about 5-10Mbps VPN traffic more or less.
Only would see a problem if you say like in your testings the speed is double or 1/3 more. :-)

-Rico

sgw

@stephenw10 coming here because I look for a quick solution to free CPU on a 2100.

We seem to outmax the hardware by running ~20 ovpn clients (1 Gbit/s WAN) plus a wireguard site to site tunnel and pfblockerng ...

A stronger hardware is in preparation but I look for a config tweak to decrease the load.

I wanted to switch from AES-256-GCM to AES-128-GCM in the ovpn-server to decrease the overall load, but the numbers mentioned above seem to tell me that that won't do much.

I am not looking for more throughput, I need to decrease the CPU load. (I already disabled telegraf, that was quite heavy as well)

hints welcome

stephenw10

What's using the CPU currently? Check the output of top -HaSP at the command line.

There may not be much you can do though. With a 1G WAN it will be possible to hit the limits of the CPU. You could potentially set limiters for connecting clients to prevent that.

sgw

@stephenw10 currently very low load.

Right now it doesn't look overloaded at all.

Now and then there is sftp-traffic between the two sites which are connected by the wireguard site to site tunnel. This, in combination with the other services maxes out the hw.

It's a multi-layered issue: the delivering server vm is restricted etc etc

Maybe I could tune the wg-tunnel somehow.

Right now I can't see any limits (of the 2100) hit ... in the morning we hit the CPU-limits all the time.

thanks

stephenw10

So what issues do you see when this happens?

If it's maxed out by traffic over the Wireguard tunnel you might apply a limit to that.

sgw

@stephenw10 thanks for asking.

I am in the process of pinpointing an issue that might not even have to do with openvpn. I think multiple things overlay each other and give different results at different times:

basically I have 2 pfSense plus 24.11 boxes in 2 sites "office" and let's call it "data center" ;-)

They are crossconnected with a wireguard site2site vpn, only the LANs are routed over wireguard.

The data center pfSense is also a ovpn-gw for customers, they access VMs via VPN. The VMs run on a 3-server proxmox cluster in the data center.
All that works fine.

They run a linux VM there also that provides update-zips for the customers via sftp. If the customer accesses the related URI via the DNS-record pointing to the WAN-IP of the datacenter-pfSense the download speed is fine.

If my customer accesses the same URI (using the same DNS-name and in turn WAN-IP of datacenter) from behind the office pfSense it's way slower. a tenth or so.

That's the initial issue, and I am digging through everything ...

Yes, sftp isn't cool, I try to switch to scp.

We outruled wireguard-usage. We used the IP only.
I upgraded the VM in terms of software, and edited the vCPU to "host". I switched to a virtio-NIC. etc etc

I have to ask the coder there if his software (the one his customers upgrade by pulling stuff via sftp) maybe caches something and that leads to this difference.

In the process of debugging yesterday I had times when the datacenter-pfsense maxxed out its CPU (that's the 2100), so I tried to remove load there by disabling telegraf etc ... / in the afternoon the load was low and the sftp-transfer still wasn't higher. The vCPU in the VM also plays a role etc

I am quite sure that I have routing and NAT set up OK. The line there is 1 Gbit/s symmetric, that also shouldn't be the bottleneck.
Still scratching my head here ;-)

thanks for reading all this, ideas welcome.

stephenw10

Hmm, so you're sure that traffic is not going over the tunnel in either direction when clients access it from behind the 2100?

Can you test between those sites using something else just to check there's no throttling in the route?

sgw

@stephenw10 I might do tests from my laptop when I visit them some day.

I even tested against a fresh debian VM, that made no real difference.

mtr looked correctly, btw.

That's a tricky one ;-)

stephenw10

Hmm, well the first thing I would do is confirm you can actually pass traffic between those sites at a reasonable speed when not using a tunnel. Because I've seen countless customers who were diagnosing VPN issues for weeks when the actual issue was some bad router/link.

sgw

@stephenw10 I agree. I think I did iperf-tests some months ago that looked much better than the scp/sftp-stuff. Sure, it has to be faster, but it was way better.

I will repeat that asap.