pfSense and OpenVPN speeds

stephenw10

@Morpheus101 said in pfSense and OpenVPN speeds:

Is this overkill?

Yes.
It will pass 230Mbps OpenVPN though.

Steve

Rico

Some OpenVPN numbers I had in my testings.

Intel Xeon Gold 6136
AES-256-CBC 441,4 Mbps
AES-256-GCM 463,8 Mbps
AES-128-CBC 441,4 Mbps
AES-128-GCM 481,9 Mbps

Intel Xeon Silver 4215
AES-256-CBC 394,1 Mbps
AES-256-GCM 434,8 Mbps
AES-128-CBC 401,0 Mbps
AES-128-GCM 432,4 Mbps

Intel Core i7-7700
AES-256-CBC 477 Mbps
AES-256-GCM 519 Mbps
AES-128-CBC 479 Mbps
AES-128-GCM 522 Mbps

Netgate XG-7100
AES-256-CBC 266 Mbps
AES-256-GCM 288 Mbps
AES-128-CBC 278 Mbps
AES-128-GCM 290 Mbps

Netgate SG-5100
AES-256-CBC 276,3 Mbps
AES-256-GCM 290,9 Mbps
AES-128-CBC 280,5 Mbps
AES-128-GCM 292,8 Mbps

Netgate SG-3100
AES-256-CBC 91,4 Mbps
AES-256-GCM 83,1 Mbps
AES-128-CBC 98,5 Mbps
AES-128-GCM 89,2 Mbps

Netgate SG-1100
AES-256-CBC 118,5 Mbps
AES-256-GCM 119,6 Mbps
AES-128-CBC 118,5 Mbps
AES-128-GCM 120,7 Mbps

The problem with OpenVPN is it can only use one CPU core per instance/thread. So clock frequency is your friend with OpenVPN or switch to IPsec. ;-)

-Rico

@Rico said in pfSense and OpenVPN speeds:

Some OpenVPN numbers I had in my testings.

Intel Xeon Gold 6136
AES-256-CBC 441,4 Mbps
AES-256-GCM 463,8 Mbps
AES-128-CBC 441,4 Mbps
AES-128-GCM 481,9 Mbps

Intel Xeon Silver 4215
AES-256-CBC 394,1 Mbps
AES-256-GCM 434,8 Mbps
AES-128-CBC 401,0 Mbps
AES-128-GCM 432,4 Mbps

Intel Core i7-7700
AES-256-CBC 477 Mbps
AES-256-GCM 519 Mbps
AES-128-CBC 479 Mbps
AES-128-GCM 522 Mbps

Netgate XG-7100
AES-256-CBC 266 Mbps
AES-256-GCM 288 Mbps
AES-128-CBC 278 Mbps
AES-128-GCM 290 Mbps

Netgate SG-5100
AES-256-CBC 276,3 Mbps
AES-256-GCM 290,9 Mbps
AES-128-CBC 280,5 Mbps
AES-128-GCM 292,8 Mbps

Netgate SG-3100
AES-256-CBC 91,4 Mbps
AES-256-GCM 83,1 Mbps
AES-128-CBC 98,5 Mbps
AES-128-GCM 89,2 Mbps

Netgate SG-1100
AES-256-CBC 118,5 Mbps
AES-256-GCM 119,6 Mbps
AES-128-CBC 118,5 Mbps
AES-128-GCM 120,7 Mbps

The problem with OpenVPN is it can only use one CPU core per instance/thread. So clock frequency is your friend with OpenVPN or switch to IPsec. ;-)

-Rico

Can you explain (AES-128-GCM 292,8 Mbps) what the ,8 mean?

If I'm looking at your numbers it appears if I want any chance of archiving the maximum bps from my ISP that I will need at least the SG-51 or at least the equivalent hardware specs from another PC?

I am all new to this and finally got pfSense setup on a VM lab in order to get familure with the software.

You are talking about IPsec in place of pfSense?

I don't have the first clue about IPSec. I don't believe I can use IPSec on a private commercial VPN provider like TorGuard?

Thanks...

SteveITS

@Morpheus101 said in pfSense and OpenVPN speeds:

what the ,8 mean?

Many countries use a comma to indicate a decimal, whereas others use a period.

I'm curious why the SG-3100 tested slower than the SG-1100?

If you're connecting out to a service you are limited to what they allow. Most requests here are for connecting from somewhere to the office VPN.

@teamits said in pfSense and OpenVPN speeds:

@Morpheus101 said in pfSense and OpenVPN speeds:

what the ,8 mean?

Many countries use a comma to indicate a decimal, whereas others use a period.

I'm curious why the SG-3100 tested slower than the SG-1100?

If you're connecting out to a service you are limited to what they allow. Most requests here are for connecting from somewhere to the office VPN.

Thanks for the , explanation.

I'm connecting to a Torguard VPN fairly close to me. They told be the low BPS speed was due to the limitation of the router I was using. They didn't tell what I could expect if I moved up to more powerful hardware. I'm only using about 12% of my bandwidth as it is with heavy usage.

SteveITS

re: hardware, "it depends"...on CPU, hardware acceleration, encryption options used, etc. This may help somewhat: https://docs.netgate.com/pfsense/en/latest/book/hardware/hardware-sizing-guidance.html#vpn-all-types

inf3rno

Wouldn't it be possible to use WireGuard instead? https://www.freshports.org/net/wireguard/ Afaik. it is a lot faster than OpenVPN and it reached 1.0 half year ago, so it should be stable and secure enough.

@teamits I asked my VPN provider about speed with the i7 processor and they are telling me that I should have no problem hitting 100mps with that hardware. I also asked them if they are throttling, and they said no.

Am am hitting 230mps on a constant basis, outside the VPN. Still trying to understand why I'm unable to get at least 200mbs using what it appears to be adequate hardware. This is a commercial VPN provider, and if they are not throttling, then why?

@inf3rno Is WireGuard able to be installed on top of pfSense, like OpenVPN?

inf3rno

@Morpheus101 I have no idea. I know that it can be installed on FreeBSD, and pfSense is FreeBSD based. So maybe. I guess trying it does not hurt. VPN speed depends on the number of the users too. If you don't have many parallel connections and your hardware is capable, then something else causes the low speed.

stephenw10

@teamits said in pfSense and OpenVPN speeds:

I'm curious why the SG-3100 tested slower than the SG-1100?

It does look like an anomaly but it's almost certainly because OpenSSL compiled for aarch64 can take advantage of the additional instructions available there. Both those numbers seem low though.

Wireguard cannot, yet, easily be added to pfSense. I believe there is a thread detailing it here but it is all manual at this point. No gui config. Nothing backed up etc.

Steve

Rico

@stephenw10 said in pfSense and OpenVPN speeds:

Both those numbers seem low though.

Are there any Netgate lab numbers around? No matter official or unofficial.
Like OpenVPN SSL/TLS between two SG-5100, settings used for TLS key, Encryption Algo, Auth digest and so on and the speed to expect?

-Rico

SteveITS

@Rico
This has IPSec numbers: https://www.netgate.com/products/appliances/

This has a chart at the bottom for TNSR but shows pfSense on a SG-5100: https://www.netgate.com/blog/choosing-the-right-netgate-appliance.html

Rico

Yes...but this thread is about OpenVPN @pfSense.
Impossible to relate anything for OpenVPN with IPsec numbers...

-Rico

stephenw10

I expect to see over 100Mbps on the 3100 if you are using a CESA supported cipher, which AES-CBC should be.

I would also expect to see over 125Mbps on the 1100 using AES-GCM.

There are many variables etc!

Steve

Rico

Well I don‘t care about 5-10Mbps VPN traffic more or less.
Only would see a problem if you say like in your testings the speed is double or 1/3 more. :-)

-Rico

sgw

@stephenw10 coming here because I look for a quick solution to free CPU on a 2100.

We seem to outmax the hardware by running ~20 ovpn clients (1 Gbit/s WAN) plus a wireguard site to site tunnel and pfblockerng ...

A stronger hardware is in preparation but I look for a config tweak to decrease the load.

I wanted to switch from AES-256-GCM to AES-128-GCM in the ovpn-server to decrease the overall load, but the numbers mentioned above seem to tell me that that won't do much.

I am not looking for more throughput, I need to decrease the CPU load. (I already disabled telegraf, that was quite heavy as well)

hints welcome

stephenw10

What's using the CPU currently? Check the output of top -HaSP at the command line.

There may not be much you can do though. With a 1G WAN it will be possible to hit the limits of the CPU. You could potentially set limiters for connecting clients to prevent that.

sgw

@stephenw10 currently very low load.

Right now it doesn't look overloaded at all.

Now and then there is sftp-traffic between the two sites which are connected by the wireguard site to site tunnel. This, in combination with the other services maxes out the hw.

It's a multi-layered issue: the delivering server vm is restricted etc etc

Maybe I could tune the wg-tunnel somehow.

Right now I can't see any limits (of the 2100) hit ... in the morning we hit the CPU-limits all the time.

thanks

stephenw10

So what issues do you see when this happens?

If it's maxed out by traffic over the Wireguard tunnel you might apply a limit to that.

sgw

@stephenw10 thanks for asking.

I am in the process of pinpointing an issue that might not even have to do with openvpn. I think multiple things overlay each other and give different results at different times:

basically I have 2 pfSense plus 24.11 boxes in 2 sites "office" and let's call it "data center" ;-)

They are crossconnected with a wireguard site2site vpn, only the LANs are routed over wireguard.

The data center pfSense is also a ovpn-gw for customers, they access VMs via VPN. The VMs run on a 3-server proxmox cluster in the data center.
All that works fine.

They run a linux VM there also that provides update-zips for the customers via sftp. If the customer accesses the related URI via the DNS-record pointing to the WAN-IP of the datacenter-pfSense the download speed is fine.

If my customer accesses the same URI (using the same DNS-name and in turn WAN-IP of datacenter) from behind the office pfSense it's way slower. a tenth or so.

That's the initial issue, and I am digging through everything ...

Yes, sftp isn't cool, I try to switch to scp.

We outruled wireguard-usage. We used the IP only.
I upgraded the VM in terms of software, and edited the vCPU to "host". I switched to a virtio-NIC. etc etc

I have to ask the coder there if his software (the one his customers upgrade by pulling stuff via sftp) maybe caches something and that leads to this difference.

In the process of debugging yesterday I had times when the datacenter-pfsense maxxed out its CPU (that's the 2100), so I tried to remove load there by disabling telegraf etc ... / in the afternoon the load was low and the sftp-transfer still wasn't higher. The vCPU in the VM also plays a role etc

I am quite sure that I have routing and NAT set up OK. The line there is 1 Gbit/s symmetric, that also shouldn't be the bottleneck.
Still scratching my head here ;-)

thanks for reading all this, ideas welcome.