Issue with VPN Bandwidth, even with scaling

s0p4L1n

Hi !

I work in company where we have 2x pfSense Hardware in HA: XG-1541 BASE
We have 1 Gbps UP/DOWN Link.

We have setted up 5 OpenVPN Servers for scaling.

Settings are the same for each servers:

Remote Access (SSL/TLS + User Auth)
UDP on IPV4 Only
Active Directory Login + OTP Code (Radius)
DH Parameter: 2048 bits
AES-128-GCM
SHA-256
Hardware Crypto: Intel RDRAND engine - RAND
Certificate Depth: Two (Client + Intermediate + Server)
Use fast I/O operations with UDP writes to tun/tap. Experimental. IS CHECKED

I've followed the documentation about scaling but even with that, it seems not improving as first announced at the beginning: Scaling OpenVPN

We use VPN to secure our employees remote access to their workstation.
They are working from home with two monitors and one workstation. They work as they are at the company.

My question is, why the bandwidth is so limited, when we have the top official hardware for pfSense ? I've already read all threads talking about tweaking with some arguments with also no result.

I've read some users can reach 800 Mbps on OpenVPN server in Linux, and same configuration can only reach 80-100 Mbps in pfSense.

What solution do you recommend ? Choose OpenVPN Server Access instead ?
I didn't try yet IPSec for remote, is there better bandwidth with this VPN solution ?

Shoud we by specific hardware for VPN only ?

Thanks,

DaddyGo

@s0p4l1n said in Issue with VPN Bandwidth, even with scaling:

can reach 800 Mbps on OpenVPN server in Linux

Hi,

OpenVPN is a single-threaded beast...
so the speed of the CPU core (1 core) is very important

the more tunnels are used, the worse the situation gets

so, if you have many clients connected, then definitely the IPSec is the good choice
(to achieve good speed)

the 800 Mbps mentioned above for OpenVPN on Linux, an incredible speed that I've never seen for OVPN

unique, one tunnel, super HW (f.e.: 11th Gen Intel Core with 5Ghz CPU clock)

Pippin

It's a 8 core CPU + AES-NI, so 5 servers=5 cores
No idea what this device is capable of but try and set Hardware Crypto to None in OpenVPN server configuration.

DaddyGo

@pippin said in Issue with VPN Bandwidth, even with scaling:

It's a 8 core CPU + AES-NI, so 5 servers=5 cores

yes, I agree

my best result ever on this Epyc 3151 4C/8T with 2,9 GHz core speed (DDR4 2666),
620 mbps, GPON 1/1Gig + Intel i710 - 1 client also with GPON ISP, and 2 Km away

I don't think Xeon 2.1GHz will give better results

s0p4L1n

@daddygo IPSec on pfSense does not match the security needs that our client is requiring
(ISO 27001) and for W10 Client IPSec, it must be SHA1

We will also have a new fiber line in July: 10G Optic Fiber
We also have a failover Fiber 1G
How could we setup 400 Remote Access VPN and for each user at least 10-15 Mbps ? Is it possible on pfSense or should we buy specific harware ?

Thanks

DaddyGo

@s0p4l1n said in Issue with VPN Bandwidth, even with scaling:

How could we setup 400 Remote Access VPN and for each user at least 10-15 Mbps ? Is it possible on pfSense or should we buy specific harware ?

That's quite a few clients (400), have you thought about this https://www.tnsr.com/, especially because the 10Gig WAN or more capacity?

BTW:
We serve 250 OVPN clients on a Cisco UCS-C240M4 with pfSense (2 Xeon CPU, DDR4 + Intel I710)

+++edit:
I just quickly looked at the Cisco HW configuration (because I didn't remember):

2 x Intel E5-2667 v4

running with CPU, because of the relatively higher CPU clock (3.2 GHz) and 8C/16T

The amount of RAM is not so much a significant factor as the speed DDR4 64G / 2 400

NIC: Intel x710-da4 + LOM I350
+VIC1227 (but that is not relevant here)

Clients with good ISP speeds reach 20 -30 Mbps, on 10 OVPN servers in total,
but as we know this also depends on the simultaneous load

s0p4L1n

@daddygo Yes that's similar to our case;

We are just limited by the number of cores (8) and the frequency (2.1GHz) of the Netgate Harwdare.

So in a theorical way, we can do:

8 VPN instances max based on our 8 cores
OpenVPN with AES-NI consume 12Mhz for each mbps transferred in one direction.
With our 2.1Ghz core frequency, we can reach 175 Mbps per tunnel (that's what I get with iperf).

The bottleneck here is our client number needs (400). And sometimes some clients use 100% of the bandwidth because they are loading high quality image on their screen.

I will investigate to find the best solution.
Thanks for the help !

DaddyGo

@s0p4l1n said in Issue with VPN Bandwidth, even with scaling:

100% of the bandwidth because they are loading high quality image

We have several radio stations, ergo we had the same problem with transmitting raw uncut *.WAV audio files.

We then deployed the Cisco UCS and its performance is satisfactory.

Good luck with your work