Measuring performance bottlenecks with IPSec?
-
Hello,
we have an IPSec link with a remote site that is used for hosting backups (both sides run with pfSense).
Although the link bandwidth is 100 Mbps, our backups (under IPSec link) rarely run at more than ~50 Mbps, with no other traffic present.
Some time ago, I tested a long transfer to the remote site via public IPs, i.e. same endpoints but outside the IPSec encapsulation, and it seemed to be faster.
So, I'm suspecting some limitation on the pfSense on the remote site (which indeed runs on a small Nano PC).
However, the f/w didn't seem to use e.g. 100% of CPU or be short on RAM, or logging error messages.So my generic question is, how do you check if an IPSec link is using too many h/w resources?
Thanks
-
Hello,
Your IPsec might not be using 100% of the CPU on your system, but it cloud be using 100% of a single core on the system, which could explain the bottleneck.You could give a try to the Optimal Encryption Settings (specially the AES128-GCM or other Algorithms that ends with GCM) and Enable Asynchronous Cryptography under the Scaling IPsec section here: https://docs.netgate.com/pfsense/en/latest/vpn/performance.html
Another option is for you to use Wireguard to connect to your remote site.
-
@minimos The most likely problem in your specific scenario is packet fragmentation because of the smaller MTU (Max packet size) inside a IPsec tunnel compared to the packet size on the RAW ethernet link. Your servers sends packets to the destination backup server with the max MTU size of the Enternet link they are using (1500 bytes). When that packet goes into the IPsec tunnel, it does not fit because of IPsec overhead, and is thus split into two packets sent to the destination pfSense that reassembles them to one packet and forwards it.
That will cost you upwards of 40-50% of the max bandwidth on “slower” links with above LAN latency.
You can diagnose this easily by doing a packet capture on the IPsec Interface on the sending pfSense. See if most frames are fragmented.
The problem is pfSense does not adhere to proper MTU Path Discovery, so your servers cannot discover that they should use smaller packets to allow them to fit into the tunnel. You can however force it work by setting up MSS Clamping in SYSTEM -> ADVANCED -> Firewall & NAT -> VPN Packet processing. Try enabling maximum MSS and leave it at the default value of 1400 bytes.
That will cause your endpoints to “downsize” all TCP packets to 1400 bytes maximum so they can fit into the tunnel unfragmented.
It only works for TCP, so please check that the backups are copied using TCP - If it’s done using UDP, this will have no effect.Also, IPsec has a “lot” of overhead, so don’t ever expect to reach 100mbit (link speed) in your transfer. It will reach maybe 85mbit at best.
Last thing. You need to move/transfer the backup with a latency efficient protocol. That leaves Microsoft SMB out. SMB is horrible at maximizing linkusage when latency and smaller packets are involved. It’s all over the place and horribly inefficient.
-
Set the MSS for AES to 1328, if you want to avoid padding.
Less padding allows you to move more Data over the tunnel.