IPSec is very slow between two pfsense routers

[patrick.pesegodinski]

@michmoor

The test is carried out by transferring files from the server at one end to computers at the other end.

Should I reduce the MSS at both ends?

Main Pfsense Configuration:

Intel(R) Core(TM) i5-8400 CPU @ 2.80GHz
Current: 2800MHz, Max: 2801MHz
6 CPUs: 1 package(s) x 6 core(s)
AES-NI CPU Crypto: Yes (active)
QAT Crypto: No

Secondary Pfsense Configuration:

Intel(R) Core(TM) i3-8100 CPU @ 3.60GHz
4 CPUs: 1 package(s) x 4 core(s)
AES-NI CPU Crypto: Yes (active)
QAT Crypto: No

[michmoor]

@patrick-pesegodinski
What NICs are used?
Try reducing MSS on both.
Are these SMB file transfers?

[patrick.pesegodinski]

@michmoor

• NICs was TP-LINK TG-3468;
• SMB file

[patrick.pesegodinski]

@michmoor Should the IPsec tunnel be restarted after MSS modification?

[michmoor]

@patrick-pesegodinski doesnt have to be i believe.

[keyser]

@patrick-pesegodinski I know this sounds like BS, but you need to try and benchmark the VPN connection with something other than SMB. SMB is NOTORIOUSLY bad on “less than 1500 bytes” MTU links like a VPN. It’s all over the place if any fragmentation is involved.
So try and clamp down your MSS and benchmark it with a iPerf3 TCP test.

[patrick.pesegodinski]

@keyser I understand your thinking, but with OpenVPN I get transfer rates 10x higher than IPSEC.

[keyser]

@patrick-pesegodinski i know, and thats likely because OpenVPN knows how to participate in MTU Discovery so the SMB Client knows the proper packet sizes to use.
IPSec VPN on pfsense does not play Nice in this Area - it’s a known bug and has been for years, But unfortunately IPSec VPN sees Very little developer love, so we have to work around it.
Thats why we need MSS clamping and an iPerf3 test - then we really know where the culprit is buried.

[patrick.pesegodinski]

@keyser hello friend.

The test iperf

Connecting to host 172.29.0.1, port 5201
[ 5] local 10.0.0.1 port 45903 connected to 172.29.0.1 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 26.3 MBytes 221 Mbits/sec 19 64.1 KBytes
[ 5] 1.00-2.00 sec 27.1 MBytes 227 Mbits/sec 15 61.8 KBytes
[ 5] 2.00-3.00 sec 26.7 MBytes 224 Mbits/sec 15 82.0 KBytes
[ 5] 3.00-4.00 sec 27.1 MBytes 228 Mbits/sec 28 57.6 KBytes
[ 5] 4.00-5.00 sec 26.1 MBytes 219 Mbits/sec 16 71.2 KBytes
[ 5] 5.00-6.00 sec 26.6 MBytes 224 Mbits/sec 19 67.0 KBytes
[ 5] 6.00-7.00 sec 27.1 MBytes 227 Mbits/sec 15 27.4 KBytes
[ 5] 7.00-8.00 sec 26.5 MBytes 223 Mbits/sec 36 68.6 KBytes
[ 5] 8.00-9.00 sec 26.8 MBytes 225 Mbits/sec 18 81.6 KBytes
[ 5] 9.00-10.00 sec 21.0 MBytes 176 Mbits/sec 67 81.7 KBytes

---

[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 261 MBytes 219 Mbits/sec 248 sender
[ 5] 0.00-10.00 sec 261 MBytes 219 Mbits/sec receiver

iperf Done.

[keyser]

@patrick-pesegodinski Is this before or after clamping down MSS?
You are seeing 220mbit which is around 25MB/s (Half your OpenVPN throughput). But I cannot see if you tried a single session iPerf or the default with multiple parallel streams.

[patrick.pesegodinski]

@keyser Test performed without changing the MSS.
I used the iperf client in Pfsense itself with the default settings.

[michmoor]

@patrick-pesegodinski

Post your iPerf syntax here so we can understand what you are doing.
Something along the lines of

iperf3 -c 192.168.70.26 -P 50 -t 30

Also post for us the before MSS change and after MSS change results.

[michmoor]

@keyser
I found something in redmine for MTU suggestions.

https://redmine.pfsense.org/issues/14508

[patrick.pesegodinski]

@michmoor Excuse my ignorance, but I'm not familiar with iperf. Which fields do I need to adjust?

[michmoor]

@patrick-pesegodinski
do not use iperf on the firewall.
use iperf between your clients (sitting behind your firewall).

[patrick.pesegodinski]

@michmoor 1f121d95-5939-4faa-a741-3386bf11b08d-TESTE.txt

The test with MSS standard.

[NOCling]

https://packetpushers.net/ipsec-bandwidth-overhead-using-aes/
Best MSS for IPsec tunnel Model 1328, if you use transport Mode and a 1500 WAN line, you can use 1372.

[patrick.pesegodinski]

@NOCling hello friend.

Thanks for helping me. Your information about MSS with 1328 solved my problem.

[iulianteodor]

What should be the average IPSec VPN speed between two pfsense with 1Gb NIC interface and 1Gbps internet?
I've seen all kinds of comments on different forums that the speed could be higher, but I can't do more than that:

[optimusprime]

@patrick-pesegodinski Did you Apply MSS on WAN interface settings and does it need to be same on both sides of tunnel ?