SG-3100 IPsec Performance Issues
-
Greetings everyone!
I recently purchased an SG-3100 for a satellite office but have been having issues with the speed through the IPsec tunnel connected to our main office. I currently am only able to get about 12-14mb down/up, our internet connection is 400/20 down/up. Because the tunnel speed is synchronous it makes me think I have reached the limit of the processor or I have made a mistake on the configuration. I played around with various crypto/hashing combinations and haven't seen much difference, I landed on AES128-SHA256-ECP256 since elliptic curve is normally faster than RSA.Am I missing something? I have also played around with various hardware crypto settings, but I haven’t found an easy way to tell what crypto devices and algorithms are compatible with IPsec, reading in the forums I see conflicting data about what is supported, and it seems that in the last few versions there have been some significant changes. I was considering trying OpenVPN to see if that has different results.
Setup:
Main Office:
Xeon E31230 custom box HA Master
SG-8860 HA Backup
Internet 100mb x 100mb (speed test 90mb x 86mb)Satellite Office:
SG-3100
Internet connection 400mb x 20mb (speed test 450mb x 22mb)Things I have tried:
Swapping Main/Backup Firewalls
Enabling and Disabling hardware crypto
Multiple ciphers/hash combos
MSS Clamping down to 1400 -
I was able to increase the speed slightly by setting the MSS Clamping to 1360 although the ping tests I performed showed that it shouldn't be required. I am still scratching my head as to why it isn't performing faster, I attached iperf3 results below.
(Server Main Office)
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.00 sec 1.50 MBytes 12.6 Mbits/sec
[ 4] 1.00-2.00 sec 1.75 MBytes 14.7 Mbits/sec
[ 4] 2.00-3.00 sec 2.12 MBytes 17.8 Mbits/sec
[ 4] 3.00-4.00 sec 1.88 MBytes 15.8 Mbits/sec
[ 4] 4.00-5.00 sec 2.12 MBytes 17.8 Mbits/sec
[ 4] 5.00-6.00 sec 2.12 MBytes 17.8 Mbits/sec
[ 4] 6.00-7.00 sec 1.75 MBytes 14.7 Mbits/sec
[ 4] 7.00-8.00 sec 2.12 MBytes 17.8 Mbits/sec
[ 4] 8.00-9.00 sec 1.88 MBytes 15.7 Mbits/sec
[ 4] 9.00-10.00 sec 2.00 MBytes 16.8 Mbits/sec
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.00 sec 19.2 MBytes 16.1 Mbits/sec sender
[ 4] 0.00-10.00 sec 19.2 MBytes 16.1 Mbits/sec receiver(Server Satellite Office)
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.00 sec 640 KBytes 5.24 Mbits/sec
[ 4] 1.00-2.00 sec 1.38 MBytes 11.5 Mbits/sec
[ 4] 2.00-3.00 sec 1.50 MBytes 12.6 Mbits/sec
[ 4] 3.00-4.02 sec 1.88 MBytes 15.5 Mbits/sec
[ 4] 4.02-5.00 sec 1.62 MBytes 13.9 Mbits/sec
[ 4] 5.00-6.01 sec 1.25 MBytes 10.4 Mbits/sec
[ 4] 6.01-7.01 sec 1.38 MBytes 11.5 Mbits/sec
[ 4] 7.01-8.01 sec 1.38 MBytes 11.5 Mbits/sec
[ 4] 8.01-9.00 sec 1.75 MBytes 14.8 Mbits/sec
[ 4] 9.00-10.01 sec 1.88 MBytes 15.6 Mbits/sec
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.01 sec 14.6 MBytes 12.3 Mbits/sec sender
[ 4] 0.00-10.01 sec 14.5 MBytes 12.2 Mbits/sec receiver -
To get the best IPSec performance from the SG-3100 you need to be sure the CESA crypto hardware is being used.
To do that you need to two things:
Set 'Cryptographic Hardware' to BSD Crypto Device in System > Advanced > Misc.Make sure you;re using a cryto algorithm that CESA supports. That means AES-CBC, 128, 192 or 256 and sha1 or sha256.
AES-CBC-128 and SHA1 will give you best performance.I expect it to fill that 86/22Mbps connection easily based purely on processing.
Steve
-
@stephenw10 would the dh group prevent hardware offloading?
-
Hmm, you are using PFS key 28? I'm using 14 and can see connections close to my line rate at 70+Mbps even between London and Austin. I'm not sure what gets off-loaded there. Nothing obvious shown in the driver:
https://github.com/pfsense/FreeBSD-src/blob/ff7d4801f1b88de656e028209818ff005e8a1353/sys/dev/cesa/cesa.c#L1229Are you able to test group14?
I'm assuming you're using IKEv2. The phase 2 settings are where the speed is decided.
Steve
-
I just tested it no real change I am seeing some more variability in the results though, I am not seeing dropped packets through the link so I wonder if there is something between the route on these two, they are just down the road a few miles but are using different providers (Main is using Timewarner/Spectrum fiber provided by a third party and the remote office is Timewarner/Spectrum Cable).
I will contact my providers and see if I can track something down. I have been testing everything I can think of for the last two weeks and am about ready to price out running a wireless point-to-point link from Ubiquiti if I can't figure it out haha.
Phase 1
AES_CBC
HMAC_SHA2_256_128
PRF_HMAC_SHA2_256
MODP_2048Phase 2
AES_CBC
HMAC_SHA2_256_128
IPComp: none -
I ran some speed tests yesterday between the two sites outside of the tunnel and was seeing similar results the issue must be the connections. I am reaching out to my providers today to see if they can resolve the problem.
-
We have, on occasion, seen paths that are decidedly unfriendly to ESP but work well using NAT-T (UDP 4500).
Since you are seeing similar issues outside the tunnel this is probably moot for you but I wanted to put it out there.
Note that there is no way for force NAT-T using IKEv2 but you can using IKEv1.
-
Hi, the ipsec speed is limited by upload speed at remote site due to ACKs.
Having said that I have issue with SG-3100 - I cannot get greater than 45 Mbps on a physical 100Mbps fiber link to my home pfsense which is 940Mbps fiber, whereas I can 380Mbps+ to a SG-5100 on physical 500Mbps link, and 180Mbps to a SG-4860 on a 200Mbps physical link. My home is a home-built pfsense running i7. I will open another thread on this.
