APU --> SG-1100, Faster at IPSec; Slower at Everything Else

TheWaterbug

@stephenw10 said in APU --> SG-1100, Faster at IPSec; Slower at Everything Else:

The APU doesn't have any specific hardware for crypto off-loading so really it's jut about speed vs relative security. I would consider AES-GCM 256 more than sufficiently secure and fast enough.

Do you think the APU would do 100 Mbps IPSec with AES-GCM 256?

stephenw10

Probably is about the best I can say. There are a lot of variables.

TheWaterbug

I don't know if this is of interest to anyone but me, but I fiddled with the encryption/hashing settings between the MBT-2220 at the office (2.4.4/Hardware crypto AES-CBC,AES-XTS,AES-GCM,AES-ICM) and the SG-1100 at my home (22.05/Hardware crypto AES-CBC,AES-CCM,AES-GCM,AES-ICM,AES-XTS,SHA1,SHA256,SHA384,SHA512).

Line speed is >>>> 250/250 at both ends. It's actually 1000/1000 nominal, but I'm router limited.

iperf reports:

AES256-GCM (256 bits) SHA256 14 (2048 bit)

[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.00 sec 157 MBytes 132 Mbits/sec sender
[ 4] 0.00-10.00 sec 157 MBytes 132 Mbits/sec receiver

AES128-GCM (128 bits) SHA256 14 (2048 bit)

[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.00 sec 165 MBytes 139 Mbits/sec sender
[ 4] 0.00-10.00 sec 165 MBytes 139 Mbits/sec receiver

AES CBC (256 bits) SHA256 14 (2048 bit)

[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.00 sec 146 MBytes 123 Mbits/sec sender
[ 4] 0.00-10.00 sec 146 MBytes 123 Mbits/sec receiver

AES CBC (128 bits) SHA256 14 (2048 bit)

[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.00 sec 149 MBytes 125 Mbits/sec sender
[ 4] 0.00-10.00 sec 149 MBytes 125 Mbits/sec receiver

AES CBC (128 bits) SHA1 14 (2048 bit)

[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.00 sec 168 MBytes 141 Mbits/sec sender
[ 4] 0.00-10.00 sec 168 MBytes 141 Mbits/sec receiver

Sampling error is ±10 Mbps for any particular configuration.

Given the modest differences, I'm going to stick with AES256-GCM (256 bits) SHA256 14 (2048 bit).

What other knobs can I turn to improve IPSec throughput between these two boxes?

TheWaterbug

I put the APU back in place of the SG-1100, temporarily, to repeat the experiment, and I got the same speed regardless of what encryption settings I chose:

[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.00 sec 67.0 MBytes 56.2 Mbits/sec sender
[ 4] 0.00-10.00 sec 66.9 MBytes 56.2 Mbits/sec receiver

I've got the SG-1100 back in place, and the 125/125 is a very nice upgrade from the 30/30 I was getting just a few days ago.

stephenw10

You have safexcel enabled at the 1100 end? If so use any of the ciphers it supports:
https://www.freebsd.org/cgi/man.cgi?query=safexcel

AES-GCM is inherently faster as it doesn't require a separate authentication step. So I would have expected AES-GCM 128 to be the fastest.

Enabling asynchronous crypto can make a huge improvement on systems that support it. Thats in the IPSec Advanced settings. It's probably enabled in the 1100 but may not be in the MBT.

Steve

TheWaterbug

@stephenw10

Yes, SafeXcel is on. That link says it "implements SHA1 and SHA2 transforms," but does not specifically list SHA256. I compared SHA1 vs. SHA256 and didn't see any difference in performance.

Async was turned on in the SG-1100 and off in the MBT-2220, so I turned it on in the MBT-2220, but that didn't make any difference in performance, either. In fact it iperfs 5-10 Mbps slower, but that could easily be sampling error.

Thanks!

TheWaterbug

@thewaterbug said in APU --> SG-1100, Faster at IPSec; Slower at Everything Else:

Async was turned on in the SG-1100 and off in the MBT-2220, so I turned it on in the MBT-2220, but that didn't make any difference in performance, either. In fact it iperfs 5-10 Mbps slower, but that could easily be sampling error.

Actually I just turned async off on both ends, and now it iperfs at 143/143, vs. 120/20 when async is on at both ends. I don't think it's sampling error, because it's repeatable (n=3 trials).

And now there's no measurable difference in throughput between AES-GCM128 and AES-GCM256. They both test right around ~135-140 Mbps.

stephenw10

Hmm, interesting. I wonder if you're hitting some other limit there then.

140Mbps is about what I expect from the 1100 though.

TheWaterbug

@stephenw10 said in APU --> SG-1100, Faster at IPSec; Slower at Everything Else:

Hmm, interesting. I wonder if you're hitting some other limit there then.

140Mbps is about what I expect from the 1100 though.

Thanks! That's confirmation that I'm not doing anything grossly incorrect.

TheWaterbug

@stephenw10 said in APU --> SG-1100, Faster at IPSec; Slower at Everything Else:

Both the MBT and the APU are capable of running the current pfSense CE version, 2.6.

I found my null modem adapter, so I now have one of my APU units up and running 2.6.

I need to run over to my 3rd site and swap it into place of the other APU, and then upgrade that one to 2.6, and then all of my devices will be at the latest release.

Thanks!