pfSense 2.5 to pfSense 2.5 IPsec tunnel fails to connect

[beejee]

After upgraded to version 21.02-RELEASE-p1, my Netgate XG-7100 kept getting disconnected to a remote Cisco RV042 (even though it showed as connected). The IPsec tunnel had been working smoothly for a year without any hiccup now decided to act up. There is no pattern when it would stop the connection, most of the time just within couple minutes. I applied the 7 patches as advised about but still no luck. My tunnel is using IKEv1, AES_CBC (128), HMAC_SHA1_96, PRF_HMAC_SHA1, MODP_768. I am under the pressure to get it back on track. Please help! Thank you.

[beejee]

I finally got my IPsec tunnel to work without interruption on my Netgate XG-7100 by turning of the Hardware Crypto. I believe it is a work around solution since the Hardware Crypto has to be off.

[jimp]

@beejee said in pfSense 2.5 to pfSense 2.5 IPsec tunnel fails to connect:

I finally got my IPsec tunnel to work without interruption on my Netgate XG-7100 by turning of the Hardware Crypto. I believe it is a work around solution since the Hardware Crypto has to be off.

Which hardware crypto option did you have enabled on there? If it was AES-NI, that sounds similar to an issue we're already tracking. If it's not, it could be a different problem. On the XG-7100 you can switch from AES-NI to QAT which should be equal to or faster in performance and potentially more stable as the issues we're aware of only affect AES-NI at the moment.

[beejee]

@jimp said in pfSense 2.5 to pfSense 2.5 IPsec tunnel fails to connect:

Which hardware crypto option did you have enabled on there?

You were right. It was "AES-NI and BSD Cryto Device (aesni,cryptodev)" option. I believe it would be the best option for my appliance overall.

I will try the "QAT" as your suggested sometime tonight. Thank you!

[beejee]

@jimp said in pfSense 2.5 to pfSense 2.5 IPsec tunnel fails to connect:

On the XG-7100 you can switch from AES-NI to QAT which should be equal to or faster in performance and potentially more stable

Yes my XG-7100 IPsec tunnel is working smoothly with "Intel QuickAssist (QAT)" on. My CPU Type is "Intel(R) Atom(TM) CPU C3558 @ 2.20GHz". I actually don't know where to check if my IPsec tunnel is really using the QAT since I didn't see it is listed anywhere in the Dashboard. The Dashboard mentioned that "AES-NI CPU Crypto: Yes (inactive)" instead.

[jimp]

The next update will show QAT on the dashboard properly, but for now you can check:

1. That the module is loaded:

   : kldstat | grep qat
    5    1 0xffffffff84322000    146e0 qat.ko
    6    1 0xffffffff84337000    9f521 qat_c3xxxfw.ko
   

2. That the device is consuming interrupts (will increase as traffic is encrypted/decrypted through IPsec):

   : vmstat -i | egrep 'total|qat'
   interrupt                          total       rate
   irq300: qat0                      489041          0
   

[beejee]

@jimp Awesome! I think my IPsec is really utilizing the QAT. Thank you so much!
I downloaded a large file through the tunnel and checked the stat:

:kldstat | grep qat
 3    1 0xffffffff83f22000    146e0 qat.ko
 4    1 0xffffffff83f37000    9f521 qat_c3xxxfw.ko

:vmstat -i | egrep 'total|qat'
interrupt                          total       rate
irq293: qat0                       43576          1
irq294: qat0                       27168          1
irq295: qat0                        7909          0
irq296: qat0                       18777          0

:vmstat -i | egrep 'total|qat'
interrupt                          total       rate
irq293: qat0                      396393          9
irq294: qat0                       39664          1
irq295: qat0                       19005          0
irq296: qat0                       47598          1

[brians]

@jimp Thanks I had problem with tunnel to a Cisco router where I could ping but then sending any traffic through would kill the tunnel. Switching my 5100 to QAT seems to have fixed this issue.

I have another tunnel between the SG-5100 and an older SG-2220.. enabling QAT on the 2200 is slower than using AES-NI - iperf3 test are 220Mbps with AES-NI and only around 100Mbps on QAT when I enable it on the SG-2220.

[danjeman]

@jimp Guessing the Dashboard display for QAT or other crypto modules didn't make it to 21.02.2 - at least my XG-7100's still show 'AES-NI CPU Crypto: Yes (inactive) - no mention of QAT anywhere that I can find on a Dashboard widget etc.

[jimp]

@danjeman said in pfSense 2.5 to pfSense 2.5 IPsec tunnel fails to connect:

@jimp Guessing the Dashboard display for QAT or other crypto modules didn't make it to 21.02.2 - at least my XG-7100's still show 'AES-NI CPU Crypto: Yes (inactive) - no mention of QAT anywhere that I can find on a Dashboard widget etc.

It's there on 21.05 snapshots, didn't make it into 21.02.2.