Netgate 2100 IPsec S2S AES GCM and SafeXcel mbuf overload

[NOCling]

Ok, now it works with GCM MBUF 3% (1776/70608).

My parents' ISP changed the provisioning of the cable modem at the beginning of the week and since then the pfsense can no longer obtain an IPv6 IP Prefix.
Now the SG-2100 only works with IPv4 and the problem is not there.
SafeXcel Crypto: Yes (active)
Asynchronous Cryptography (active)

When I encounter the error, the IPsec tunnel connects using the IPv6 WAN IP on both sides using port 500, now it is the IPv4 using port 4500.
Looks like it interacts with IPv6 in some way, which triggers the MBUF overload.
So now I have to see if I can fix the IPv6 problem in order to generate the error again.

Interesting issue...

[keyser]

@nocling said in Netgate 2100 S2S mbuf overload:

Ok, now it works with GCM MBUF 3% (1776/70608).

My parents' ISP changed the provisioning of the cable modem at the beginning of the week and since then the pfsense can no longer obtain an IPv6 IP Prefix.
Now the SG-2100 only works with IPv4 and the problem is not there.
SafeXcel Crypto: Yes (active)
Asynchronous Cryptography (active)

When I encounter the error, the IPsec tunnel connects using the IPv6 WAN IP on both sides using port 500, now it is the IPv4 using port 4500.
Looks like it interacts with IPv6 in some way, which triggers the MBUF overload.
So now I have to see if I can fix the IPv6 problem in order to generate the error again.

Interesting issue...

Very good observation that it is related to tunneling over IPv6 vs IPv4. When connected on port 500 it’s using ESP directly where as on port 4500 its using NAT traversal. If that has influence or if its only the protocol version needs to be tested.

Good find!

[NOCling]

Yesterday 8h of VPN Backup with GCM and SafeXcel MBUF Overload incoming.

At the moment it looks like SafeXcel is triggering the MBUF overload, but I'll watch it again for another 24 hours.

[keyser]

@nocling said in Netgate 2100 S2S AES GCM and SafeXcel mbuf overload:

Yesterday 8h of VPN Backup with GCM and SafeXcel MBUF Overload incoming.

At the moment it looks like SafeXcel is triggering the MBUF overload, but I'll watch it again for another 24 hours.

Over IPv6 again, or this time over IPv4?

[NOCling]

IPv4, IPv6 is broken by the ISP and i don't have the time to investigate for a fix, IPsec is more important.

I had only changed SaveXcel to inactiv.

[keyser]

@nocling said in Netgate 2100 S2S AES GCM and SafeXcel mbuf overload:

IPv4, IPv6 is broken by the ISP and i don't have the time to investigate for a fix, IPsec is more important.

I had only changed SaveXcel to inactiv.

Okay, so it’s a general safeXcel issue when using GCM on the 2100 in your situation. Interesting if anyone can confirm this, or it’s some setting/parameter in your specific situation

[NOCling]

1,5 Days without SaveXcel activ, no problem here:

[NOCling]

AES-GCM-128 and SafeXcel active again, the MBUF is already running full again.

Also, the GUI is slower than usual when I access it through the S2S tunnel.

Now im back to CBC-256 and a Reboot to clear the MBUF.

[keyser]

@nocling said in Netgate 2100 S2S AES GCM and SafeXcel mbuf overload:

AES-GCM-128 and SafeXcel active again, the MBUF is already running full again.

Also, the GUI is slower than usual when I access it through the S2S tunnel.

Now im back to CBC-256 and a Reboot to clear the MBUF.

Seeams pretty clear where the issue is
Whats the speed difference (throughput) between CBC-256 and GCM-128 in your setup where i assume the SG-2100 is the bottleneck and not your WAN speed

[NOCling]

The WAN upload is the limit.
But with the nes appliances on both ends, it would be nice to be able to use GCM.
So it would be nice if someone from Netgate could now take a look at the whole thing and see if the error can be reproduced on them.

[otsego]

@NOCling

Thanks for posting this! I have a Netgate 6100 connected to a 2100 through a VTI IPSec tunnel. Once there was medium+ traffic from the 6100 to the 2100, such as file transfer from one NAS to another, nothing too heavy, internet speed of 100 mbits, the entire VPN tunnel crashed and would not come up again until a reboot. I couldn't understand what on earth it was and tried every single setting and detail but it never fell on me that AES-GCM could be the issue.

I now changed from AES-GCM to AES-CBC on the site to site tunnel and it suddenly became rock stable.

So there is definitively something to the AES-GCM theory on the Netgate 2100

[keyser]

@phlmike said in Netgate 2100 S2S AES GCM and SafeXcel mbuf overload:

@nocling If it is a reproduceable bug in the 2100, then you need to file a bug report in redmine.
https://redmine.pfsense.org/

Please you guys - remember to fill out the redmine bugreport. Otherwise this won’t get fixed.

[forum1]

It appears Bug #13074 ( https://redmine.pfsense.org/issues/13074 ) has been created for this.