Netgate 6100 openvpn DCO acceleration issue
-
I have found an issue with the latest pfsense+ on Netgate 6100 when attempting to use DCO in a site-to-site tunnel.
With QAT as the hardware crypto , selecting DCO causes site to site traffic to drop all the way down to 20mbit. Same tunnel with DCO unchecked does 100mbit to the other side (only limited by the other side which is a SG 3100. Oddly the 3100 on the other side is sped up considerably by enabling DCO, and it has no hardware crypto that I can tell. AES-128 with a SHA256 is what is being used.
If I turn off DCO on the 6100, it maxes the connection at 100+mbit to the 3100.
I have tried selecting IPSEC-MB (no change.) I have tried AES-NI + cryptodev instead of QCAT. No change, turning DCO on still drops speed to ~20mbit from 100.
Anyone have anything I can try? thanks. This is more of a datapoint item as I really don't necessarilly need DCO on the 6100 side as it is capable enough for this tunnel, with CPU usage in the 15-17% range at 100+mbit (so I assume it isn't even maxing a single core, or it would show 25%+ since it is a quad core.) I am more posting this for others who may want to accelerate an OpenVPN site to site tunnel with their 6100 at speeds over 150-200+mbit or wherever a single core would be maxed on the 6100, as then you would need DCO to do something.
-
Is this in 23.09.1? There is an OpenSSL bug in 23.09 that could present like that.
You would have to reboot or manually unloaded the QAT module after selecting a different hardware crypto type.
Steve
-
THis is on 23.09.1 . I rebooted after trying all of AES-NI w/CryptoBSD alone, AES-NI with IPSEC-MB checked, AES-NI alone, rebooting between each change of setting, and trying each with both DCO 'on' and 'off'. In all cases, traffic slowed to a crawl at 20mbit/s when DCO was turned on with AES-128-GCM as the cipher.
Something is not right. As I said above, the 3100 on the other end with no crypto acceleration shows 25%+ gains when DCO is turned on, so I left it on on that side of the tunnel.
-
Hmm, and with DCO disabled you just get 100M in all cases because that's the link limit?
It shouldn't make any difference but which side is the server here?
-
Yes, the rate is limited by Fiber line on the client end.
The server is the 3100, The client on the tunnel is the 6100. This is a new tunnel, so I have no data on DCO before this version.
The testing I am doing is with a veeam backup copy job, so is consistently trying to fill the pipe. It might hit 30mbit with DCO on, as I am looking at the traffic graph, but it is mostly sitting around 20. (turning it off, saving the OPENVPN client and rebooting, and the job immediately fills the pipe up to near 100mbit.
-
Hmm, I can't replicate that here. How exactly are you testing?
I'm testing here using iperf directly on each pfSense device using the server side LAN IP as the target address.
I see ~125Mbps without DCO and ~220Mbps with DCO enabled on the 6100. In the later case it's limited by the server side CPU.
-
I will test again after I get out of some meetings here, but just to verify what you set:
AES-128-GCM w/ SHA256 cipher
SSL/TLS Open VPN tunnel
QAT on , DCO on (did you set IPSEC-MB on as well, or ?)I will run another test and see what I get, but I am just running a job that goes over the tunnel network and sends data to a sever on the switched interface on the 3100 (not LAN interface, but the 'switched LAN' interface of the 3100.
It can fill a 1gig pipe, so it easily fills all available bandwidth being provided. It is so extreme, that I just watch the traffic graph and can see what it is limited at. I am unsure on the parameters of an iperf test, but I could try that as well with some research.
-
I did also have IPSEC-MB enabled on the 6100 for that test but it shouldn't make any difference. But DCO also shouldn't behave like this so I'll retest without it.
-
Just tested again. Actual maximum is about 15mbit when DCO turned on and rebooted. When DCO is turned off, and the tunnel reconnects, it immediately goes back to 100mbit.
-
Are you able to test with iperf on the firewall as I was doing?
That might rule out some obscure routing issue etc.
-
@stephenw10
Let me look into this weekend. I won't have access to the server site until Sunday. Thank you for your attention on this; hopefully can help someone with similar issue who needs more from the 6100 in openvpn.
