Hardware support for Intel QuickAssist?

[oletuv]

@BlueKobold:

–---
So its really nice to know it now better, but I am also a little bit sad about that information.

Yes, I was also very eager to buy a Xeon D-15x8 Mini-ITX pc and is very disappointed about the confirmation that the otherwise attractive Xeon D-15x8 processors don´t have QuickAssist onboard. Well, life will go on, sort of  :D

[Guest]

this may be redundant by this point but I want to say that I'm personally a little bit disenfranchised with the AES-NI naming convention. It suggests that it is something new entirely when in this is not the case when in all actuality, it is more of an addition. I wish Intel would have provided a better name for this new tech but for me it helps to think of it this way; AES + NI = AES-NI because the chip is using the added instructions to assist AES.

[oletuv]

@oletuv:

Since support for QuickAssist probably will be added to pfSense during 2016, I think a Atom C2758 SKU would be a better option for a dedicated pfSense box. Personally I´m going to order a prebuilt Mini-ITX with A1SRi-2758F motherboard from Supermicro. QuickAssist onboard and a lot cheaper than a Xeon D-15x8 based config too.

Hm.. Common sense tells me that Atom C2758 with onboard QuickAssist acceleration is a better option than the more costly Xeon D-15x8 with no onboard QuickAssist acceleration for a pfSense build. However, Atom Rangeley is an older processor released Q3/13 while the Xeon D-15x8 processors are newly released.

I suppose the new Xeon D processors will outperform Atom Rangeley except for crypto-heavy stuff like VPN. I will configure my pfSense firewall with IPSEC or OpenVPN, but will probably use it infrequently, typically to access my home network in Norway from my Spanish home.

The Supermicro X10SDV-6C+-TLN4F board (Xeon D-1528) with active CPU-cooling looks extremely tempting. ;)

Frank, what do you think?  ;D

[Guest]

Hm.. Common sense tells me that Atom C2758 with onboard QuickAssist acceleration is a better option than the more costly Xeon D-15x8 with no onboard QuickAssist acceleration for a pfSense build.

For SMB or home usage it will be easy to answer, for sure it is likes you were telling!

However, Atom Rangeley is an older processor released Q3/13 while the Xeon D-15x8 processors are newly released.

It is also the CPU design and the circumstance that not each cpu core is comparable to any other cpu core.
If the netmap-fwd, QAT and perhaps DPDK over AVX/AVX2 registers will be available it could really be that
the Intel Atom C2000 (Rangely) platform will be attractive as on its first release day or much more then this.

I suppose the new Xeon D processors will outperform Atom Rangeley except for crypto-heavy stuff like VPN.

QAT, netmap-fwd and OpenVPN 2.4 in pfSense 2.3 will be able to change many things.

I will configure my pfSense firewall with IPSEC or OpenVPN, but will probably use it infrequently, typically to access my home network in Norway from my Spanish home.

As for now I am still using IPSec VPN (AES-GCM) it is the best supported VPN form the AES-NI CPU instructions.
If you have many side-to-side VPNs it will be really useful to get the best performance as it is able to realize
and with AES-NI it is able to get something around the x4 or x5 of the normal throughput. Again, this can be
turned around if the intel QAT is in usage inside of pfSense and then the cards will mixed up new again.

Also the iOS devices from apple are coming together with IPSec APPs and it is a fine thing if on both
side a pfSense firewall is able to use the AES-NI to speed up the entire IPSec tunnel.

The Supermicro X10SDV-6C+-TLN4F board (Xeon D-1528) with active CPU-cooling looks extremely tempting. ;)

M.2 SSD slot, 2 x 10 GbE ports, many CPU core, HT and TurboBoost, DDR4-2133MHz UDIMM support
PCIe 3.0 x16 what should I say a really nice platform to go with and the ability to add if needed

• Chelsio 2 Port 10 GbE NIC
• Netgate Intel QAT Adapter
• Intel QAT Adapter with 4 GBit/s LAN Ports

Frank, what do you think?  ;D

For home and for SMB usage go with the Intel Atom C2x58 SoC or similar from Negate store or the
pfSense store. If you will needing more horse power and/or throughput the Xeon D-15x8 will be a
really nice option to know too.

[reggie14]

@oletuv:

@oletuv:

Since support for QuickAssist probably will be added to pfSense during 2016, I think a Atom C2758 SKU would be a better option for a dedicated pfSense box. Personally I´m going to order a prebuilt Mini-ITX with A1SRi-2758F motherboard from Supermicro. QuickAssist onboard and a lot cheaper than a Xeon D-15x8 based config too.

Hm.. Common sense tells me that Atom C2758 with onboard QuickAssist acceleration is a better option than the more costly Xeon D-15x8 with no onboard QuickAssist acceleration for a pfSense build. However, Atom Rangeley is an older processor released Q3/13 while the Xeon D-15x8 processors are newly released.

Just be aware that QAT support might not be coming to the C2758.

When asked about QAT support earlier today, gonzopancho wrote:

"When it's done." Maybe 2.4, and then maybe only for 895x and newer.
I'm still not decided if it will go in the community edition.

[MikeV7896]

All I'll say is I hope that isn't the case, both for the devices supported (C2x58 processors should be supported!) and for the lack of presence in the community edition. I have a lot more that I want to say, but I'll refrain for the time being, since there's still lots of time before it sees any sort of daylight in a released fashion.

[Keljian]

So much FUD…

Quickassist is a Very Good Thing TM IF you are pushing >20GB across VPNs or are doing extensive UTM type tasks. Or if you are looking to get line rate over 10-40Gbit ..

This is unnecessary for home use at this time.

It is safe to say that with AES-NI most desktop grade processors can push sufficient packets for up to about 5 Gigabit/s worth of speed  (source: https://calomel.org/aesni_ssl_performance.html - Note these are based on a single core) - For example an i3-6100 should be able to stretch to about 20Gbps 256bit AES-CBC in a pinch.

For a home connection, this is more than sufficient. You need 15-20Mbit for 4k Streaming, so at 100mbps you can handle 4-5 streams happily - there is plenty of hardware out there that can push this, encrypted if you want.

While I appreciate having "all the cool stuff" is nice, it is totally unnecessary for home users of pfSense and holding off purchasing hardware based on whether QAT is supported is silly if your use case is "in the home".

If on the other hand it is "in the office" then you should be buying supported hardware (you are aren't you?) from the pfSense store or Netgate, preferably the former, to support the project, and in turn get support for things like QAT.

If you are NOT buying supported hardware then if you are using it in an office environment you should be paying for support, and realistically using server grade xeon hardware which will have AES-NI and Lotsa CoresTM so speed will become largely irrelevant - AND you should be able to afford a coleto creek card if you need huge bandwidth..

[reggie14]

@Keljian:

So much FUD…

Quickassist is a Very Good Thing TM IF you are pushing >20GB across VPNs or are doing extensive UTM type tasks. Or if you are looking to get line rate over 10-40Gbit ..

This is unnecessary for home use at this time.

It is safe to say that with AES-NI most desktop grade processors can push sufficient packets for up to about 5 Gigabit/s worth of speed  (source: https://calomel.org/aesni_ssl_performance.html - Note these are based on a single core) - For example an i3-6100 should be able to stretch to about 20Gbps 256bit AES-CBC in a pinch.

For a home connection, this is more than sufficient. You need 15-20Mbit for 4k Streaming, so at 100mbps you can handle 4-5 streams happily - there is plenty of hardware out there that can push this, encrypted if you want.

While I appreciate having "all the cool stuff" is nice, it is totally unnecessary for home users of pfSense and holding off purchasing hardware based on whether QAT is supported is silly if your use case is "in the home".

I largely agree with this conclusion.  Except some of your figures are quite a bit off.  As one example, AES-CBC performance isn't what you should be looking at.  AES-GCM is.  And that's assuming you're using AES-GCM for message authentication.  If you're using a legacy suite based on AES-CBC and HMAC-SHA1, then HMAC-SHA1 will become a performance issue.  This is an issue for OpenVPN, which still doesn't have GCM in their stable release.  A home user with a high-speed connection running over OpenVPN on a C2758 might get a benefit from QAT, although there are other performance bottlenecks with OpenVPN that probably become an issue first.

So yes, I certainly agree QAT is unnecessary for all but the most extraordinary home use cases. If you're really in that boat, then it probably makes sense to get something faster than a Rangeley.

[manaox2]

QAT was originally in the road map for 3.0 back in February of 2015, so I can be patient. I hope it does not require a hardware purchase to receive support, as many people are looking forward to this for home usage after being recommended so much on the forums.

I wish I that knew then what I know now, I agree that I'll probably need something more powerful than my c2758 system which is a shame.

Still looking forward to seeing the results of the DevSummit 2016 in June where they are working on the QAT FreeBSD driver port. I would like to squeeze as much as possible running both traffic through OpenVPN while using Suricata at home.

https://twitter.com/gonzopancho/status/715262054832033792

#Intel #QuickAssist driver update at #BSDCan (#FreeBSD dev summit) https://wiki.freebsd.org/DevSummit/201606#Schedule-1 … Likely to be ready for #pfSense 2.4

[Guest]

This is an issue for OpenVPN, which still doesn't have GCM in their stable release.

This will be there in the OpenVPN version 2.4 and this could be going into the pfSense version of
2.3 or 2.4 stable. So the OpenVPN users might be getting the same benefit from that AES-NI
CPU registers likes the IPSec users that get something about 400% speed improvement.

But the Intel QAT will be not only doing crypto work it is also for compression and decompression
and this might be perhaps also a real gain and benefit that could be really interesting, either for
home or professional usage. If it is in action on both sides I pretty sure it will be speed up more
then only the encryption part.

I hope it does not require a hardware purchase to receive support,

Could be perhaps for the first one year that pfSense shop or Netgate customers are getting some more
benefit from their units, but all after all I would surely also prefer and love it to see that it finds its way into
the community version of pfSense.

as many people are looking forward to this for home usage after being recommended so much on the forums.

Me too.

I wish I that knew then what I know now, I agree that I'll probably need something more powerful than my c2758 system which is a shame.

For a home usage that board will be really well as I see it right and I also think about it, that we all could
not imagine the real or full potential that can be unleashed. And mostly if many things comes together, the
impact is more then only one long awaited thing. With netmap-fwd, QAT, OpenVPN 2.4, perhaps later DPDK
over AVX/AVX2 registers of that SoC and multi core usage in PPPoE at the WAN port it must be really rocking.

while using Suricata at home.

In former times Intel was announcing that IDS/IPS software will also benefit from that QAT,
but this statement was then taken back from the public and so only the encryption and
compression part are only profiting from that QAT only.

[zanthos]

Anyone have seen this:
Lanner AV-ICE01 - VPN Acceleration Card with Intel Cave Creek DH8910CC
Lanner AV-ICE02 - VPN Acceleration Card with Intel Coleto Creek 8925/8950
Lanner AV-ICE04 - The Gen. 3 PCIe x8 Network Processing/Acceleration Card with Intel Coleto Creek 8955 PCH

So far I've been offered:
AV-ICE01 ~250€
AV-ICE02 ~440€

I think the AV-ICE01 would be a real deal breaker. Up to 10Gbps hardware offload assistance should be enough for most of us…
Therefore I hope the upcoming implementation of Intel QAT in FreeBSD will support Intel Communication Chipset 8910 Series.

[Guest]

@Zanthos

I first only was finding the ADI and Netgate boards at a higher price point.
Cryptographic Accelerator CPIC Adapter 8955 with QuickAssist
CPIC: Intel 8920/8955

But now I found also a plugin module that will be fitting right, but only for some
appliances from the same vendor! And yes they are not really low in price too. :-[
No price labeling was there to get a good overview, but nice and interesting looking.
[url=http://www.axiomtek.de/Default.aspx?MenuId=Products&FunctionId=ProductView&ItemId=15145&upcat=233]Axiomtek NA361R
Axiomtek NA570
Axiomtek NA552
Axiomtek VPN Module

[shelbystripes]

@Blade:

BlueKobold is correct although info is not readily apparent. The Xeon D 15x8 series does support AES-NI and QuickAssist Technology.

Link 1

http://www.servethehome.com/intel-xeon-d-15x8-networking-accelerated-skus/

Link 2

http://www.cpu-world.com/CPUs/Xeon_D/Intel-Xeon%20D-1518.html

The above link shows AES instructions however it's identical to AES-NI.

I want to point out that the STH page has a slide showing "Intel QuickAssist Technology Crypto Accelerator (Coleto Creek) Support". It doesn't say QuickAssist is integrated into any Xeon D models. "Coleto Creek" is Intel's code name for its 8950-series PCIe QuickAssist accelerator, which is a standalone chip typically sold on a PCIe add-in card. Intel also usually doesn't use the word "support" to describe an integrated feature, they use it to mean compatibility with external hardware or software.

It sounds like Intel is just saying the Xeon-D works with an 8950 card if you want QuickAssist. That implies that QuickAssist is NOT in the CPU/SoC itself. And the cpuworld link doesn't show QuickAssist built-in either… I wish it did, but nothing I can find says it's actually there.

[jwt]

@shelbystripes:

@Blade:

BlueKobold is correct although info is not readily apparent. The Xeon D 15x8 series does support AES-NI and QuickAssist Technology.

Link 1

http://www.servethehome.com/intel-xeon-d-15x8-networking-accelerated-skus/

Link 2

http://www.cpu-world.com/CPUs/Xeon_D/Intel-Xeon%20D-1518.html

The above link shows AES instructions however it's identical to AES-NI.

I want to point out that the STH page has a slide showing "Intel QuickAssist Technology Crypto Accelerator (Coleto Creek) Support". It doesn't say QuickAssist is integrated into any Xeon D models.

True, it doesn't say that….. I can't say more.

@shelbystripes:

"Coleto Creek" is Intel's code name for its 8950-series PCIe QuickAssist accelerator,

Technically, Coleto Creek also includes some DH892x models. 
http://ark.intel.com/products/codename/60172/Coleto-Creek#@Embedded

Confusingly, the previous generation (Cave Creek) includes the DH8920
http://ark.intel.com/products/codename/44946/Cave-Creek#@Embedded

@shelbystripes:

which is a standalone chip typically sold on a PCIe add-in card.

Like this: http://store.netgate.com/ADI/QuickAssist8955.aspx

@shelbystripes:

Intel also usually doesn't use the word "support" to describe an integrated feature, they use it to mean compatibility with external hardware or software.

It sounds like Intel is just saying the Xeon-D works with an 8950 card if you want QuickAssist. That implies that QuickAssist is NOT in the CPU/SoC itself. And the cpuworld link doesn't show QuickAssist built-in either… I wish it did, but nothing I can find says it's actually there.

yeah, well…note that every QuickAssist part is a Platform Controller Hub, and that when you see these in a PCIe card form factor, they're being used in "end-point" mode.  Xeon-D (Broadwell-DE) has an integrated PCH.

See if you can't piece it together from there.

[jwt]

@zanthos:

Anyone have seen this:
Lanner AV-ICE01 - VPN Acceleration Card with Intel Cave Creek DH8910CC
Lanner AV-ICE02 - VPN Acceleration Card with Intel Coleto Creek 8925/8950
Lanner AV-ICE04 - The Gen. 3 PCIe x8 Network Processing/Acceleration Card with Intel Coleto Creek 8955 PCH

So far I've been offered:
AV-ICE01 ~250€
AV-ICE02 ~440€

I think the AV-ICE01 would be a real deal breaker. Up to 10Gbps hardware offload assistance should be enough for most of us…
Therefore I hope the upcoming implementation of Intel QAT in FreeBSD will support Intel Communication Chipset 8910 Series.

It won't.

[jwt]

@BlueKobold:

There are some Intel based SoCs that supports Intel QuickAssist and also some Intel chips (coleto creek)
that can be assembled or soldered on add on PCIe cards or modules that are supporting Intel QuickAssist.

This SoCs and the Coleto Creek chips are used by ADI Engineering who is assembling the whole range of
hardware for the Netgate store and pfSense store. You might be able to buy either you want both parts,
PCIe cards and also appliances. Actual now, or as today this Intel QuickAssist code isn´t flown inside of
the pfSense code. I am pretty sure that we will see this working between the version 2.3 final and 3.0
final. This is not based on proofed informations that you can count on, but more a guess personally from
my self about this. And I am glad about that the developers were waiting with this function!

SG-2220, 2440, 4860, 8860 C2758 1U and XG-2758 appliances are using the Intel Atom C2x58 (Rangeley)
SoCs, but Intel is upgrading actual the whole Intel Xeon D-1500 SoC series and some SKUs will be extra
network accelerated SoCs and so it might be that the pfSense store is also changing their Intel based
Xeon D-15xx platforms against the newer ones that comes network accelerated. So we will some more
time waiting, but after this time we get perhaps two series of appliances that is using then Intel
QuickAssist and not only one.

This might be causing why this will be not inserted in pfSense actual yet. The newer Intel Xeon D-15x8
SoCs are coming with;

• AES-NI
• Intel QuickAssist
• DPDK support (enabled software)

The actual Intel Atom C2x58 (Rangely) SoC that is used is supporting;

• AES-NI
• Intel QuickAssist

IPSec is actually pushed by using the AES-NI instruction set to speed up the entire throughput
to the x4 or x5 by using the AES-GCM algorithm.

OpenVPN might be pushed over the Intel QuickAssist in the near future or it gets also the AES-GCM
algorithm inserted that it might be also benefiting from the AES-NI instruction set. Who knows?

As an upgrade for systems without Intel QuickAssist:
ADI Engineering PCIe Intel QuickAssist accelerator only
Netgate PCIe Intel QuickAssist accelerator w/ four Intel GB LAN Ports

So much wrong…

SG-2220, 2440, 4860, 8860 C2758 1U and XG-2758 appliances are using the Intel Atom C2x58 (Rangeley)
SoCs

SG-2220 uses a C2338, which doesn't have QAT on-die.  http://ark.intel.com/products/77976/Intel-Atom-Processor-C2338-1M-Cache-1_70-GHz

Intel Atom C2xxx supports DPDK (you implied it doesn't).  We are doing a bit over 12Mpps routed on this: http://store.netgate.com/ADI/RCC-2758-1U.aspx  (note, not with pfSense)

IPSec is actually pushed by using the AES-NI instruction set to speed up the entire throughput
to the x4 or x5 by using the AES-GCM algorithm.

WAT?

AES-GCM is faster than AES-CBC + HMAC-SHA1 for two reasons:

• AES-GCM is a bit faster than AES-CBC

• AES-GCM is an AEAD algorithm.  It generates the HMAC as a side-effect of running the algorithm.

The second is most of the reason you see AES-GCM as 'faster'.  Only one pass over the data needs to be made, and that pass
is accelerated via AES-NI instructions.  QuickAssist can accelerate AES-GCM, AES-CBC and HMAC-SHA*, so in theory, turning on QAT would make for a faster IPsec setup, even with AES-CBC + HMAC-SHA1.

In fact, we've proved it.  We can do 17Gbps on two tunnels between a pair of Xeon E3-1275v3 boxes with the 8955-based card we sell, and a 82599 10G Ethernet, using strongswan.  For all I know, it will get quite close to the Intel-claimed performance figure of 40Gbps with a few more tunnels.  I've just never bothered to purchase the 40G cards (or 4x10G xl710 cards) to find out.

Adrian Chadd (who works on the FreeBSD wireless drivers) has my Chelsio 40G cards, and won't give them back.

Now for the bad news.  As we've already shown, FreeBSD on those same boxes, running NULL encryption in IPsec can only do 4Gbps throughput.  No, I did not drop a zero.

Until this is fixed, investing the effort in QAT is moot for IPsec (and anything else that uses the Open Crypto Framework in FreeBSD).

OpenVPN might be pushed over the Intel QuickAssist in the near future or it gets also the AES-GCM
algorithm inserted that it might be also benefiting from the AES-NI instruction set. Who knows?

OpenVPN runs over tun/tap.  Until this is changed, no amount of hardware acceleration will help.  Yes, we've already tried it.

Yes, we have a plan, but it is unlikely to be in pfSense, because of individuals like this:  https://forum.pfsense.org/index.php?topic=112074.0  Not singling that individual out, it's just the last example I ran across.

"I built my own and saved a few bucks!" doesn't induce me to invest the huge sums of money involved in fixing all of this.  There isn't enough glory to make up the spend.

Free Software isn't free to make.  Someone gets paid to design, write, test, debug, document and support it.

[jwt]

@oletuv:

@cmb:

@jbhowlesr:

My assumption about AES-NI and QuickAssist is that they are widely unnecessary if you have medium to high power CPU

Not true at all. Not even close. Check the performance stats.
http://store.netgate.com/ADI/QuickAssist8955.aspx

I don´t get it. The 8955 adapter costs $899 while the Atom C2000 processors have QuickAssist built-in for a fraction of the cost.

The QAT unit in some (not all) C2000 SoCs is a cut-down (about 1/2 the execution units) version of  the older "Cave Creek" core.  This is also why the Rangeley variants of C2000 have 4 "i350" Ethernet interfaces.  See elsewhere in this thread for a short discussion on "PCH", and note that Coleto Creek does NOT have any Ethernet devices on-die.

The Rangeley QAT is good for maybe 8Gbps IPsec.  According to Intel's marketing, the DH8955 is good for around 40Gbps IPsec.

[Guest]

So an Intel Xeon D-15x8 platform together with a QAT adapter will be then the best option to
get all three things in one pfSense box such as;

• AES-NI
• Intel QAT
• Intel DPDK

[jwt]

Any Xeon (or potentially a fast i7) with a QAT card.

But remember, Rangely supports AES-NI, QAT and DPDK.

[heper]

"I built my own and saved a few bucks!" doesn't induce me to invest the huge sums of money involved in fixing all of this.  There isn't enough glory to make up the spend.

@jwt
well that might be true in the US. In europe the price difference between a pfsense branded unit or a similar supermicro unit is significant.
for example:
sg-4860 @pf store = $699
sg-4860 @EU-webshop = $1066
so around $367 difference in price between US & Europe.

Now the thing is, you can get a supermicro c2758 for around $1100 in europe / a c2558 for around$850. (in 1U case)

To conclude:
there is a $200 price difference .I work in the education/non-profit business, they are scraping by as it is …. $200 is a big deal for them.

As a member of this community i would love to pay the extra $$ because i think it is more then worth it.
Unfortunately at work, it'll be supermicro until the price difference is <= $100