Hardware support for Intel QuickAssist?

manaox2

QAT was originally in the road map for 3.0 back in February of 2015, so I can be patient. I hope it does not require a hardware purchase to receive support, as many people are looking forward to this for home usage after being recommended so much on the forums.

I wish I that knew then what I know now, I agree that I'll probably need something more powerful than my c2758 system which is a shame.

Still looking forward to seeing the results of the DevSummit 2016 in June where they are working on the QAT FreeBSD driver port. I would like to squeeze as much as possible running both traffic through OpenVPN while using Suricata at home.

https://twitter.com/gonzopancho/status/715262054832033792

#Intel #QuickAssist driver update at #BSDCan (#FreeBSD dev summit) https://wiki.freebsd.org/DevSummit/201606#Schedule-1 … Likely to be ready for #pfSense 2.4

Guest

This is an issue for OpenVPN, which still doesn't have GCM in their stable release.

This will be there in the OpenVPN version 2.4 and this could be going into the pfSense version of
2.3 or 2.4 stable. So the OpenVPN users might be getting the same benefit from that AES-NI
CPU registers likes the IPSec users that get something about 400% speed improvement.

But the Intel QAT will be not only doing crypto work it is also for compression and decompression
and this might be perhaps also a real gain and benefit that could be really interesting, either for
home or professional usage. If it is in action on both sides I pretty sure it will be speed up more
then only the encryption part.

I hope it does not require a hardware purchase to receive support,

Could be perhaps for the first one year that pfSense shop or Netgate customers are getting some more
benefit from their units, but all after all I would surely also prefer and love it to see that it finds its way into
the community version of pfSense.

as many people are looking forward to this for home usage after being recommended so much on the forums.

Me too.

I wish I that knew then what I know now, I agree that I'll probably need something more powerful than my c2758 system which is a shame.

For a home usage that board will be really well as I see it right and I also think about it, that we all could
not imagine the real or full potential that can be unleashed. And mostly if many things comes together, the
impact is more then only one long awaited thing. With netmap-fwd, QAT, OpenVPN 2.4, perhaps later DPDK
over AVX/AVX2 registers of that SoC and multi core usage in PPPoE at the WAN port it must be really rocking.

while using Suricata at home.

In former times Intel was announcing that IDS/IPS software will also benefit from that QAT,
but this statement was then taken back from the public and so only the encryption and
compression part are only profiting from that QAT only.

zanthos

Anyone have seen this:
Lanner AV-ICE01 - VPN Acceleration Card with Intel Cave Creek DH8910CC
Lanner AV-ICE02 - VPN Acceleration Card with Intel Coleto Creek 8925/8950
Lanner AV-ICE04 - The Gen. 3 PCIe x8 Network Processing/Acceleration Card with Intel Coleto Creek 8955 PCH

So far I've been offered:
AV-ICE01 ~250€
AV-ICE02 ~440€

I think the AV-ICE01 would be a real deal breaker. Up to 10Gbps hardware offload assistance should be enough for most of us…
Therefore I hope the upcoming implementation of Intel QAT in FreeBSD will support Intel Communication Chipset 8910 Series.

Guest

@Zanthos

I first only was finding the ADI and Netgate boards at a higher price point.
Cryptographic Accelerator CPIC Adapter 8955 with QuickAssist
CPIC: Intel 8920/8955

But now I found also a plugin module that will be fitting right, but only for some
appliances from the same vendor! And yes they are not really low in price too. :-[
No price labeling was there to get a good overview, but nice and interesting looking.
[url=http://www.axiomtek.de/Default.aspx?MenuId=Products&FunctionId=ProductView&ItemId=15145&upcat=233]Axiomtek NA361R
Axiomtek NA570
Axiomtek NA552
Axiomtek VPN Module

shelbystripes

@Blade:

BlueKobold is correct although info is not readily apparent. The Xeon D 15x8 series does support AES-NI and QuickAssist Technology.

Link 1

http://www.servethehome.com/intel-xeon-d-15x8-networking-accelerated-skus/

Link 2

http://www.cpu-world.com/CPUs/Xeon_D/Intel-Xeon%20D-1518.html

The above link shows AES instructions however it's identical to AES-NI.

I want to point out that the STH page has a slide showing "Intel QuickAssist Technology Crypto Accelerator (Coleto Creek) Support". It doesn't say QuickAssist is integrated into any Xeon D models. "Coleto Creek" is Intel's code name for its 8950-series PCIe QuickAssist accelerator, which is a standalone chip typically sold on a PCIe add-in card. Intel also usually doesn't use the word "support" to describe an integrated feature, they use it to mean compatibility with external hardware or software.

It sounds like Intel is just saying the Xeon-D works with an 8950 card if you want QuickAssist. That implies that QuickAssist is NOT in the CPU/SoC itself. And the cpuworld link doesn't show QuickAssist built-in either… I wish it did, but nothing I can find says it's actually there.

jwt

@shelbystripes:

@Blade:

BlueKobold is correct although info is not readily apparent. The Xeon D 15x8 series does support AES-NI and QuickAssist Technology.

Link 1

http://www.servethehome.com/intel-xeon-d-15x8-networking-accelerated-skus/

Link 2

http://www.cpu-world.com/CPUs/Xeon_D/Intel-Xeon%20D-1518.html

The above link shows AES instructions however it's identical to AES-NI.

I want to point out that the STH page has a slide showing "Intel QuickAssist Technology Crypto Accelerator (Coleto Creek) Support". It doesn't say QuickAssist is integrated into any Xeon D models.

True, it doesn't say that….. I can't say more.

@shelbystripes:

"Coleto Creek" is Intel's code name for its 8950-series PCIe QuickAssist accelerator,

Technically, Coleto Creek also includes some DH892x models.
http://ark.intel.com/products/codename/60172/Coleto-Creek#@Embedded

Confusingly, the previous generation (Cave Creek) includes the DH8920
http://ark.intel.com/products/codename/44946/Cave-Creek#@Embedded

@shelbystripes:

which is a standalone chip typically sold on a PCIe add-in card.

Like this: http://store.netgate.com/ADI/QuickAssist8955.aspx

@shelbystripes:

Intel also usually doesn't use the word "support" to describe an integrated feature, they use it to mean compatibility with external hardware or software.

It sounds like Intel is just saying the Xeon-D works with an 8950 card if you want QuickAssist. That implies that QuickAssist is NOT in the CPU/SoC itself. And the cpuworld link doesn't show QuickAssist built-in either… I wish it did, but nothing I can find says it's actually there.

yeah, well…note that every QuickAssist part is a Platform Controller Hub, and that when you see these in a PCIe card form factor, they're being used in "end-point" mode. Xeon-D (Broadwell-DE) has an integrated PCH.

See if you can't piece it together from there.

jwt

@zanthos:

Anyone have seen this:
Lanner AV-ICE01 - VPN Acceleration Card with Intel Cave Creek DH8910CC
Lanner AV-ICE02 - VPN Acceleration Card with Intel Coleto Creek 8925/8950
Lanner AV-ICE04 - The Gen. 3 PCIe x8 Network Processing/Acceleration Card with Intel Coleto Creek 8955 PCH

So far I've been offered:
AV-ICE01 ~250€
AV-ICE02 ~440€

I think the AV-ICE01 would be a real deal breaker. Up to 10Gbps hardware offload assistance should be enough for most of us…
Therefore I hope the upcoming implementation of Intel QAT in FreeBSD will support Intel Communication Chipset 8910 Series.

It won't.

jwt

@BlueKobold:

There are some Intel based SoCs that supports Intel QuickAssist and also some Intel chips (coleto creek)
that can be assembled or soldered on add on PCIe cards or modules that are supporting Intel QuickAssist.

This SoCs and the Coleto Creek chips are used by ADI Engineering who is assembling the whole range of
hardware for the Netgate store and pfSense store. You might be able to buy either you want both parts,
PCIe cards and also appliances. Actual now, or as today this Intel QuickAssist code isn´t flown inside of
the pfSense code. I am pretty sure that we will see this working between the version 2.3 final and 3.0
final. This is not based on proofed informations that you can count on, but more a guess personally from
my self about this. And I am glad about that the developers were waiting with this function!

SG-2220, 2440, 4860, 8860 C2758 1U and XG-2758 appliances are using the Intel Atom C2x58 (Rangeley)
SoCs, but Intel is upgrading actual the whole Intel Xeon D-1500 SoC series and some SKUs will be extra
network accelerated SoCs and so it might be that the pfSense store is also changing their Intel based
Xeon D-15xx platforms against the newer ones that comes network accelerated. So we will some more
time waiting, but after this time we get perhaps two series of appliances that is using then Intel
QuickAssist and not only one.

This might be causing why this will be not inserted in pfSense actual yet. The newer Intel Xeon D-15x8
SoCs are coming with;

AES-NI

Intel QuickAssist

DPDK support (enabled software)

The actual Intel Atom C2x58 (Rangely) SoC that is used is supporting;

AES-NI

Intel QuickAssist

IPSec is actually pushed by using the AES-NI instruction set to speed up the entire throughput
to the x4 or x5 by using the AES-GCM algorithm.

OpenVPN might be pushed over the Intel QuickAssist in the near future or it gets also the AES-GCM
algorithm inserted that it might be also benefiting from the AES-NI instruction set. Who knows?

As an upgrade for systems without Intel QuickAssist:
ADI Engineering PCIe Intel QuickAssist accelerator only
Netgate PCIe Intel QuickAssist accelerator w/ four Intel GB LAN Ports

So much wrong…

SG-2220, 2440, 4860, 8860 C2758 1U and XG-2758 appliances are using the Intel Atom C2x58 (Rangeley)
SoCs

SG-2220 uses a C2338, which doesn't have QAT on-die. http://ark.intel.com/products/77976/Intel-Atom-Processor-C2338-1M-Cache-1_70-GHz

Intel Atom C2xxx supports DPDK (you implied it doesn't). We are doing a bit over 12Mpps routed on this: http://store.netgate.com/ADI/RCC-2758-1U.aspx (note, not with pfSense)

IPSec is actually pushed by using the AES-NI instruction set to speed up the entire throughput
to the x4 or x5 by using the AES-GCM algorithm.

WAT?

AES-GCM is faster than AES-CBC + HMAC-SHA1 for two reasons:

AES-GCM is a bit faster than AES-CBC
AES-GCM is an AEAD algorithm. It generates the HMAC as a side-effect of running the algorithm.

The second is most of the reason you see AES-GCM as 'faster'. Only one pass over the data needs to be made, and that pass
is accelerated via AES-NI instructions. QuickAssist can accelerate AES-GCM, AES-CBC and HMAC-SHA*, so in theory, turning on QAT would make for a faster IPsec setup, even with AES-CBC + HMAC-SHA1.

In fact, we've proved it. We can do 17Gbps on two tunnels between a pair of Xeon E3-1275v3 boxes with the 8955-based card we sell, and a 82599 10G Ethernet, using strongswan. For all I know, it will get quite close to the Intel-claimed performance figure of 40Gbps with a few more tunnels. I've just never bothered to purchase the 40G cards (or 4x10G xl710 cards) to find out.

Adrian Chadd (who works on the FreeBSD wireless drivers) has my Chelsio 40G cards, and won't give them back.

Now for the bad news. As we've already shown, FreeBSD on those same boxes, running NULL encryption in IPsec can only do 4Gbps throughput. No, I did not drop a zero.

Until this is fixed, investing the effort in QAT is moot for IPsec (and anything else that uses the Open Crypto Framework in FreeBSD).

OpenVPN might be pushed over the Intel QuickAssist in the near future or it gets also the AES-GCM
algorithm inserted that it might be also benefiting from the AES-NI instruction set. Who knows?

OpenVPN runs over tun/tap. Until this is changed, no amount of hardware acceleration will help. Yes, we've already tried it.

Yes, we have a plan, but it is unlikely to be in pfSense, because of individuals like this: https://forum.pfsense.org/index.php?topic=112074.0 Not singling that individual out, it's just the last example I ran across.

"I built my own and saved a few bucks!" doesn't induce me to invest the huge sums of money involved in fixing all of this. There isn't enough glory to make up the spend.

Free Software isn't free to make. Someone gets paid to design, write, test, debug, document and support it.

jwt

@oletuv:

@cmb:

@jbhowlesr:

My assumption about AES-NI and QuickAssist is that they are widely unnecessary if you have medium to high power CPU

Not true at all. Not even close. Check the performance stats.
http://store.netgate.com/ADI/QuickAssist8955.aspx

I don´t get it. The 8955 adapter costs $899 while the Atom C2000 processors have QuickAssist built-in for a fraction of the cost.

The QAT unit in some (not all) C2000 SoCs is a cut-down (about 1/2 the execution units) version of the older "Cave Creek" core. This is also why the Rangeley variants of C2000 have 4 "i350" Ethernet interfaces. See elsewhere in this thread for a short discussion on "PCH", and note that Coleto Creek does NOT have any Ethernet devices on-die.

The Rangeley QAT is good for maybe 8Gbps IPsec. According to Intel's marketing, the DH8955 is good for around 40Gbps IPsec.

Guest

So an Intel Xeon D-15x8 platform together with a QAT adapter will be then the best option to
get all three things in one pfSense box such as;

AES-NI
Intel QAT
Intel DPDK

jwt

Any Xeon (or potentially a fast i7) with a QAT card.

But remember, Rangely supports AES-NI, QAT and DPDK.

heper

"I built my own and saved a few bucks!" doesn't induce me to invest the huge sums of money involved in fixing all of this. There isn't enough glory to make up the spend.

@jwt
well that might be true in the US. In europe the price difference between a pfsense branded unit or a similar supermicro unit is significant.
for example:
sg-4860 @pf store = $699
sg-4860 @EU-webshop = $1066
so around $367 difference in price between US & Europe.

Now the thing is, you can get a supermicro c2758 for around $1100 in europe / a c2558 for around$850. (in 1U case)

To conclude:
there is a $200 price difference .I work in the education/non-profit business, they are scraping by as it is …. $200 is a big deal for them.

As a member of this community i would love to pay the extra $$ because i think it is more then worth it.
Unfortunately at work, it'll be supermicro until the price difference is <= $100

razzfazz

So between this:

@jwt:

The QAT unit in some (not all) C2000 SoCs is a cut-down (about 1/2 the execution units) version of the older "Cave Creek" core.

and your earlier statement that Cave Creek will not be supported by the QAT implementation in FreeBSD, does that mean that FreeBSD won't support the built-in QAT in Rangeley, either?

Hakker

@heper:

"I built my own and saved a few bucks!" doesn't induce me to invest the huge sums of money involved in fixing all of this. There isn't enough glory to make up the spend.

@jwt
well that might be true in the US. In europe the price difference between a pfsense branded unit or a similar supermicro unit is significant.
for example:
sg-4860 @pf store = $699
sg-4860 @EU-webshop = $1066
so around $367 difference in price between US & Europe.

Now the thing is, you can get a supermicro c2758 for around $1100 in europe / a c2558 for around$850. (in 1U case)

To conclude:
there is a $200 price difference .I work in the education/non-profit business, they are scraping by as it is …. $200 is a big deal for them.

As a member of this community i would love to pay the extra $$ because i think it is more then worth it.
Unfortunately at work, it'll be supermicro until the price difference is <= $100

Even worse here I guess.

when I buy a SG4860 from Germany I pay €980,56 +shipping at around €20 (around $1110)
https://shop.voleatech.de/en/shop/sg-4860/ (so Yes a pfSense partner)

When I build one myself:
A Supermicro A1SRi-2558F (€290), Supermicro SC-101i case (€70), 8 GB DDR3L ECC memory (€50), 120 GB Adata SP550 SSD (€45) and PicoPSU of 80 watts with a brick (€75) I pay €530,00 (around $600)

That's almost half the price. Dont get me wrong here jwt if it was affordable I would get it from the pfSense shop/partner but it clearly isn't in my case and it's not that Supermicro is an obscure manufacturer.