Pfsense hardware for home

Stewart

@Stewart

I guess I'm only adding to the confusion. I would expect the encryption to work better with aes-ni loaded, but it definitely doesn't appear to.

Yes, maybe add to confusion but you seem to confirm it again.

The way I understand it/picture it in my head, your result could be expected.
When loading the module which, for what I understand means crypto in kernel, then it boils down to what the CPU is capable of.

If you are willing, you could do as described in Reply: #47 and post the four results.

@Pippin

I can run those tests on Monday but it's my understanding that setting the aes-ni in the GUI is really just loading/unloading aesni.ko.

switchman

Here is the result of my system which does not support aes-ni.

Intel(R) Celeron(R) CPU 1037U @ 1.80GHz
2 CPUs: 1 package(s) x 2 core(s)

env OPENSSL_ia32cap=0 openssl speed -elapsed -evp aes-256-cbc

You have chosen to measure elapsed time instead of user CPU time.
Doing aes-256-cbc for 3s on 16 size blocks: 8821510 aes-256-cbc's in 3.00s
Doing aes-256-cbc for 3s on 64 size blocks: 2379023 aes-256-cbc's in 3.00s
Doing aes-256-cbc for 3s on 256 size blocks: 606261 aes-256-cbc's in 3.00s
Doing aes-256-cbc for 3s on 1024 size blocks: 334626 aes-256-cbc's in 3.00s
Doing aes-256-cbc for 3s on 8192 size blocks: 42234 aes-256-cbc's in 3.01s
OpenSSL 1.0.1s-freebsd 1 Mar 2016
built on: date not available
options:bn(64,64) rc4(8x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: clang
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes
aes-256-cbc 47048.05k 50752.49k 51734.27k 114219.01k 115027.43k

openssl speed -elapsed -evp aes-256-cbc

You have chosen to measure elapsed time instead of user CPU time.
Doing aes-256-cbc for 3s on 16 size blocks: 22248563 aes-256-cbc's in 3.01s
Doing aes-256-cbc for 3s on 64 size blocks: 5986655 aes-256-cbc's in 3.00s
Doing aes-256-cbc for 3s on 256 size blocks: 1515738 aes-256-cbc's in 3.00s
Doing aes-256-cbc for 3s on 1024 size blocks: 382211 aes-256-cbc's in 3.00s
Doing aes-256-cbc for 3s on 8192 size blocks: 47915 aes-256-cbc's in 3.00s
OpenSSL 1.0.1s-freebsd 1 Mar 2016
built on: date not available
options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: clang
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes
aes-256-cbc 118350.80k 127715.31k 129342.98k 130461.35k 130839.89k

Pippin

Added:
CPU spend on userland
CPU spend on kernel/system
aes-256-cbc-hmac-sha1
8 threads -multi 8

Command for top in second SSH window:


top -s 1 -aSCHIP

No module loaded:
1. Run without hardware crypto support


env OPENSSL_ia32cap=0 openssl speed -elapsed -evp aes-256-cbc -multi 8
evp              97659.74k   117140.11k   123383.16k   307494.50k   339199.79k

CPU user 99-100% on all cores
CPU kernel 0-0,3% on all cores

No module loaded:
2. Run with hardware crypto support


openssl speed -elapsed -evp aes-256-cbc -multi 8
evp             618920.09k   882027.93k   1003271.00k  1041262.20k  1150214.31k

CPU user 98-100% on all cores
CPU kernel 0-0,3% on all cores

^^^ Compare 1 and 2. Improvement using hardware crypto support

No module loaded:
3. Run with hardware crypto support and hmac-sha1


openssl speed -elapsed -evp aes-256-cbc-hmac-sha1 -multi 8
evp             214362.25k   408669.56k   440537.10k   468988.33k   505701.81k

CPU user 98-100% on all cores
CPU kernel 0-0,3% on all cores

^^^ Compare 2 and 3. Decrease when including SHA1 HMAC.
Compare 1 and 3. Improvement, that would be more if I could test 1 with
aes-256-cbc-hmac-sha1, but it doesn`t let me.


kldload aesni.ko

Module loaded.
4. Run with module loaded


openssl speed -elapsed -evp aes-256-cbc -multi 8
evp              15317.98k    62119.05k   211655.98k   516341.52k   917814.69k

CPU user 1-10% on all cores
CPU kernel 89-99% on all cores

Module loaded.
5. Run with module loaded and hmac-sha1


openssl speed -elapsed -evp aes-256-cbc-hmac-sha1 -multi 8
evp             244592.48k   407765.54k   444189.26k   458533.81k   501885.04k

CPU user 98-100% on all cores
CPU kernel 0-0,3% on all cores

Compare 2 and 4. Decrease when using module.
Compare 3 and 5. About the same.
Compare 4 and 5. Decrease when including SHA1 HMAC.

See the difference between CPU in 4 and 5.
Why?

Stewart

@Stewart:

@Pippin:

@Stewart

I guess I'm only adding to the confusion. I would expect the encryption to work better with aes-ni loaded, but it definitely doesn't appear to.

Yes, maybe add to confusion but you seem to confirm it again.

The way I understand it/picture it in my head, your result could be expected.
When loading the module which, for what I understand means crypto in kernel, then it boils down to what the CPU is capable of.

If you are willing, you could do as described in Reply: #47 and post the four results.

@Pippin

I can run those tests on Monday but it's my understanding that setting the aes-ni in the GUI is really just loading/unloading aesni.ko.


kldload aesni.ko
openssl speed -elapsed -evp aes-256-cbc -multi 4
evp               5616.59k    21923.05k    78318.57k   221039.27k   460087.30k

kldunload aesni.ko
openssl speed -elapsed -evp aes-256-cbc -multi 4
evp             383593.98k   500903.62k   577343.57k   599402.27k   597577.82k

Those are reproducible.

Pippin

Yeah, reproducible here too.
The difference I pretty noticeable.

But difficult thing to dissect. :)

Stewart

However, as confusing as it is, I think we can reliably state that AES-128 (at over 567MB/s) and aes-256 (Over 442MB/s) will both perform well (by the benchmarks that I ran) regardless if the aes-ni box is checked in the GUI. I don't think any of us are using the APU2 boxes in environments where we are pushing more than 400MB/s encrypted traffic. And if they can do greater than 400MB/s encrypted then they can certainly do faster than that on pure NAT. While my iperf numbers are consistently low no matter what I try (gave up on it), speed tests show I have no problem cracking 200MB/s with Squid+SquidGuard+AV+Snort running and that shows a peak of 33% utilization in the pfSense GUI.

Can they to GB Line speed with just NAT? Probably. I can't really test it. We can safely say, however, that the APU2D4 can certainly do at least 500MB/s UTM which places it within arms reach of the SonicWall TZ400 for 2/3 less and no annual fees. Aside from expandability and convenience factors it's better than all Cisco ISA models and many Cisco ASA models. (Although I fear saying that since I'm sure some CCNA will find this thread and jump in to talk about how nothing can touch a Cisco, ever!)

Do I wish it could do full GB no matter what we throw at it? Absolutely! But since the whole kit is $200 I'm not sure there's room to complain. Does this need to be fixed/clarified? Certainly! Something needs to be addressed here but as long as we know the limitations, it's just something to work around for now. Any other thoughts?

Pippin

@Stewart:

However, as confusing as it is, I think we can reliably state that AES-128 (at over 567MB/s) and aes-256 (Over 442MB/s) will both perform well (by the benchmarks that I ran) regardless if the aes-ni box is checked in the GUI.

It depends I would think.
On a system under load where the load is taking CPU cycles it could be better to use the hardware crypto support (not loading module).

400MB/s
200MB/s

Think you mean Mbit/s…

Any other thoughts?

Saw a ticket last week about they will change something about the crypto module, maybe they try to address something, I don`t know.

Stewart

@Pippin:

@Stewart:

However, as confusing as it is, I think we can reliably state that AES-128 (at over 567MB/s) and aes-256 (Over 442MB/s) will both perform well (by the benchmarks that I ran) regardless if the aes-ni box is checked in the GUI.

It depends I would think.
On a system under load where the load is taking CPU cycles it could be better to use the hardware crypto support (not loading module).

400MB/s
200MB/s

Think you mean Mbit/s…

Any other thoughts?

Saw a ticket last week about they will change something about the crypto module, maybe they try to address something, I don`t know.

1. Yes. My tests show anywhere from 99% loss to 4% gain when enabling the module in the GUI. It's definitely better to just leave it off. It is perplexing, though, how the APU2C lags behind the APU1D in some of the tests (particularly in small sizes). As for load, it's just easier to compare raw numbers against other products since that's what they are using. When you see a product listed at 300Mb/s VPN throughput you know it's max theoretical with no other load. I would like to know if the real world usage with the aes-ni disabled in the GUI is more akin to the command "openssl speed -elapsed aes-256-cbc -multi " or "openssl speed -elapsed -evp aes-256-cbc -multi".

2. You are correct in that it should have been Mb/s. MB/s would be rather nice! :)

Is there any way to test IPSEC performance? I'm under the impression that these tests aren't the same. What I really care about are IPSEC speeds since those are the site-to-site tunnels that would most likely be symmetrical fiber. Anything else would be limited by the ISP upload anyway.

chercheur

@user09:

Intel Pentium Processor N3700

X11SBA-LN4F Supermicro

8 GB S0-DDR3

Kingston SV300S37A/60G SSDNow V300 interne SSD-Festplatte 60GB

Hello user09,
Did you make your choice ?

user09

thank you all for your answers.

@chercheur:

@user09:

Intel Pentium Processor N3700

X11SBA-LN4F Supermicro

8 GB S0-DDR3

Kingston SV300S37A/60G SSDNow V300 interne SSD-Festplatte 60GB

Hello user09,
Did you make your choice ?

I have tried the Supermicro Board, but I sent it back, because the idle power usage was 14 - 15 W. In my opinion is that too high for my purposes.

So, I take a look at the expensive SG-2220. For clarity I started a new topic at https://forum.pfsense.org/index.php?topic=117873.0

A Former User

Hi

@user09:

I have tried the Supermicro Board, but I sent it back, because the idle power usage was 14 - 15 W. In my opinion is that too high for my purposes.

So, I take a look at the expensive SG-2220. For clarity I started a new topic at https://forum.pfsense.org/index.php?topic=117873.0

It needs some tweaks for power settings in pfSense but I got the X11SBA-LN4F down to 10 to 11 Watt at idle. This board contains essentially a second computer that runs the IPMI remote management and it draws 3.5 Watt constantly, and it doesn't seem possible to turn it off. The four network ports account for a watt or 2 even if not being used but it may be possible to add some settings to get them to power down if not used. Without the second on board computer for the remote management it would idle at around 7 to 8 Watts which isn't too bad.

Regards

Phil

user09

@Phil_D:

10 to 11 Watt at idle.

Unfortunately, for me is that too much.

So now i have two options:

SG-2220: idle: 6 W
german server builder:

Intel Celeron N2930, no AES-NI
8 GB DDR3
80 GB SATA III Intel SSD MLC 2,5“ (DC S3510)
max power consumption: 10 W
idle power consumption: unknown
LAN Onboard: 2x 1 GBit/s LAN (RJ-45) Intel 82583V
details: https://www.thomas-krenn.com/en/products/rack-server/technical-datasheet/frame.only_content/key.13078.html

Which would you prefer?

My min. requirements:

600 Mbit Firewall Throughput
100 Mbit OpenVPN

GruensFroeschli

Hate to repeate myself, but the APU2 fits your requirements.

messerchmidt

using a core2 e6420 with 4gb ram and a 160gb hd here (old old pc) and added 2x intel pcie gigabit nic. for one user, me - works fine for 250/20 cable using squaid, squidguard, snort, pfblocker,etc

would suggest zoltac ci323 with 8gb and a ssd for new home builds on the cheap