Zotac CI 321 Dual NIC Nano

interfasys

That's probably the cipher you need for OpenVPN

# openssl speed -elapsed -evp aes-256-cbc
You have chosen to measure elapsed time instead of user CPU time.
Doing aes-256-cbc for 3s on 16 size blocks: 944551 aes-256-cbc's in 3.01s
Doing aes-256-cbc for 3s on 64 size blocks: 874272 aes-256-cbc's in 3.00s
Doing aes-256-cbc for 3s on 256 size blocks: 707852 aes-256-cbc's in 3.01s
Doing aes-256-cbc for 3s on 1024 size blocks: 410113 aes-256-cbc's in 3.03s
Doing aes-256-cbc for 3s on 8192 size blocks: 80373 aes-256-cbc's in 3.01s
OpenSSL 1.0.2f  28 Jan 2016
built on: reproducible build, date unspecified
options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: clang37 -I. -I.. -I../include  -fPIC -DOPENSSL_PIC -DOPENSSL_THREADS -pthread -D_THREAD_SAFE -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DL_ENDIAN -O3 -Wall -O2 -pipe -D_FORTIFY_SOURCE=2 -flto -march=native  -fstack-protector-strong --param ssp-buffer-size=4 -fno-strict-aliasing -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM -DECP_NISTZ256_ASM
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
aes-256-cbc       5024.52k    18651.14k    60246.48k   138542.09k   218901.82k

You'll easily reach 100Mbit, using one core, which leaves you with 3 for other things to do.

perth

@interfasys:

That's probably the cipher you need for OpenVPN
…
You'll easily reach 100Mbit, using one core, which leaves you with 3 for other things to do.

Thank you interfasys! Here's the result of the aes-256-cbc scheme speed test on the CI321:

[2.2.6-RELEASE][admin@redacted]/root: openssl speed -elapsed -evp aes-256-cbc
You have chosen to measure elapsed time instead of user CPU time.
Doing aes-256-cbc for 3s on 16 size blocks: 14575858 aes-256-cbc's in 3.03s
Doing aes-256-cbc for 3s on 64 size blocks: 3854920 aes-256-cbc's in 3.00s
Doing aes-256-cbc for 3s on 256 size blocks: 973141 aes-256-cbc's in 3.01s
Doing aes-256-cbc for 3s on 1024 size blocks: 245621 aes-256-cbc's in 3.01s
Doing aes-256-cbc for 3s on 8192 size blocks: 30538 aes-256-cbc's in 3.00s
OpenSSL 1.0.1l-freebsd 15 Jan 2015
built on: date not available
options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx) 
compiler: clang
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
aes-256-cbc      76936.49k    82238.29k    82825.67k    83620.87k    83389.10k

Applying the same formula I made up earlier:
((blocksEncrypted * bytesPerBlock) * 8 / seconds)/(1024^2) = Mb/s
16B = 587 Mb/s
64B = 627 Mb/s
256B = 631 Mb/s
1KB = 637 Mb/s
8KB = 636 Mb/s
(decimals truncated)

Did a little more looking around on the internet regarding throughput on custom firewalls, and the internet says we are paying too much attention to the CPU.
We should be looking at the bus the NIC is on. https://calomel.org/network_performance.html

interfasys

Interesting to see how much slower the C321 (1.1Ghz) is in this openssl test compared to the C323 (1.6Ghz) with AESNI.

But regardless, the speed is going to depend a lot on how you're able to tune your connection. If you're connecting to a public provider, you'll be limited to what they offer and ~100Mbit is what you're going to reach.

Regarding the bus, there is nothing to worry about today. PCI Express 1x is enough to drive even a dual-port Intel NIC.

And beware of tips such as

Ideally you want to use a server based add on card with a TCP offload engine or TCP accelerator.

Netmap is fast and requires all hardware acceleration to be turned off.
It's still a good idea to get Intel NICs for Gigabit WAN connections (or peace of mind), but it's tricky to add to a Zotac Nano…

perth

@interfasys:

Interesting to see how much slower the C321 (1.1Ghz) is in this openssl test compared to the C323 (1.6Ghz) with AESNI.
…

Interfasys, are you running the CI323? All my tests are on the CI321. If one of my posts was misleading please let me know which one & I'll edit it for clarity.
Secondly, I find the differences between our encryption speed tests very interesting, the CI321 is stomping your platform on the smaller block sizes. Yet your platform is annihilating mine on the large block sizes.

Comparing results of 'openssl speed -elapsed -evp aes-256-cbc' on our platforms:


CI321 ------------------------------------------------------------------------------
type                  16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
aes-256-cbc           76936.49k    82238.29k   82825.67k   83620.87k    83389.10k
Recalculated as Mb/s  587          627         631         637          636
Interfasys -------------------------------------------------------------------------
type                  16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
aes-256-cbc           5024.52k    18651.14k    60246.48k   138542.09k   218901.82k
Recalculated as Mb/s  38           142         459         1057         1668

Seeing the compiler string in your output, I wonder:
Did you custom compile openssl?
Is that compiler string caused by installing openssl as a port instead of a pkg?
Looking forward to a little more detail on the platform you ran that test on. I think I have more questions for you. :-)

interfasys

All my tests are on the CI321. If one of my posts was misleading please let me know which one & I'll edit it for clarity.

It was clear that you were on the CI321. I provided my numbers for @unknownUser since he was considering the CI323 to fill his requirements.

Secondly, I find the differences between our encryption speed tests very interesting, the CI321 is stomping your platform on the smaller block sizes. Yet your platform is annihilating mine on the large block sizes.

Indeed, here are the results without using evp

type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
aes-256 cbc      24665.50k    26819.67k    27680.79k    75580.20k    76382.63k

I find it unusual to get lower numbers for small block sizes using AES-NI vs not using it.

Your formula is wrong by the way, the numbers given are already in MB/s
(80373 * 8192) / 3.01 ~= 218.9 MB/s

Did you custom compile openssl?
Is that compiler string caused by installing openssl as a port instead of a pkg?

Yes

interfasys

Look at the APU2 numbers, same thing happens when enabling AES-NI.
https://forum.pfsense.org/index.php?topic=106444.msg593101#msg593101

interfasys

A test you can run

# openvpn --genkey --secret /tmp/secret
# time openvpn --test-crypto --secret /tmp/secret --verb 0 --tun-mtu 20000 --cipher aes-256-cbc
Sat Feb  6 18:39:04 2016 WARNING: file '/tmp/secret' is group or others accessible

real    0m26.146s
user    0m24.424s
sys     0m1.709s

interfasys

And another one

[code]# dd if=/dev/zero bs=1M count=100 >> /dev/null
100+0 records in
100+0 records out
104857600 bytes transferred in 0.015134 secs (6928676210 bytes/sec)

# dd if=/dev/zero bs=1M count=100 | openssl aes-256-cbc -e -pass pass:secretpwd | openssl aes-256-cbc -d -pass pass:secretpwd >> /dev/null
100+0 records in
100+0 records out
104857600 bytes transferred in 0.872586 secs (120168784 bytes/sec)[/code]

Rango

@perth:

So the Disable Hardware Checksum Offload setting may have improved the reliability of my connection.
However I'm still getting excessive disconnects running this setup.

I think I'm going to try setting System > Advanced > Firewall and NAT > Disable Firewall Scrub "Disables the PF scrubbing option which can sometimes interfere with NFS and PPTP traffic." Since PPPOE is PPTP OE, if I understand correctly. Lets also disable IPv6, thought I had already turned that off…

Need to replace the modem and see what that does.

Can you guys confirm the wan disconnections have been fixed by doing this optioned mentioned above?
Thanks you in advance. BTW could it be possible it's cause you're on PPPOE?

How long after you guys placed the order did you guys get it?

Rango

Does this box ci321 mobo support dual channel memory and does it have 2 slots for memory?
I'm looking at cpu and seems to support it but that's based on mobo not cpu.
If so would it be better for performance to install 2x4GB ram as it would utilize dual channel ?

Also i'm assuming you guys are running sata drive not msata correct?

Memory Specifications
Max Memory Size (dependent on memory type) 8 GB
Memory Types DDR3L-1600
Max # of Memory Channels 2
ECC Memory Supported ‡ No

http://ark.intel.com/products/87258/Intel-Celeron-Processor-N3150-2M-Cache-up-to-2_08-GHz

hardsense

@Perth

Would you do us a favour by testing below ?

It's a simple and effective test .

Test 1:

Connect 1 host to WAN interface of your pfsense box, connect another host to LAN interface of your pfsense box and perform a iperf test and provide us with screenshots as usual.

Test 2:

Install the DHCP service on the host connected on the WAN side and configured as WAN DHCP server and let it run for 24 hour or so and see if the connection drop or not.

P.S The ideal test -> The host are equipped with Intel LAN card and running current Desktop processor and hardware . If you don't have it , just run the test with whatever you have in your inventory.

Thanks.

Rango

@G.D.:

Another deficiency (though maybe not very important in case of pfSense) is that despite having two memory slots this box is configured for single channel operation.

Anyway, has anyone tried Zotac CI321 with pfSence yet?

Where did you get that this box is single channel operation?

In fact cpu cpecs on i323 say opposite that it has 2 channels for memory.

Rango

Regarding the bus, there is nothing to worry about today. PCI Express 1x is enough to drive even a dual-port Intel NIC.

Hmmm…..looking at the box it seems impossible to add any pci-e card unless one changes the box and it would have to be sideways not vertical?
hmmm seems impossible to add any external nic to this???

I hope the realtek nics are working good with pfsense?

g1bson

Sorry for my broken english,
but I didn't understand - did anyone install Pfsense on Zbox CI323?
Zotac tichnical support asserts, that no other OS can be installed on CI323 except for Windows cause of UEFI BIOS.
Here is the anwser on question about other OS and CSM/MBR support in BIOS:
"As for the legacy OS support unfortunately no, its wont support such OSes but Windows 7 can be installed with the use of the custom Windows 7 installation tool that can be downloaded on the unit's download center."
Can anyone explain this situation to me? :)

perth

Holy Carp there's activity in this thread. Guess I better update my e-mail preferences… :-)

I'll give this update regarding the reliability of pfSense on the CI321. I've replaced the modem w/ the ISP's all in one modem/router and no longer have disconnection issues. I can't SSH into my pfsense box from the outside, but I've really not put too much effort info fixing that & I've prob got the firewall rule/NAT wrong. Or it's that stupid router I really didn't want to use any more. :/
~~Really need to call the modem's vendor and have them replace it & see what happens.~~ RMA process has begun.

@Hardsense
@hardsense:

@Perth
Would you do us a favour by testing below ? …

Sorry hardsense I'll take a look at doing that this weekend. However test 2 seems… pointless. I must not be understanding something. Perhaps you want me to run some load during those 24 hours? If you're trying to identify the instability issue I mentioned for PPPoE on ADSL, see above. I'm using a static IP for the WAN port in this new setup, but I could change that to DHCP if you like.

@Rango
@Rango:

@perth:

So the Disable Hardware Checksum Offload setting may have improved the reliability of my connection.
However I'm still getting excessive disconnects running this setup.
…
I think I'm going to try setting System > Advanced > Firewall and NAT > Disable Firewall Scrub "Disables the PF scrubbing option which can sometimes interfere with NFS and PPTP traffic." Since PPPOE is PPTP OE, if I understand correctly. Lets also disable IPv6, thought I had already turned that off…

Can you guys confirm the wan disconnections have been fixed by doing this optioned mentioned above?
Thanks you in advance. BTW could it be possible it's cause you're on PPPOE?

How long after you guys placed the order did you guys get it?

Those steps did not fix my disconnection issue. I ordered the CI321 from Amazon and received it in the normal fast fashion (Prime!).

@Rango:

Hmmm…..looking at the box it seems impossible to add any pci-e card unless one changes the box and it would have to be sideways not vertical?
hmmm seems impossible to add any external nic to this???

I hope the realtek nics are working good with pfsense?

You could replace the included wireless nic? ;)

272KB full res.

Nah, I'd look into a USB3 Ethernet adaptor.

messerchmidt

i would got for the ci323 if purchasing new…