Hardware for Gigabit Fiber and Openvpn?
-
If it helps anyone on this thread, I built this system a few months ago:
CPU: Intel Core i3-7320 4.1GHz Dual-Core Processor
CPU Cooler: Noctua NH-L9i 33.8 CFM CPU Cooler
Motherboard: ASRock H270M-ITX/ac Mini ITX LGA1151 Motherboard
Memory: Corsair Vengeance LPX 8GB (2 x 4GB) DDR4-2400 Memory
Storage: Samsung 830 Series 128GB 2.5" Solid State Drive
Case: Mini-Box M350 Case w/ 150W PicoPSU
pfSense 2.4 betaOpenVPN throughput test:
openvpn –genkey --secret /tmp/secret
time openvpn --test-crypto --secret /tmp/secret --verb 0 --tun-mtu 20000 --cipher aes-256-cbcThis test finishes in 6.39s, giving about 500Mbps theoretical limit. During the test, htop shows 50-70% CPU usage on one core although it does vary quite a bit. I can max out my 70Mbps cable connection and it doesn't break a sweat, but plan to move to 1Gb fiber in the near future.
Snort:
I haven't finished setting Snort up yet, but @datum posted some benchmarks here https://forum.pfsense.org/index.php?topic=123801.msg689437#msg689437. Snort CPU usage is highly dependent on the rule sets enabled though. It's really hard to get an idea of performance impact since people usually don't post their rule/inspection settings.
Power Usage
13W Idle/low load, 45-55W heavy load -
Similar Results on 2.4 beta:
CPU: i3-6100 3.7GHz
openvpn –genkey --secret /tmp/secret
time openvpn --test-crypto --secret /tmp/secret --verb 0 --tun-mtu 20000 --cipher aes-256-cbcThis test finishes in 6.99s,
Running: i340-t4 nic, in-line Surricate, Squid, PfBlockerNG, OpenVPN
Speed test runs at 70/70 down/up and cpu runs at 10-13% -
Recently got gigabit fiber and run Ethernet straight form the ONT. I currently have pfsense virtualized on a poweredge with a decent 6x12 running on top of ESXI 6.5. I've tried e1000 and vmxnet3 drivers and messed around with hardware offloading. (better results with e1000) Long story short after much testing and tweaking.. my throughput was kind of meh. I saw around half the throughput with openvpn client. I have an edgerouter x sfp for lab purposes and I was getting around 800mbps.
That led me to getting this:
IN WIN CE685.FH300TB3 Black MicroATX Slim Case
ASRock H270M-ITX/ac LGA 1151 Intel H270 Mini ITX Motherboards
Intel Core i3-7350K Kaby Lake Dual-Core 4.2 GHz LGA 1151 61W
CORSAIR Vengeance LPX 8GB 288-Pin DDR4 SDRAM DDR4 2400
Noctua NH-L9i 95mm SSO2 CPU Cooler
Spare 120GB SSD
I will report back with my results. For science!
-
For those on pfSense 2.4, curious what –cipher aes-256-gcm and aes-128-gcm will give compared to just aes-256-cbc and aes-128-cbc. For fun and profit!
-
For those on pfSense 2.4, curious what –cipher aes-256-gcm and aes-128-gcm will give compared to just aes-256-cbc and aes-128-cbc. For fun and profit!
https://forum.pfsense.org/index.php?topic=105238.msg709164#msg709164
A synthetic test was all I could really do. My server is GCM but upload speed is only 15Mbps. Very few VPN providers use GCM at this time.
TL;DR, It doesn't make a big difference in synthetic tests but I'm not convinced that test is valuable for testing this.
https://calomel.org/aesni_ssl_performance.html
This study shows a big difference in AES-NI for GCM over CBC.
-
@Finger79, some additional data points for you. With the i3-7320:
time openvpn --test-crypto --secret /tmp/secret --verb 0 --tun-mtu 20000 --cipher aes-256-cbc Sun May 7 08:18:22 2017 disabling NCP mode (--ncp-disable) because not in P2MP client or server mode 6.41 real 6.40 user 0.00 sys time openvpn --test-crypto --secret /tmp/secret --verb 0 --tun-mtu 20000 --cipher aes-256-gcm Sun May 7 08:18:38 2017 disabling NCP mode (--ncp-disable) because not in P2MP client or server mode 5.89 real 5.88 user 0.00 sys time openvpn --test-crypto --secret /tmp/secret --verb 0 --tun-mtu 20000 --cipher aes-128-cbc Sun May 7 08:18:54 2017 disabling NCP mode (--ncp-disable) because not in P2MP client or server mode 6.28 real 6.28 user 0.00 sys time openvpn --test-crypto --secret /tmp/secret --verb 0 --tun-mtu 20000 --cipher aes-128-gcm Sun May 7 08:19:09 2017 disabling NCP mode (--ncp-disable) because not in P2MP client or server mode 5.73 real 5.72 user 0.01 sysWhich gives us:
AES-256-CBC, 3200/6.41 => 499.22 Mbps
AES-256-GCM, 3200/5.89 => 543.29 Mbps
AES-128-CBC, 3200/6.28 => 509.55 Mbps
AES-128-GCM, 3200/5.73 => 558.46 Mbps -
@eptesicus - Were you able to get the asus to install pfsense?
I can't for the life of me get it to boot/ install off a thumbdrive. It just gives me an error code AE and sits there.
The computer is working fine, I've tested it with a windows thumbdrive and it has no problem booting off of that.
I created the thumb drive with the AMD64 img file using win32 disk imager
Edit- So 2.3.4 would not install for the life of me. Switched to 2.4 beta from 5/9, worked like a champ… Now I just need to go through the exercise of setup
-
Could be a UEFI setting in the BIOS… I don't think the 2.3.x branch supports UEFI like 2.4 does.
If you change the boot mode to Legacy, you'll probably be able to install 2.3.4.
-
@eptesicus - Were you able to get the asus to install pfsense?
I ended up installing OPNSense instead.
-
That led me to getting this:
Intel Core i3-7350K Kaby Lake Dual-Core 4.2 GHz LGA 1151 61W
I will report back with my results. For science!
How did this end up working out for you?
What kind of OpenVPN speeds are you getting, at what encryption levels and at what CPU utilization? Have you used IDS/IPS on this at all?
-
Be aware that if you are using SMB you will see horrible performance yet if you do an iperf test you will saturate the maximum. I thought IPSec was having problems on my pfsense. I could only get 4mbit on a 10mbit link yet iperf could push 9.86 mbit.
-
Your best bet (within a reasonable budget) for OpenVPN performance would be an i3-7350K, it's relatively inexpensive and has I think one of the highest clock speeds of any intel CPU.
It's an awesome value gaming CPU too: https://www.cpubenchmark.net/singleThread.html
Number three on the list - amazing for an i3.
