Best 2017 hardware for gigabit fiber + VPN
-
That's a lot of reading to digest but also a lot of useful information, thank you guys.
So to sum it up, with OpenVPN now supporting GCM, and regarding the benchmarks here, we can safely say that any 2 GHz kaby lake processor would be enough to handle 1 Gbps AES-256-GCM encryption. (the benchmarks shows 2.20 GHz broadwell encrypting @ 1.08 cycles per byte).
The openssl crypto can perform at multiple gigabytes per second, but the openvpn overhead will make your throughput limit much, much lower. It's possible to configure openvpn to use larger blocks (setting tunnel mtu to something like 32 or 64k) which will let you hit 1gbps, but then the tunnel is extremely sensitive to packet loss and your real world performance may actually be lower. (Also, why on earth would you use AES-256 instead of AES-128?)
The problem appears to be that OpenVPN has some bottleneck elsewhere so it's not enough to have 1 cycle per byte for GCM encryption. To get the maximum performance, we can use gateways pfsense groups as @whosmatt suggests. That implies that we need 4 cores to scale well, with OpenVPN running on 3 or 4 cores. Or maybe pfsense / FreeBSD / OpenVPN supports intel hyper threads and two cores with 4 HT could be enough.
If this sort of scaling is sufficient, you likely don't need one core per openvpn process. (The way openvpn hits its bottleneck tends to be that it can't keep enough packets in flight to saturate the CPU, not that it maxes out the CPU.) Note that you won't ever get a full gigabit on a single connection this way, but that might be sufficient for your needs.
-
I'm planning to build my first pfSense box. I live in Europe and I have a 1 gigabit fiber optical connection. I plan to use OpenVPN or IPSec or what is the fastest VPN I can get on the box. Therefore, the CPU must be at least AES-NI.
If the CPU is strong or powerful enough it might be not necessary to have AES-NI inside, it might be good for IPsec for
sure but at the moment for OpenVPN it is may be not really pushing the throughput. Doing an OpenSSL test on the
same unit is not the same as doing it between two units over the internet.After much reading, it seems that the best platform is a CPU released in 2013: Intel Atom C2758. It supports:
- AES-NI
- Intel QuickAssist
- DPDK support (enabled software)
- AES-NI is nice to have and I personally would even buy a CPU or SoC with it inside, for sure.
- QuickAssist hm, not so really to answer but it can be a really gainer and for the professional and the enterprise segment
it might be in the future a really need or nice to have it in! If I install such cards in my Linux, BSD and MS Windows Servers
and then the firewall would be the bottleneck? Why? And if on both or more sites of a VPN connection QuickAssist is handling
the internally compression it might be a good sounding thing to get also profit from that in the lower end areas such the home
network area, for sure not for all peoples this must matching well or fitting their needs. - DPDK is a SDK from Intel for network based units and the SDPK is a SDK for storage units to code better and more nearly
the hardware or to unleash their full potential, so if this might be getting inside of the FreeBSD and/or pfSense source code
it can be pushing the TCP/IP transporting rate up to x3 faster then today, and please remember with the same hardware!
So many limitations like single CPU core threaded PPPoE, the entire and common throughput, up to the ability to saturate
and fairly use a 10 GBit/s line or link, will be then gone! So it might be nice to know that Supermicro is starting to produce
the next generation of the boards in the mid of 2017 based on an Intel D-1500 series that is offering all of this three things
natively! (DPDK & SPDK, AES-NI & QuickAssist)
Source : here
Things often changes more then we love it.
Therefore, I'm about to go with the configuration listed here :
- Super Micro SUPERMICRO A1SRI-2758F - Motherboard - Mini-ITX - Intel Atom C2758 - USB3.0 - 4 x Gigabit LAN - Onboard-Grafik (A1SRI-2758F-O)
- MS-Tech CI-57 - Ultra Small Form Factor - Mini-ITX - 120 Watt (CI-57-120)
- SanDisk SSD - SSD - 64GB - 6,4 cm (2.5") - SATA-600 (SDSSDP-064G-G25)
- Kingston KVR16LSE11/8HB
Might be a real pfSense bomb but not really hitting the 1 GBit/s mark! Perhaps around 850 MBit/s!
Without PPPoE you will be really nice sorted, perhaps to much, but this might be also pointed to the use case
and the installed packets.It seems a shame to buy a 2013 cpu in 2017 that has been reported by Intel to have a bug that will prevent him from booting :(
Who is pressing you? If the PPPoE will be surely not there a smaller unit will be also matching well to your needs!
Jetway NF9HG-2930 is doing then the job for you for around 350 Euro and able to realize much more!So, what do you guys think about it ? Should I go for the above setup or use a i3/i7 skylake/kaby lake that supports AES-NI and forget about QAT/DPDK ? Any coments are welcome !
Only for SPI/NAT & VPN it will be to much or with other words go and have a look for the Qoton J1900 box for $260
that will not route that 1 GBit/s at the WAN interface but is nice and cheap. If you are willing to be future proof the
C2758 will be a really bomb, or you take something between this parts, like a Intel core i3 cpu and you will be happy! -
I'd never heard of DPDK before this thread (thanks for that!) but it looks like you don't need to worry about it when picking a CPU.
I just found some more info on DPDK and pfSense and thought I'd share here since you asked:
https://www.netgate.com/blog/pfsense-around-the-world-better-ipsec-tryforward-and-netmap-fwd.html
Back in February, I wrote a blog post that discussed our plans for pfSense software version 2.3, which is now in alpha, and our plans for pfSense 3.0. While I promoted DPDK then, we’ve since found that netmap provides a simpler API, and substantially better safety, as the device drivers remain in the kernel, rather than running in userspace with DPDK. Still, DPDK provides a set of libraries, such as longest-prefix match, which uses a variation of the DIR-24-8 algorithm for routing lookups, which we should find useful in our pursuit of the ultimate open source software router.
It looks like the intent is to move away from DPDK but will still implement it in some form? Hopefully someone more knowledgeable can chime in but I thought I'd share the post.
-
It looks like the intent is to move away from DPDK but will still implement it in some form?
Perhaps not for the general or public usage? Who knows. Link
Hopefully someone more knowledgeable can chime in but I thought I'd share the post.
Would be nice to hear some news about that DPDK, netmap-fwrd and QuickAssist this might be the
most interesting things for me to hear about. -
@BlueKobold:
I'm planning to build my first pfSense box. I live in Europe and I have a 1 gigabit fiber optical connection. I plan to use OpenVPN or IPSec or what is the fastest VPN I can get on the box. Therefore, the CPU must be at least AES-NI.
If the CPU is strong or powerful enough it might be not necessary to have AES-NI inside, it might be good for IPsec for
sure but at the moment for OpenVPN it is may be not really pushing the throughput. Doing an OpenSSL test on the
same unit is not the same as doing it between two units over the internet.I really wish you would stop spreading FUD about AES-NI and OpenVPN. There is a significant performance benefit from using AES-NI hardware with OpenVPN, end of story.
So, what do you guys think about it ? Should I go for the above setup or use a i3/i7 skylake/kaby lake that supports AES-NI and forget about QAT/DPDK ? Any coments are welcome !
Only for SPI/NAT & VPN it will be to much or with other words go and have a look for the Qoton J1900 box for $260
that will not route that 1 GBit/s at the WAN interface but is nice and cheap. If you are willing to be future proof the
C2758 will be a really bomb, or you take something between this parts, like a Intel core i3 cpu and you will be happy!Neither the C2758 nor the J1900 can do gigabit OpenVPN. If that's the requirement, go with a fast current i3 (you still probably won't hit a full gigabit without some questionable tuning tradeoffs, but you'll get a lot closer). Forget QAT, it's a pipe dream. You don't need DPDK.
-
careful with the c2758 as it may have the clock generator bug depending on the revision: https://www.theregister.co.uk/2017/02/06/cisco_intel_decline_to_link_product_warning_to_faulty_chip/
you could get a ryzen 1700 with a b350 board as well
-
you could get a ryzen 1700 with a b350 board as well
I am looking forward to seeing Ryzen CPU's popping up in the pfSense world to see where they fit in for budget horsepower.
-
you could get a ryzen 1700 with a b350 board as well
It would be nice to see some relevant benchmarks for that platform.
-
This maybe helps goo.gl/WWGIcT
That Ryzen really kicks Xeon behinds.
The past two weeks I'm like a hawk on reading steroids. Trying to find the definitive answer on the "how to obtain 1 gbps (VPN) throughput without applying for a new mortgage" question. This thread is a great help so thank you all guys!
-
This maybe helps goo.gl/WWGIcT
Nope, those are pretty irrelevant benchmarks. :) At this point I don't have any real reason to think that a $400 ryzen will outperform a $120 i3 for this application. (It's not an application that scales well with increasing core count, and it does respond well to increasing clock speed. Up to a point–it also gets bottlenecked by packet/buffer sizing issues and increasing CPU performance tends to see diminishing returns.) It's a conversation that will get more interesting when we see what AMD offers on the lower end because building a pfsense on a 1700x doesn't make a whole lot more sense than building one on an E5. If AMD offers something comparable to denverton at a much better price/performance ratio than we've seen so far from avoton/rangely (intel's denverton strategy is still mostly a mystery), then that will shake up this space.
Trying to find the definitive answer on the "how to obtain 1 gbps (VPN) throughput without applying for a new mortgage" question.
Either that's hyperbole or you have a really small mortgage. The answer to this is pretty straightforward–buy the highest clocked i3 or i5 you can find/afford. You are unlikely to hit 1Gbps with a single OpenVPN stream regardless of the CPU. If you use multiple OpenVPN instances or use a different implementation (ipsec) then the 1Gbps target isn't that hard or expensive. Certainly less than the ryzen 1700x.
-
This thread has been cold for a few months, but it's still seems relevant. I'm also looking for the best inexpensive hardware for a full time VPN tunnel to saturate a 1GB fibre connection.
Looking about, it seems like these are good candidates:
- http://www.lannerinc.com/products/x86-network-appliances/desktop/?option=com_content&view=article&id=1879:nca-1031&catid=26:desktop
http://www.jetwaycomputer.com/JBC38AF542AA.html
http://www.jetwaycomputer.com/JBC390F541AA.html
Although the J1900, while cheap doesn't offer the AES-NI instructions, which seems like a non-starter for a VPN device.
Any thoughts?
Most polished/appliance-like look. Atom x7-3950… less CPU power than the Haswell device
Most power, and bad-ass looking heat fin case. Haswell processor
Least powerful, but cheap. There are even cheaper J1900 devices from Qoton
- http://www.lannerinc.com/products/x86-network-appliances/desktop/?option=com_content&view=article&id=1879:nca-1031&catid=26:desktop
-
Not even close. You won't fit gigabit VPN even with the latest high clock i7.
Gigabit openvpn is limited by openvpn at this point.
You can get gigabit openvpn with gateway groups on an i3, but that setup has its own set of limitations and advantages.
-
Not even close. You won't fit gigabit VPN even with the latest high clock i7.
Gigabit openvpn is limited by openvpn at this point.
You can get gigabit openvpn with gateway groups on an i3, but that setup has its own set of limitations and advantages.
Hmmm… what about IPSec IKEv2? I'm less worried about industrial espionage and more worried about my ISP selling/analyzing my connection log, traffic and browser history. Perhaps a lower level of encryption would be adequate?
-dw
-
I've never used ipsec but I would guess you'd have no problem with that. You can use the oldest most broken / compromised encryption you want for that.
Your ISP will not attempt to decrypt your encrypted traffic no matter how easy it might be to do so.
-
Not even close. You won't fit gigabit VPN even with the latest high clock i7.
Gigabit openvpn is limited by openvpn at this point.
You can get gigabit openvpn with gateway groups on an i3, but that setup has its own set of limitations and advantages.
Hmmm… what about IPSec IKEv2? I'm less worried about industrial espionage and more worried about my ISP selling/analyzing my connection log, traffic and browser history. Perhaps a lower level of encryption would be adequate?
-dw
IPsec isn't less secure than openvpn, it's just more of a pain to set up and much harder to reliably access from arbitrary locations on the internet. If you can use IPsec it's likely to perform better, but if you don't control both ends it might be hard to get working.
-
Did someone say an I5-7600k can't do a gigabit/s but they think the lastest I3 can?
-
Did someone say an I5-7600k can't do a gigabit/s but they think the lastest I3 can?
No, they said even the fastest CPU can't achieve single stream gigabit because of non-CPU bottlenecks which dominate far below 1gbps. With multiple OpenVPN instances you can achieve 1gbps in aggregate even with a relatively modest CPU.
-
Explain what this bottle neck is clearly then. Can I just use one PFsense router and run multiple instances of the VPN. Then tell the same Pfsense router to merge it as a multi-WAN connection?
-
Somewhere in openvpn software it simply does not scale to gigabit.
So you create multiple instances, which will utilize multiple cores/threads. Create a gateway group and you can bypass the restriction for some types of traffic but not all. I.e., anything that uses only one connection will be limited to the max throughout of one openvpn instance.
-
Explain what this bottle neck is clearly then.
On OpenVPN the TUN/TAP architecture for sure.
Can I just use one PFsense router and run multiple instances of the VPN.
On OpenVPN you might be able to set up several tunnels and they all can be running on one cpu core each!
Over IPsec you might be able to set up also more then one IPsec tunnel too, but with the need of more IP addresses.