Hardware Configuration for Optimal VPN Connection Speeds
-
Good afternoon all. I'm in the process of setting up and configuring a network which will exist permanently behind a pfSense box, which will be utilized as a DHCP server, proxy, firewall, and VPN access point. This VPN service is the primary driving force for the following hardware configuration, as this network will require several simultaneous VPN connections with minimal bandwidth limitations (beyond what is reasonable). The network is served by a 100/25 Mbit connection, which is able to sustain multiple clients at this speed. Based on CPU single-thread PassMark data and other user data on OpenVPN performance, this hardware configuration should be able to facilitate up to 300Mbps throughput on the VPN with 256-bit AES encryption.
-
Chassis/Board: https://www.supermicro.com/products/system/Mini-ITX/1018/SYS-1018L-MP.cfm?parts=SHOW#jump
-
CPU: http://www.newegg.com/Product/Product.aspx?Item=9SIA85V4R42234&cm_re=i3-4170--19-117-543--Product
-
HDD: http://www.newegg.com/Product/Product.aspx?Item=9SIA12K2EV4048&cm_re=850_evo_120gb--20-147-371--Product
-
Mem: http://www.newegg.com/Product/Product.aspx?Item=9SIA8BU43H2071&cm_re=crucial_ddr3_laptop--9SIA8BU43H2071--Product
I will post updates and my findings as this project continues. Comments, feedback/input appreciated.
-
-
Looks promising ! Why didn't you opt for the low power CPU? Is that not something you are interested in?
-
Power demands are not a concern. While a low power chip would certainly have done the trick for standard connectivity, the point of this box is maximum VPN throughout. Sure, the chip runs at 54W under load, but the low power alternative (like an Atom 2758) has barely 25% the single-threaded performance. Since encryption through OpenVPN is only compiled to take advantage of single-core performance, this is the better option. Even though it's only dual-core, it features hyper-threading and the remaining physical/virtual cores will be more than capable of running the remaining functions while permitting the VPN to run at its maximum speed. For that, the trade off of having to pay $70 or so a year in power is well worth it.
-
if power draw is no concern then why don't you just get an old desktop off of ebay or craigslist or just a garage sale?
I got an old used business class desktop with an i5-2400 for significantly less than the price of just the cpu you listed.
I run a VPN client and server that then routes traffic through the client. The client encrypts at AES-256, as do some of the clients served by the server, and some use 128 bit.At about 50Mbps over then VPN it uses about 15% CPU, it's usually less than 5%.
It's also running PfblockerNG w/ DNSBL and has all settings set for max performance.Basically, I'm sure that what you're buying will do what you want it to, but I'm betting that you are paying WAY too much for it. Buy used and you can probably spend less than 25% of what you're planning on spending.
-
Basically, I'm sure that what you're buying will do what you want it to, but I'm betting that you are paying WAY too much for it. Buy used and you can probably spend less than 25% of what you're planning on spending.
I certainly see your point, and if I were building this for personal use I would probably go with that option, but the client I'm working on this for would prefer a COTS custom build to this spec, and they're footing the bill as well. I could probably get something like 70% of the throughput with an i5-2400, but the sheer single-threaded performance of the i3-4170 just kicks butt. It ranks at #77 on PassMark's single-threaded CPU benchmark with a rating of 2,129 at stock clock, whereas the latest and greatest i7-7700k only hits 400 or so points above it at 2,599, and even some of that can be attributed to its significantly higher clock speed.
All in all, this is the best performance scenario I could discover, and that along with client preference was the reasoning behind this build.
-
If you already pulled the trigger then disregard the rest of this. :) At this point I'd definitely get a Kaby Lake i3 over a Haswell i3, especially for VPN use (the newer architecture has improved AES GCM performance). An i3-7100 is faster, can take faster RAM, has slightly lower TDP, and is basically the same price.
-
Hi all,
I've looked through so many posts my head is hurting! :)
I purchased the Qotom Q190 yesterday and cancelled the order today as I discovered that via a VPN connection the speeds would be the same as what I get via my current Asus RT3200 router!
My current setup is with my ISP's device in modem mode connected to the Asus router which has OpenVPN running on it, the best speeds I can achieve is 40/10Mbps and from what I've read online the speeds are a result of the router hardware.
I'd like to be able to buy a fanless small device that can handle a 150/10Mbps VPN connection plus more to future proof myself, maybe something that can handle 300/100Mbps..?
Like my current setup I'd like all devices that are connected to my router to pass through the VPN….I wanted to keep cost's to a minimum but looking at several threads I need to throw half decent money at this which I am prepared to do within reason....
I have sent an email to the pfSense sales people to see if any of their off the shelf devices meet my requirements....Any help would be greatly welcomed and appreciated!
-
For fanless, not a lot of money and VPN then consider a J3355 mini itx, about $55 for the CPU/Board combo. It's got AES-NI @ 2.00 Ghz base clock. The J3455 & 4205 have 4 cores but worse single core performance, so unless multi-thread openvpn is coming soon to pfsense the cheapest 3355 might be for you unless you're going to be doing other multi-core intensive stuff.
https://www.newegg.com/Product/Product.aspx?Item=N82E16813157730
One of the drawbacks to this board is that it's PCIe slot runs at 1x. On paper I'm pretty sure it supports two gigabit connections, but I know the Intel PRO/1000 says it needs x4 speeds. I don't know what performance you would get in reality.I've seen people posting on here that they get 100Mbps on a J1900 which has about the same single core performance as a 3355, but does not have AES-NI, so based on that the 3355 should more than meet your VPN needs.
https://forum.pfsense.org/index.php?topic=99536.msg554576#msg554576Add an intel PRO/1000 or i350 and it should work great for you.
This is the cheap solution, I see you are considering the pfsense products, which will be much better as they are purpose built whereas this is not.
-
One of the drawbacks to this board is that it's PCIe slot runs at 1x. On paper I'm pretty sure it supports two gigabit connections, but I know the Intel PRO/1000 says it needs x4 speeds. I don't know what performance you would get in reality.
PRO/1000 PT is a PCIe v1 card, so it will be right at the theoretical bandwidth of a x1 slot if you're running gigabit both ways between them. In practice it's probably not going to be a noticeable limit and should work fine if you have x4 or better physical slot.
(And i350-t2 will run with no bandwidth constraint in a PCIe v2 x1 slot, you'll just cap the rate at which you can transfer buffers.)
-
One of the drawbacks to this board is that it's PCIe slot runs at 1x. On paper I'm pretty sure it supports two gigabit connections, but I know the Intel PRO/1000 says it needs x4 speeds. I don't know what performance you would get in reality.
PRO/1000 PT is a PCIe v1 card, so it will be right at the theoretical bandwidth of a x1 slot if you're running gigabit both ways between them. In practice it's probably not going to be a noticeable limit and should work fine if you have x4 or better physical slot.
(And i350-t2 will run with no bandwidth constraint in a PCIe v2 x1 slot, you'll just cap the rate at which you can transfer buffers.)
Ah, good to know!
I actually also noticed that the mini-ITX version actually has a slot running at x2, it's smaller and it costs the same. So that might be better for you if you end up going that route!
https://www.newegg.com/Product/Product.aspx?Item=N82E16813157726 -
I actually also noticed that the mini-ITX version actually has a slot running at x2, it's smaller and it costs the same. So that might be better for you if you end up going that route!
https://www.newegg.com/Product/Product.aspx?Item=N82E16813157726Note that's the dual core rather than quad core, and it looks like they screwed up the memory description for the j3455m (it says so-dimm, but it looks like a regular dimm–in which case the so-dimm on the mini-itx board will also be a bit more expensive). But the x2 slot could potentially be a performance advantage. Computer specs are such a strange game.
-
Yeah, I don't know any hard numbers to equate CPU clock to AES-xxx OpenVPN speed throughput but my understanding is that it is a single-thread affair for now and probably the foreseeable future?
If there really is no multi-core support for VPN I would think two cores with a higher clock would be better than four cores clocked lower for VPN. (Unless the OP is going to be using a lot of other CPU-intensive things).
The j3345 base is 1.5Ghz vs the 3355 @ 2.0Ghz, the 3355 also bursts 200Mhz higher than the 3455.I don't know what kind of throughput you can get on one core at 1.5Ghz with AES-NI encrypting at AES-256, but if it can do 300Mbps then that seems the way to go!
I'm pretty sure the so-dimm spec is correct, I have a J3355B mini-ITX running LibreElec for an HTPC and I'm pretty sure it has old laptop RAM in it.
-
Yeah, I don't know any hard numbers to equate CPU clock to AES-xxx OpenVPN speed throughput but my understanding is that it is a single-thread affair for now and probably the foreseeable future?
If there really is no multi-core support for VPN I would think two cores with a higher clock would be better than four cores clocked lower for VPN. (Unless the OP is going to be using a lot of other CPU-intensive things).
The j3345 base is 1.5Ghz vs the 3355 @ 2.0Ghz, the 3355 also bursts 200Mhz higher than the 3455.I don't know what kind of throughput you can get on one core at 1.5Ghz with AES-NI encrypting at AES-256, but if it can do 300Mbps then that seems the way to go!
Agree that the 30% higher single thread would be better for openvpn. As to what the actual VPN performance would end up being, I haven't seen any openssl or openvpn benchmarks on goldmont/apollo lake. Extrapolating from the older silvermont/braswell systems it wouldn't hit 300Mbps but intel specifically improved the aes-ni and pclmulqdq instructions, and made massive changes to the restrictions on instruction ordering and added a third element to the pipeline so an extrapolation from silvermont is completely bogus. (The clock is meaningless across microarchitectures, what's important is the number of cycles per byte of crypto on a given architecture. For comparison, haswell can do aes-128-gcm in a bit more than 1 cycle per byte, broadwell & skylake can do it in less than one cycle per byte, and silvermont is about 7 cycles per byte–so a skylake is more than 7 times as fast as a silvermont at the same clock speed. goldmont will probably be somewhere between 1 and 7 cycles per byte, but I have no idea where--and without that number there's no way to predict performance.)
-
Yeah, I don't know any hard numbers to equate CPU clock to AES-xxx OpenVPN speed throughput but my understanding is that it is a single-thread affair for now and probably the foreseeable future?
If there really is no multi-core support for VPN I would think two cores with a higher clock would be better than four cores clocked lower for VPN. (Unless the OP is going to be using a lot of other CPU-intensive things).
The j3345 base is 1.5Ghz vs the 3355 @ 2.0Ghz, the 3355 also bursts 200Mhz higher than the 3455.I don't know what kind of throughput you can get on one core at 1.5Ghz with AES-NI encrypting at AES-256, but if it can do 300Mbps then that seems the way to go!
Agree that the 30% higher single thread would be better for openvpn. As to what the actual VPN performance would end up being, I haven't seen any openssl or openvpn benchmarks on goldmont/apollo lake. Extrapolating from the older silvermont/braswell systems it wouldn't hit 300Mbps but intel specifically improved the aes-ni and pclmulqdq instructions, and made massive changes to the restrictions on instruction ordering and added a third element to the pipeline so an extrapolation from silvermont is completely bogus. (The clock is meaningless across microarchitectures, what's important is the number of cycles per byte of crypto on a given architecture. For comparison, haswell can do aes-128-gcm in a bit more than 1 cycle per byte, broadwell & skylake can do it in less than one cycle per byte, and silvermont is about 7 cycles per byte–so a skylake is more than 7 times as fast as a silvermont at the same clock speed. goldmont will probably be somewhere between 1 and 7 cycles per byte, but I have no idea where--and without that number there's no way to predict performance.)
That is a ton of good info! Just shows me how little I know about all of this.
Out of curiosity, how much will Intel quickAssist increase performance and are there any low power products out now that support it in a reasonable price range? I don't really fully understand quick assist, is it completely separate tech from AES-NI? Does it work along with it or is a replacement?
Also, unrelated, but how do the SoC Intel i-series NICs compare to the PCIe versions? any difference? -
Out of curiosity, how much will Intel quickAssist increase performance and are there any low power products out now that support it in a reasonable price range? I don't really fully understand quick assist, is it completely separate tech from AES-NI? Does it work along with it or is a replacement?
quickassist is an off-cpu coprocessor. It can do AES, SHA, RSA, even things like compress/decompress. Just like CPUs, there are different revisions with different performance characteristics. (The one built into the rangeley product line is different from the current add-in cards, and incompatible.) In theory it has really high performance, and the CPU doesn't have to be involved in whatever operations happen on the card, but the catch is that the CPU has to bundle up the data to be processed and send it off to the card–and there's quite a bit of overhead involved in doing that. The optimal use case is that you have a whole bunch of data bundled up, you send it off to quickassist in one operation, you go do something else, you come back and the maybe goes almost directly to the NIC to get broken up into packets and sent. If the package of data is big enough and you can do something else in the meantime, then you can get good performance and the CPU can be busy doing other things. Now, VPN: you get a little tiny packet, you set it up, send it off to quickassist, and then you have to wait for it to be done before much else happens. The CPU is waiting for quickassist, and the time it takes to send the packet off to quickassist is as much time as it would take to do the work! For this use case, a $700 quickassist add-in card performs worse than a sub $100 skylake. What's more, the skylake does AES-GCM roughly 5 times as fast as AES-CBC+SHA1 and as far as I know the quickassist doesn't do GCM at all. And goldmont and the upcoming cannonlake add SHA instructions to the existing AES-NI instructions on the CPU, making AES-CBC+SHA1 even faster relative to the quickassist. To be fair, the current quickassist implementations are years old, and there's a new one coming out in purley and presumably there will be a new generation of add in cards as well--and I have no idea how those will perform. Fundamentally, though, on-cpu crypto is always going to outperform off-cpu crypto when dealing with small blocks; quickassist is a huge win for servers doing SSL on streams of data, not for VPNs dealing with packets at a time.
Also, unrelated, but how do the SoC Intel i-series NICs compare to the PCIe versions? any difference?
not as far as I know. Both are PCIe as far as the software is concerned, the one on the SOC just happens to live in the same package and is connected via a special bus. The i354 in avoton/rangeley is actually 4x2.5Gbps rather than 4x1Gbps, but that doesn't matter when you're attaching it to a gigabit ethernet PHY.
-
Thank you so much! Your descriptions are very detailed and easy to follow, I really appreciate you taking your time to explain this stuff to me.
-
Thank you both so much, your discussion has helped guide me into the things that I should be looking for in my setup!
psSense have replied to say that they don't have any products that would meet my requirements so a self build seems to be my only option…I asked what people were using on my VPN providers forum and had this interesting piece of information...
'When I benchmarked my CPUs for OpenVPN with AirVPN parameters I was getting from 120Mb/s for a Intel N3105 to ~350MB/s for a i5 2500.
A modern fast i3 Xeon or otherwise might do 500Mb/s but I would check.'
-
I'm currently using an old i5-2400, it's enormous overkill for my needs but it got me started into pfsense. I'll be replacing in the future with either one of these new Atom boards or an apollo lake board if the atoms don't pan out within a reasonable amount of time.
My system draws about 40W under normal load and will bump up maybe a few watts with a lot of use and down a few at idle. But 40W is a lot to pay for a 24/7 system, depending on where you live that's somewhere between $40-100/yr to run an appliance.
With the apollo lake boards running between $55-100 for a one time purchase it's a pretty easy choice.It seems reasonable to me that by eliminating all moving parts (no fans or hdd) and switching to a much lower power board that I could get power draw down to an average of 15W or less. That comes out to $15-40/yr, and that's still a hog compared to something like the SG-1000.
So basically I would just recommend that you shop around to find the combo that gets your job done and meets the best compromise between initial buy in and cost over time.
What VAMike explained about how different architectures handle AES-NI drastically differently will have a lot of impact on parts for your use case.
If an i5-2500 can put out the VPN throughput you need then maybe an Celeron G3930T would do the same for 60W less TDP? Not saying it will but just that a modern CPU will probably get you way more bang for the buck over time.
I bought an old one not knowing a thing about pfsense or what to expect and now I'll just be buying another part. -
I've found this which I think will far exceed my requirements but good for future proofing..?
http://www.ebay.co.uk/itm/182347604580
Let me have your opinions please…
-
I've found this which I think will far exceed my requirements but good for future proofing..?
http://www.ebay.co.uk/itm/182347604580
Let me have your opinions please…
Overall it doesn't seem great for a pfsense box. It's pretty expensive, I don't see anywhere what kind of NICs or Wifi card it has, so they might not work well with pfsense, and if that's the case you're kind of screwed. The CPU is probably a great compromise between power draw and performance, but I don't know how great it would be at VPN. I don't know if OpenVPN on pfsense can utilize multiple actual cores, but I'm almost positive it can't do anything with hyperthreading. If that's the case then for VPN you effectively have a dual core CPU clocked 20% higher than a $55 J3355 Celeron, but with older AES-NI than the celeron, so it might be a draw, or maybe even worse at VPN.
Also if you're looking for wifi it might be better to get something like a ubiquiti AP, or reuse an old router as an AP than use an integrated wifi card.I'd like to be able to buy a fanless small device that can handle a 150/10Mbps VPN connection plus more to future proof myself, maybe something that can handle 300/100Mbps..?
'When I benchmarked my CPUs for OpenVPN with AirVPN parameters I was getting from 120Mb/s for a Intel N3105 to ~350MB/s for a i5 2500.
I don't know how old that benchmark is or what kind of VPN they were using or encryption level, but if the N3150 was getting 120Mb/s then based on what VAMike pointed out about the AES improvements to Goldmont the Apollo Lake lineup should be noticeably faster. Maybe not 300Mbps, but at $55 for the motherboard and CPU you could just upgrade in the future when it can't keep up anymore.
Throw some LP RAM, a little innodisk SLC sata-dom, an energy efficient PSU or pico-PSU, and an ebay i350 in with it and you have a very solid set of components.When the day comes that the CPU can't meet your needs anymore, there will probably be a new cheap/low power SoC with a little higher clock and a better set of AES-NI that you can swap in.