Help with hardware build
-
No worries at all, I enjoy the discussion and learn from it!
It sounds like ultimately you value a small box that you can setup, forget about and get respectable performance from. Dropping the IDS and offloading the majority of the VPN usage to the individual devices is a game changer. If that's in line with your priorities, then yes I think the J3355B will be your best bet. It will NAT gigabit and give you respectable VPN throughput when you want it. It will let you play with packages if you want at the expense of overall performance.
The big advantages for you being that if you pair it with an SSD, and a picoPSU you can put it in a very small case and it will make literally no noise. It's also very cheap, the SoC + a used i340-t4 will run you ~$90I agree, thinking on it, the smaller silent package is probably the highest priority. The VPN is lower down on my priority list, even compared to random packages that do cool things.
Is there a rough idea of what packages would cause problems with performance?
You already have the hardware and it works, this will offload work from your router by pretty much replacing pfBNG & DNSBL.
The Pi-Hole doesn't do a ton. It is basically a DNS server that has some black lists in place. The kind of cool part is the extra graphs and reporting data for seeing what domains are getting hit. I suspect I'll still use the pfSense box as the DHCP server. I assume that doesn't have a ton of overhead.
-
The heavy hitters are IDS/IPS & VPN. I put some load on my network that would make the main packages I run to include pfBNG work and took a screenshot of the top output. Not a definitive answer, but you can see that suricata has a ton of CPU time, OpenVPN instances (I have two clients and a server) are next, then pfBNG.
Squid might be resource intensive but I doubt it. I tried using it when i first got pfSense. But it was a huge PITA and on a home network I saw no performance increase whatsoever. pfBNG & DNSBL also do everything I wanted out of squidguard better, and without the ass pain.
-
@teh:
The Pi-Hole doesn't do a ton. It is basically a DNS server that has some black lists in place. The kind of cool part is the extra graphs and reporting data for seeing what domains are getting hit. I suspect I'll still use the pfSense box as the DHCP server. I assume that doesn't have a ton of overhead.
Yeah, sorry for not clarifying. Use pfSense for DHCP and pi-hole just for ad blocking, if you're so inclined.
-
The heavy hitters are IDS/IPS & VPN. I put some load on my network that would make the main packages I run to include pfBNG work and took a screenshot of the top output. Not a definitive answer, but you can see that suricata has a ton of CPU time, OpenVPN instances (I have two clients and a server) are next, then pfBNG.
Squid might be resource intensive but I doubt it. I tried using it when i first got pfSense. But it was a huge PITA and on a home network I saw no performance increase whatsoever. pfBNG & DNSBL also do everything I wanted out of squidguard better, and without the ass pain.
Thanks for all the info. I'll have to see what I can put together. I'll make a summary once I order my parts.
-
No worries, I hope it was helpful. I look forward to hearing what you decide on and how it performs for you, especially once you get a gigabit connection!
-
No worries, I hope it was helpful. I look forward to hearing what you decide on and how it performs for you, especially once you get a gigabit connection!
One more question. Will there be a significant different between the J3355B and the J3455? It sounds like the lower clock speed on the J3455 might impact VPN performance to an extent, but would the additional cores allow for more "playing" with packages?
EDIT: Also, thoughts on the newer chips in the Celeron line? They don't appear to be that much "better" from what I am seeing.
Can you review these notes I put together? I tried to consolidate as much as I could down.
Important Details
-
If using VPN, how much throughput do you want?
-
What packages do you plan on running?
-
Passively cooled, or actively cooled?
Pieces of Info
-
Just NAT on gigabit with light, occasional VPN usage can be done with a passively cooled Celeron
-
Higher VPN speeds require more CPU power, and eventually active cooling
-
Most likely will not see gigabit throughput on VPN
Packages
-
DarkStat has little noticeable impact on CPU
-
SquidGuard is cool, but a bit of a PITA to setup. Minimal noticeable improvements in most home use scenarios. You also need to use MiTM techniques if you have a lot of HTTPS traffic. There should be little impact on CPU with SquidGuard
-
Any kind of IDS/IPS will have a significant impact. Higher speeds directly correlate to higher CPU needs.
-
Suricata is multithreaded while Snort is single threaded.
-
Unbound for DNS resolution, combined with pfBlockerNG & DNSBL will allow for DNS filtering (blocking ads, etc)
Hardware
-
NICs – i340-T4, more power efficient than the PRO/1000's and more affordable than the i350s.
-
i3-7100 – Won't do any serious IDS/IPS at gigabit speeds, and won't hit gigabit VPN. Should definitely hit 250 Mbps on VPN easily. Offers a good compromise between performance and cost.
-
C2758 – Caps out at ~218 Mbps UDP AES-128. Has also run into some issues
-
G4620 – Can do gigabit NAT, ~250 Mbps OpenVPN, and handle a good number of packages. Would need a larger case with good ventilation. A large heat sink and large case fan should do the trick for a quiet package.
-
J3355B and J3455 are the two most recommended for home use cases. They are cheap, modern, and passively cooled SoCs.
-
J3355B – Two cores at a higher clock compared to the J3455. Maxed out a 150 Mbps line at AES-128-CBC with about 33% CPU usage, see here. Should do gigabit NAT, and it will hit 250 Mbps OpenVPN AE-128 (not at the same time).
-
J3455 – More powerful than the J3355B overall with four cores that are vlocked lower. Requires either physically modifying the NIC/motherboard, or buying a Micro-ATX board to make it work.
-
-
-
I'm also curious what the official hardware is capable of compared to these builds cost / functionality wise.
-
@teh:
One more question. Will there be a significant different between the J3355B and the J3455? It sounds like the lower clock speed on the J3455 might impact VPN performance to an extent, but would the additional cores allow for more "playing" with packages?
If I was in your shoes I would get a J3455 ITX board and carefully remove the back wall of the PCIe port, knowing full well that if I screwed it up I would have to buy a new board.
If you are comfortable with voiding the warranty and modifying either the board or the card, or buying the micro-ATX size board then yes I would say J3455 will be better for you as it will give you more headroom to play with packages and figure out what you do and don’t want to use.
Another thing to keep in mind is that even though OpenVPN is single threaded, you can add multiple clients and create a gateway group. If you create one client per CPU core you can take advantage of multithreading and see more throughput.
It isn’t a perfect solution though. Not all kinds of traffic can use multiple threads so a lot of things you do will still cap out at your single core max, but IMO it’s worth configuring as you will see practical improvements to performance.@teh:
EDIT: Also, thoughts on the newer chips in the Celeron line? They don't appear to be that much "better" from what I am seeing.
You can’t just determine which CPUs are better based on clock speeds and cores.
Passmark isn’t the best benchmark in the world but it’s common and gives you a general idea of CPU horsepower so I’ll use it here.
J1900 @ 4 cores 2.0/2.4GHz = 1868MT/534ST
J3455 @ 4 cores 1.5/2.3GHz = 2128MT/783MT
For a more extreme example:
i7-980X @ 6HT cores 3.3/3.6GHz = 8902MT/1453ST (MSRP $1059 @ 2010)
i7-6700T @ 4HT cores 2.8/3.6GHz = 9051MT/1970ST (MSRP $303 @ 2015)Just looking at cores/clocks you would think that the older CPU’s in these cases would be more powerful but the new CPUs are completely new architectures so the specs don’t translate anymore. Features also get added/updated. For example, the i7-980X was the flagship CPU the year that AES-NI came out. So on paper it’s a faster CPU with more cores and they both have AES-NI but AES-NI has been updated multiple times since 2010.
All that to say that when a new architecture comes out it may look weaker than the old stuff on paper but it probably is not.
@teh:
Packages
-
DarkStat has little noticeable impact on CPU
-
…There should be little impact on CPU with SquidGuard
-
Any kind of IDS/IPS will have a significant impact. Higher speeds directly correlate to higher CPU needs.
I’ve never used darkstat and never paid attention to CPU usage on squid/squidguard/lightsquid, so I’m totally guessing there.
IDS/IPS CPU usage also depends on the rules you are using with it. Basically there are two general variables for IDS/IPS CPU usage: How many packets does it have to process, and how much data does it have to compare the contents of each packet to?
So for a given ruleset, if you increase the bandwidth and thus the number of packets being processed yes the CPU usage will increase. But if you were to use just a few rules on your IDS/IPS you could see less CPU usage than someone with a slower connection but a large ruleset.
For a point of reference on the top output I posted my rules consist of the free Snort Connectivity & ET Open rules after disabling rules for false positives, and a couple of custom rules. I forget what my bandwidth was at the time I took that screenshot, but it wasn’t maxing out my 150/10 line because I was on an old laptop with wifi that can’t hit that but I’m guessing about 80-90Mbps.
@teh:
Hardware
-
J3355B and J3455 are the two most recommended for home use cases. They are cheap, modern, and passively cooled SoCs.
- J3355B – Two cores at a higher clock compared to the J3455. Maxed out a 150 Mbps line at AES-128-CBC with about 33% CPU usage, see here. Should do gigabit NAT, and it will hit 250 Mbps OpenVPN AE-128 (not at the same time).
The Apollo lake Celerons are only my recommended SoC’s for home use. You’ll see others who will recommend a full-blown desktop CPU or an 8 core C2758 for just about anything. Not saying they don’t have a reason for doing this, I just don’t agree with it which is why I keep trying to recommend people buy the cheapest hardware that will meet their needs and nothing else, instead of paying a few hundred $ more for something that will meet their needs at 20% CPU.
Also, I can only speak for the performance that I saw the J3355B achieve. While it seems reasonable to me that if the CPU can achieve 150Mbps @ 33% that it can achieve 300Mbps @100%, I’ve never seen it do that and that may not be true.
Anything above and beyond what I’ve actually seen it do is just my best guess.
-
-
@teh:
I'm also curious what the official hardware is capable of compared to these builds cost / functionality wise.
You can absolutely get official hardware that will meet your needs! It will also come with a year of Gold Subscription, official support, and automated backups. Those are very useful things especially if you are looking to really learn pfSense.
In general, you will pay more for less with official hardware than DIY but this is true in just about any market.
You are paying for a guarantee that it will work out of the box and all of the above mentioned features, so if those have value to you then by all means, the pfSense store has excellent products!If you do decide for DIY over official, you can still get a Gold Subscription for $99/yr. you'll get access to the updated pfSense book and all of their monthly hangout videos. Very cool stuff, probably worth checking out even if you only do it for the first year to learn the product!
-
You can absolutely get official hardware that will meet your needs! It will also come with a year of Gold Subscription, official support, and automated backups. Those are very useful things especially if you are looking to really learn pfSense.
In general, you will pay more for less with official hardware than DIY but this is true in just about any market.
You are paying for a guarantee that it will work out of the box and all of the above mentioned features, so if those have value to you then by all means, the pfSense store has excellent products!If you do decide for DIY over official, you can still get a Gold Subscription for $99/yr. you'll get access to the updated pfSense book and all of their monthly hangout videos. Very cool stuff, probably worth checking out even if you only do it for the first year to learn the product!
Heck, if they are half as helpful as you have been, it is well worth the $99…
Thanks for all the info. The DIY option seems like it will be the best. Now to find a nice case and triple check which parts I want.
-
@teh:
Heck, if they are half as helpful as you have been, it is well worth the $99…
Thanks for all the info. The DIY option seems like it will be the best. Now to find a nice case and triple check which parts I want.
haha thanks! I'm glad I could be helpful.
I promise you they are WAY better than me!
-
If I was in your shoes I would get a J3455 ITX board and carefully remove the back wall of the PCIe port, knowing full well that if I screwed it up I would have to buy a new board.
If you are comfortable with voiding the warranty and modifying either the board or the card, or buying the micro-ATX size board then yes I would say J3455 will be better for you as it will give you more headroom to play with packages and figure out what you do and don’t want to use.
Hmm, I might go to the micro-ATX. It will be a bigger case, but at least I won't have to do any modding.
-
@teh:
Hmm, I might go to the micro-ATX. It will be a bigger case, but at least I won't have to do any modding.
Definitely the better choice if you are OK with the size.
One more option I'll throw out is a PCIe riser. You can get a passive PCIe x1 to PCIe x4 riser and it will work fine. The problems with it are that it can be difficult to mount the card in your case with a riser, it just depends on the case and the riser. Also I don't know of any good quality ones to recommend.
I hesitate to even mention this option because if you get a crappy one it could cause issues, just getting the micro-ATX is the most reliable way to go. -
Definitely the better choice if you are OK with the size.
One more option I'll throw out is a PCIe riser. You can get a passive PCIe x1 to PCIe x4 riser and it will work fine. The problems with it are that it can be difficult to mount the card in your case with a riser, it just depends on the case and the riser. Also I don't know of any good quality ones to recommend.
I hesitate to even mention this option because if you get a crappy one it could cause issues, just getting the micro-ATX is the most reliable way to go.Size isn't a huge issue for me. It makes it so I can be lazier about fitting cables :D .
I found a few Intel I350 NICs on eBay, is that the normal place to get them? They seem too cheap and are "refurbished". They do claim to be Intel chips from an OEM…
-
I personally would go for an i340 server pull over a refurb i350 (unless you need SR-IOV or Ethernet Power Management).
That's really just a matter of opinion though, I've read about plenty of people using obvious knockoff i340/i350's with great results. I've bought plenty of things used and refurbished and I've never had an issue.
One thing to watch out for on i350's is that it's an i350v2. Apparently there was some sort of power spike issue on the original and they discontinued it. I don't know how serious the problem is for home use though, probably negligible but I just don't know.
-
I personally would go for an i340 server pull over a refurb i350 (unless you need SR-IOV or Ethernet Power Management).
That's really just a matter of opinion though, I've read about plenty of people using obvious knockoff i340/i350's with great results. I've bought plenty of things used and refurbished and I've never had an issue.
One thing to watch out for on i350's is that it's an i350v2. Apparently there was some sort of power spike issue on the original and they discontinued it. I don't know how serious the problem is for home use though, probably negligible but I just don't know.
I'll have to poke around at the eBay deals. They definitely seem a bit sketchy…. Would any Intel PCIe NIC work?
I was thinking of getting two SSDs for a mirrored ZFS array. Probably overkill, but would help if a disk dies. This means I can't get one of those picoPSUs, since they appear to only have one SATA power cable. Any recommendations for a decent PSU? I assume the lower the wattage, the better, since it starts hurting efficiency at the wattages I am sitting at.
-
Yeah, intel NICs are solid. The three main PCIe NICs for gigabit are in PRO/1000, i340 and i350.
In the quad port configuration at least, the PRO/1000 can consume more power than a J3455, almost three times as much as an i340. i340 also supports virtualization if you ever go that route. Finally, the PRO/1000 is PCIe v 1.0, so if you want to fully utilize a quad port unit you must have a slot at x4 speeds.
For all of those reasons I recommend searching around for a good used i340-t4. You can find them fairly regularly in the $35-$40 range, the best I've seen is I think $25.I personally would say that SSD's in a mirror for home use is totally unnecessary. You will almost certainly not see your SSD fail in the lifetime of the firewall. In the event that it does fail, so long as you have a config.xml backed up, you can reinstall to just about any thumb drive you have lying around and restore your config file. Your machine would be back up in minutes and then you could order a replacement SSD.
However, if you are more comfortable with SSDs in a mirror, then you can still keep the picoPSU, just use a SATA splitter or MOLEX to SATA cable.
https://www.newegg.com/Product/Product.aspx?Item=N82E16812119010
https://www.amazon.com/StarTech-com-Power-Splitter-Adapter-PYO4SATA/dp/B0086OGN9E -
Thanks again for all the info and help pfBasic. You rock!
I haven't hit order just yet, but I settled on these parts:
CPU/Mobo: ASUS Intel Celeron Quad-Core SoC fanless MicroATX Motherboard (J3455M-E)
RAM: Corsair Vengeance 8 GB (2 x 4 GB) DDR3 1600 MHz PC3 12800 240-Pin DDR3 Dual Channel Memory Kit 1.5V
SSD: Kingston Digital 120GB SSDNow V300 SATA 3 2.5 (7mm height) Desktop Bundle Kit with Adapter Solid State Drive SV300S3D7/120G
Case: Thermaltake CORE V21 Black Extreme Micro ATX Cube Chassis CA-1D5-00S1WN-00
PSU: picoPSU-120 + 120W Adapter Power Kit
NIC: Intel i340-T4The case is a bit bigger than I originally thought, but the extra convenience of tons of room, and not having to mod the NIC or motherboard to fir in ITX is a plus. With no fans, I can also just dump that anywhere, it isn't a huge issue as long as it gets some natural airflow.
Edit: Fixed the links because BBcode is basically the worst :P
-
I think that will work out great for you! Please let us know how it goes once you get it up and running and feel free to ask any questions yo may have in the configuration process.
You can definitely use the picoPSU 80(non-WI) with 60W AC/DC Converter kit for that build, you will probably pull less than 30W from the wall under max load on everything.
-
I think that will work out great for you! Please let us know how it goes once you get it up and running and feel free to ask any questions yo may have in the configuration process.
You can definitely use the picoPSU 80(non-WI) with 60W AC/DC Converter kit for that build, you will probably pull less than 30W from the wall under max load on everything.
I just ordered my parts. Managed to get the i340-T4 for ~$35 on eBay. Not too shabby!
Any "standard" benchmarks I should run so I can share info?