Help with hardware build
-
I don't know much of anything about cooling hardware. Noctua is from what I know a great brand, but they can be expensive. All I can say is that in any application if you need X amount of airflow you can spin something big slower or you can spin something small faster to achieve X airflow, the big thing spinning slowly will always be substantially quieter.
So putting a little fan directly on your cpu cooler will result in higher dB than putting a huge case fan in the side of your box blowing over the whole CPU.If you get a case that has one whole side of it vented, then get a really big ass fan that covers as much of that vented surface area as possible and plug the fan into the CPU fan controller, it will probably work very well for you while being inaudible more than a few feet away.
For example, I use a 230mm fan in the side of my desktop, and can barely hear it when the case is open and I'm looking at it inches from my face because it operates at very low RPM.
https://smile.amazon.com/gp/product/B008UYZ102/ref=oh_aui_search_detailpage?ie=UTF8&psc=1Something like this will get you great cooling as it can fit a 200mm fan and a 140mm cooler, but it's not small.
https://www.hardocp.com/article/2014/08/15/thermaltake_core_v1_miniitx_case_review/7
https://www.amazon.com/Thermaltake-Core-Gaming-Computer-CA-1B8-00S1WN-00/dp/B00M2UKGSM?psc=1&SubscriptionId=AKIAIS7SSXKLFPKG5TPA&tag=11018812-20&linkCode=xm2&camp=2025&creative=165953&creativeASIN=B00M2UKGSM
I wouldn't expect your CPU to thermally throttle using something like this, but this is where you have to decide what kind of compromises you want to make.
Again, you could get a really small case that fits a water cooler, but that just seems wrong for a firewall.Gigabit NAT can be done by an old low end celeron, so don't worry about that with a full blown desktop CPU. Even running VPN maxed out, VPN is single threaded so it will only max one core unless you use gateway groups.
IDK what that pentium will max out on VPN, as a total guess I would think 4-500Mbps @ AES-128-CBC?
I imagined liquid cooling in that case for a router and laughed a bit :D
Would one of the other CPUs you suggested allow for passive cooling, the various packages we talked about (DarkStats, the DNS tools) and hit some lower OpenVPN numbers? I am trying to get a picture in my head of what I would "lose" if I went with a smaller form factor and zero noise. It sounds like the biggest CPU hogs would be VPN and any IDS/IPS packages. If I can accomplish all of my main goals, hit semi-decent OpenVPN numbers the times I do use it, and keep a smaller form factor with no noise, that may be a better solution.
Sleeping on the OpenVPN usage, I suspect I will not use PIA on the router itself. Most of my devices don't need to be routed through a VPN. I also think my work wouldn't like my traffic being sent through a third party VPN. I might rarely use the router as an OpenVPN server to get some of the adblocking benefits of the device, but that will probably be it.
Sorry for being so wishy washy on features. Talking through the cost/benefit with you has been extremely helpful and probably saved me a ton of time and money! Here is another updated list of needs. I think the CPU you mentioned in your benchmark post will hit my requirements, and I think it is passively cooled or at least extremely quiet.
Changes are bolded
-
250/25 internet speeds now
-
Going to have gigabit internet Soon
-
Very rarely, if ever will use PIA on the pfSense box. So I will not need maximum throughput
-
Low chance for OpenVPN server on the box, only need ~100mbps throughput
-
Some basic firewalling on the pfSense
-
I'd prefer a small box, but am open to larger
-
Biggest goal will be something that is either silent or has minimal noise
-
Currently only three devices plugged directly into my router (two PCs and my Pi-Hole)
-
Will leverage my current wireless router in bridge mode to give WiFi to my house
-
Packages I will install: Darkstat, Unbound, pfBlockerNG, DNSBL and maybe other cool ones that I find
Here are some must haves, and why I am annoyed with my current router. This might help get to clearer requirements…
-
Assign static IPs to devices via the router. I really like knowing what is doing what on my network, and having the same IP makes it WAY easier
-
View traffic data since I love graphs
-
Block ads and malicious domains (currently done on Pi-Hole device)
-
Maybe setup a local proxy to fix some issues that PS4 has
-
-
@teh:
Yup, Pi-Hole is a DNS level ad-blocker.
I'd stick with pi-hole on the network regardless of what router you use. And I may be in the minority here but I find Unbound lacking as a local DNS server (as a forwarder or recursive resolver it's ok) and use a couple of low power BIND boxes on my network to handle the local zone, then they forward external queries to my pi-hole VM and do recursive lookups as a last resort (if the pi-hole box is down, for example).
-
@teh:
Would one of the other CPUs you suggested allow for passive cooling, the various packages we talked about (DarkStats, the DNS tools) and hit some lower OpenVPN numbers? I am trying to get a picture in my head of what I would "lose" if I went with a smaller form factor and zero noise. It sounds like the biggest CPU hogs would be VPN and any IDS/IPS packages. If I can accomplish all of my main goals, hit semi-decent OpenVPN numbers the times I do use it, and keep a smaller form factor with no noise, that may be a better solution.
Sleeping on the OpenVPN usage, I suspect I will not use PIA on the router itself. Most of my devices don't need to be routed through a VPN.
Sorry for being so wishy washy on features. Talking through the cost/benefit with you has been extremely helpful and probably saved me a ton of time and money! Here is another updated list of needs. I think the CPU you mentioned in your benchmark post will hit my requirements, and I think it is passively cooled or at least extremely quiet.
No worries at all, I enjoy the discussion and learn from it!
It sounds like ultimately you value a small box that you can setup, forget about and get respectable performance from. Dropping the IDS and offloading the majority of the VPN usage to the individual devices is a game changer. If that's in line with your priorities, then yes I think the J3355B will be your best bet. It will NAT gigabit and give you respectable VPN throughput when you want it. It will let you play with packages if you want at the expense of overall performance.
The big advantages for you being that if you pair it with an SSD, and a picoPSU you can put it in a very small case and it will make literally no noise. It's also very cheap, the SoC + a used i340-t4 will run you ~$90.I'd stick with pi-hole on the network regardless of what router you use.
You already have the hardware and it works, this will offload work from your router by pretty much replacing pfBNG & DNSBL.
-
No worries at all, I enjoy the discussion and learn from it!
It sounds like ultimately you value a small box that you can setup, forget about and get respectable performance from. Dropping the IDS and offloading the majority of the VPN usage to the individual devices is a game changer. If that's in line with your priorities, then yes I think the J3355B will be your best bet. It will NAT gigabit and give you respectable VPN throughput when you want it. It will let you play with packages if you want at the expense of overall performance.
The big advantages for you being that if you pair it with an SSD, and a picoPSU you can put it in a very small case and it will make literally no noise. It's also very cheap, the SoC + a used i340-t4 will run you ~$90I agree, thinking on it, the smaller silent package is probably the highest priority. The VPN is lower down on my priority list, even compared to random packages that do cool things.
Is there a rough idea of what packages would cause problems with performance?
You already have the hardware and it works, this will offload work from your router by pretty much replacing pfBNG & DNSBL.
The Pi-Hole doesn't do a ton. It is basically a DNS server that has some black lists in place. The kind of cool part is the extra graphs and reporting data for seeing what domains are getting hit. I suspect I'll still use the pfSense box as the DHCP server. I assume that doesn't have a ton of overhead.
-
The heavy hitters are IDS/IPS & VPN. I put some load on my network that would make the main packages I run to include pfBNG work and took a screenshot of the top output. Not a definitive answer, but you can see that suricata has a ton of CPU time, OpenVPN instances (I have two clients and a server) are next, then pfBNG.
Squid might be resource intensive but I doubt it. I tried using it when i first got pfSense. But it was a huge PITA and on a home network I saw no performance increase whatsoever. pfBNG & DNSBL also do everything I wanted out of squidguard better, and without the ass pain.
-
@teh:
The Pi-Hole doesn't do a ton. It is basically a DNS server that has some black lists in place. The kind of cool part is the extra graphs and reporting data for seeing what domains are getting hit. I suspect I'll still use the pfSense box as the DHCP server. I assume that doesn't have a ton of overhead.
Yeah, sorry for not clarifying. Use pfSense for DHCP and pi-hole just for ad blocking, if you're so inclined.
-
The heavy hitters are IDS/IPS & VPN. I put some load on my network that would make the main packages I run to include pfBNG work and took a screenshot of the top output. Not a definitive answer, but you can see that suricata has a ton of CPU time, OpenVPN instances (I have two clients and a server) are next, then pfBNG.
Squid might be resource intensive but I doubt it. I tried using it when i first got pfSense. But it was a huge PITA and on a home network I saw no performance increase whatsoever. pfBNG & DNSBL also do everything I wanted out of squidguard better, and without the ass pain.
Thanks for all the info. I'll have to see what I can put together. I'll make a summary once I order my parts.
-
No worries, I hope it was helpful. I look forward to hearing what you decide on and how it performs for you, especially once you get a gigabit connection!
-
No worries, I hope it was helpful. I look forward to hearing what you decide on and how it performs for you, especially once you get a gigabit connection!
One more question. Will there be a significant different between the J3355B and the J3455? It sounds like the lower clock speed on the J3455 might impact VPN performance to an extent, but would the additional cores allow for more "playing" with packages?
EDIT: Also, thoughts on the newer chips in the Celeron line? They don't appear to be that much "better" from what I am seeing.
Can you review these notes I put together? I tried to consolidate as much as I could down.
Important Details
-
If using VPN, how much throughput do you want?
-
What packages do you plan on running?
-
Passively cooled, or actively cooled?
Pieces of Info
-
Just NAT on gigabit with light, occasional VPN usage can be done with a passively cooled Celeron
-
Higher VPN speeds require more CPU power, and eventually active cooling
-
Most likely will not see gigabit throughput on VPN
Packages
-
DarkStat has little noticeable impact on CPU
-
SquidGuard is cool, but a bit of a PITA to setup. Minimal noticeable improvements in most home use scenarios. You also need to use MiTM techniques if you have a lot of HTTPS traffic. There should be little impact on CPU with SquidGuard
-
Any kind of IDS/IPS will have a significant impact. Higher speeds directly correlate to higher CPU needs.
-
Suricata is multithreaded while Snort is single threaded.
-
Unbound for DNS resolution, combined with pfBlockerNG & DNSBL will allow for DNS filtering (blocking ads, etc)
Hardware
-
NICs – i340-T4, more power efficient than the PRO/1000's and more affordable than the i350s.
-
i3-7100 – Won't do any serious IDS/IPS at gigabit speeds, and won't hit gigabit VPN. Should definitely hit 250 Mbps on VPN easily. Offers a good compromise between performance and cost.
-
C2758 – Caps out at ~218 Mbps UDP AES-128. Has also run into some issues
-
G4620 – Can do gigabit NAT, ~250 Mbps OpenVPN, and handle a good number of packages. Would need a larger case with good ventilation. A large heat sink and large case fan should do the trick for a quiet package.
-
J3355B and J3455 are the two most recommended for home use cases. They are cheap, modern, and passively cooled SoCs.
-
J3355B – Two cores at a higher clock compared to the J3455. Maxed out a 150 Mbps line at AES-128-CBC with about 33% CPU usage, see here. Should do gigabit NAT, and it will hit 250 Mbps OpenVPN AE-128 (not at the same time).
-
J3455 – More powerful than the J3355B overall with four cores that are vlocked lower. Requires either physically modifying the NIC/motherboard, or buying a Micro-ATX board to make it work.
-
-
-
I'm also curious what the official hardware is capable of compared to these builds cost / functionality wise.
-
@teh:
One more question. Will there be a significant different between the J3355B and the J3455? It sounds like the lower clock speed on the J3455 might impact VPN performance to an extent, but would the additional cores allow for more "playing" with packages?
If I was in your shoes I would get a J3455 ITX board and carefully remove the back wall of the PCIe port, knowing full well that if I screwed it up I would have to buy a new board.
If you are comfortable with voiding the warranty and modifying either the board or the card, or buying the micro-ATX size board then yes I would say J3455 will be better for you as it will give you more headroom to play with packages and figure out what you do and don’t want to use.
Another thing to keep in mind is that even though OpenVPN is single threaded, you can add multiple clients and create a gateway group. If you create one client per CPU core you can take advantage of multithreading and see more throughput.
It isn’t a perfect solution though. Not all kinds of traffic can use multiple threads so a lot of things you do will still cap out at your single core max, but IMO it’s worth configuring as you will see practical improvements to performance.@teh:
EDIT: Also, thoughts on the newer chips in the Celeron line? They don't appear to be that much "better" from what I am seeing.
You can’t just determine which CPUs are better based on clock speeds and cores.
Passmark isn’t the best benchmark in the world but it’s common and gives you a general idea of CPU horsepower so I’ll use it here.
J1900 @ 4 cores 2.0/2.4GHz = 1868MT/534ST
J3455 @ 4 cores 1.5/2.3GHz = 2128MT/783MT
For a more extreme example:
i7-980X @ 6HT cores 3.3/3.6GHz = 8902MT/1453ST (MSRP $1059 @ 2010)
i7-6700T @ 4HT cores 2.8/3.6GHz = 9051MT/1970ST (MSRP $303 @ 2015)Just looking at cores/clocks you would think that the older CPU’s in these cases would be more powerful but the new CPUs are completely new architectures so the specs don’t translate anymore. Features also get added/updated. For example, the i7-980X was the flagship CPU the year that AES-NI came out. So on paper it’s a faster CPU with more cores and they both have AES-NI but AES-NI has been updated multiple times since 2010.
All that to say that when a new architecture comes out it may look weaker than the old stuff on paper but it probably is not.
@teh:
Packages
-
DarkStat has little noticeable impact on CPU
-
…There should be little impact on CPU with SquidGuard
-
Any kind of IDS/IPS will have a significant impact. Higher speeds directly correlate to higher CPU needs.
I’ve never used darkstat and never paid attention to CPU usage on squid/squidguard/lightsquid, so I’m totally guessing there.
IDS/IPS CPU usage also depends on the rules you are using with it. Basically there are two general variables for IDS/IPS CPU usage: How many packets does it have to process, and how much data does it have to compare the contents of each packet to?
So for a given ruleset, if you increase the bandwidth and thus the number of packets being processed yes the CPU usage will increase. But if you were to use just a few rules on your IDS/IPS you could see less CPU usage than someone with a slower connection but a large ruleset.
For a point of reference on the top output I posted my rules consist of the free Snort Connectivity & ET Open rules after disabling rules for false positives, and a couple of custom rules. I forget what my bandwidth was at the time I took that screenshot, but it wasn’t maxing out my 150/10 line because I was on an old laptop with wifi that can’t hit that but I’m guessing about 80-90Mbps.
@teh:
Hardware
-
J3355B and J3455 are the two most recommended for home use cases. They are cheap, modern, and passively cooled SoCs.
- J3355B – Two cores at a higher clock compared to the J3455. Maxed out a 150 Mbps line at AES-128-CBC with about 33% CPU usage, see here. Should do gigabit NAT, and it will hit 250 Mbps OpenVPN AE-128 (not at the same time).
The Apollo lake Celerons are only my recommended SoC’s for home use. You’ll see others who will recommend a full-blown desktop CPU or an 8 core C2758 for just about anything. Not saying they don’t have a reason for doing this, I just don’t agree with it which is why I keep trying to recommend people buy the cheapest hardware that will meet their needs and nothing else, instead of paying a few hundred $ more for something that will meet their needs at 20% CPU.
Also, I can only speak for the performance that I saw the J3355B achieve. While it seems reasonable to me that if the CPU can achieve 150Mbps @ 33% that it can achieve 300Mbps @100%, I’ve never seen it do that and that may not be true.
Anything above and beyond what I’ve actually seen it do is just my best guess.
-
-
@teh:
I'm also curious what the official hardware is capable of compared to these builds cost / functionality wise.
You can absolutely get official hardware that will meet your needs! It will also come with a year of Gold Subscription, official support, and automated backups. Those are very useful things especially if you are looking to really learn pfSense.
In general, you will pay more for less with official hardware than DIY but this is true in just about any market.
You are paying for a guarantee that it will work out of the box and all of the above mentioned features, so if those have value to you then by all means, the pfSense store has excellent products!If you do decide for DIY over official, you can still get a Gold Subscription for $99/yr. you'll get access to the updated pfSense book and all of their monthly hangout videos. Very cool stuff, probably worth checking out even if you only do it for the first year to learn the product!
-
You can absolutely get official hardware that will meet your needs! It will also come with a year of Gold Subscription, official support, and automated backups. Those are very useful things especially if you are looking to really learn pfSense.
In general, you will pay more for less with official hardware than DIY but this is true in just about any market.
You are paying for a guarantee that it will work out of the box and all of the above mentioned features, so if those have value to you then by all means, the pfSense store has excellent products!If you do decide for DIY over official, you can still get a Gold Subscription for $99/yr. you'll get access to the updated pfSense book and all of their monthly hangout videos. Very cool stuff, probably worth checking out even if you only do it for the first year to learn the product!
Heck, if they are half as helpful as you have been, it is well worth the $99…
Thanks for all the info. The DIY option seems like it will be the best. Now to find a nice case and triple check which parts I want.
-
@teh:
Heck, if they are half as helpful as you have been, it is well worth the $99…
Thanks for all the info. The DIY option seems like it will be the best. Now to find a nice case and triple check which parts I want.
haha thanks! I'm glad I could be helpful.
I promise you they are WAY better than me!
-
If I was in your shoes I would get a J3455 ITX board and carefully remove the back wall of the PCIe port, knowing full well that if I screwed it up I would have to buy a new board.
If you are comfortable with voiding the warranty and modifying either the board or the card, or buying the micro-ATX size board then yes I would say J3455 will be better for you as it will give you more headroom to play with packages and figure out what you do and don’t want to use.
Hmm, I might go to the micro-ATX. It will be a bigger case, but at least I won't have to do any modding.
-
@teh:
Hmm, I might go to the micro-ATX. It will be a bigger case, but at least I won't have to do any modding.
Definitely the better choice if you are OK with the size.
One more option I'll throw out is a PCIe riser. You can get a passive PCIe x1 to PCIe x4 riser and it will work fine. The problems with it are that it can be difficult to mount the card in your case with a riser, it just depends on the case and the riser. Also I don't know of any good quality ones to recommend.
I hesitate to even mention this option because if you get a crappy one it could cause issues, just getting the micro-ATX is the most reliable way to go. -
Definitely the better choice if you are OK with the size.
One more option I'll throw out is a PCIe riser. You can get a passive PCIe x1 to PCIe x4 riser and it will work fine. The problems with it are that it can be difficult to mount the card in your case with a riser, it just depends on the case and the riser. Also I don't know of any good quality ones to recommend.
I hesitate to even mention this option because if you get a crappy one it could cause issues, just getting the micro-ATX is the most reliable way to go.Size isn't a huge issue for me. It makes it so I can be lazier about fitting cables :D .
I found a few Intel I350 NICs on eBay, is that the normal place to get them? They seem too cheap and are "refurbished". They do claim to be Intel chips from an OEM…
-
I personally would go for an i340 server pull over a refurb i350 (unless you need SR-IOV or Ethernet Power Management).
That's really just a matter of opinion though, I've read about plenty of people using obvious knockoff i340/i350's with great results. I've bought plenty of things used and refurbished and I've never had an issue.
One thing to watch out for on i350's is that it's an i350v2. Apparently there was some sort of power spike issue on the original and they discontinued it. I don't know how serious the problem is for home use though, probably negligible but I just don't know.
-
I personally would go for an i340 server pull over a refurb i350 (unless you need SR-IOV or Ethernet Power Management).
That's really just a matter of opinion though, I've read about plenty of people using obvious knockoff i340/i350's with great results. I've bought plenty of things used and refurbished and I've never had an issue.
One thing to watch out for on i350's is that it's an i350v2. Apparently there was some sort of power spike issue on the original and they discontinued it. I don't know how serious the problem is for home use though, probably negligible but I just don't know.
I'll have to poke around at the eBay deals. They definitely seem a bit sketchy…. Would any Intel PCIe NIC work?
I was thinking of getting two SSDs for a mirrored ZFS array. Probably overkill, but would help if a disk dies. This means I can't get one of those picoPSUs, since they appear to only have one SATA power cable. Any recommendations for a decent PSU? I assume the lower the wattage, the better, since it starts hurting efficiency at the wattages I am sitting at.
-
Yeah, intel NICs are solid. The three main PCIe NICs for gigabit are in PRO/1000, i340 and i350.
In the quad port configuration at least, the PRO/1000 can consume more power than a J3455, almost three times as much as an i340. i340 also supports virtualization if you ever go that route. Finally, the PRO/1000 is PCIe v 1.0, so if you want to fully utilize a quad port unit you must have a slot at x4 speeds.
For all of those reasons I recommend searching around for a good used i340-t4. You can find them fairly regularly in the $35-$40 range, the best I've seen is I think $25.I personally would say that SSD's in a mirror for home use is totally unnecessary. You will almost certainly not see your SSD fail in the lifetime of the firewall. In the event that it does fail, so long as you have a config.xml backed up, you can reinstall to just about any thumb drive you have lying around and restore your config file. Your machine would be back up in minutes and then you could order a replacement SSD.
However, if you are more comfortable with SSDs in a mirror, then you can still keep the picoPSU, just use a SATA splitter or MOLEX to SATA cable.
https://www.newegg.com/Product/Product.aspx?Item=N82E16812119010
https://www.amazon.com/StarTech-com-Power-Splitter-Adapter-PYO4SATA/dp/B0086OGN9E