Common Build Guide
-
Hi,
I have been using PfSense for a long time, when it was m0n0wall. However whenever I ask about recommended build's I just get redirected to the distro HCL.
Two Questions:
1. Are there recommended HARDWARE VERIFIED servers in the i5/i7 territory with 4xGigE? Like Dell Optiplex-666 or something?
2. If not, are there any HARDWARE VERIFIED recommended builds in the i5/i7 with 4xGigE? Willing to spend more to make it smaller, lower power, quiet, 1U or Small-er form factor preferred.I just got Fios GigE and it has blown my SG2440 out of the water. Looking to upgrade but don't want to deal with any kind of driver or kernel panic's.
Thanks!
-
… I just get redirected to the Debian HCL.
That would be a bad re-direction since pfSense is built on FreeBSD. For reference you would want to check for example:
https://www.freebsd.org/releases/11.0R/hardware.html
That would be correct for the up coming 2.4 release (or current RC snapshots).
Steve
-
… I just get redirected to the Debian HCL.
That would be a bad re-direction since pfSense is built on FreeBSD. For reference you would want to check for example:
https://www.freebsd.org/releases/11.0R/hardware.html
That would be correct for the up coming 2.4 release (or current RC snapshots).
Steve
Right FreeBSD, but the distro doesn't really matter.
All the HCL's look the same.I am looking for one level "up" from the HCL in the form of a Manufacturer model number. eg: Dell Optiplex "666 Hell Edition"
I was hoping someone from the pfSense forum would have some suggestions.
-
Just to prevent any confusion for future readers the distro here very definitely does matter because FreeBSD is not a Linux distro. It has an entirely different hardware list, different drivers etc.
Generally speaking the compatibility of hardware in FreeBSD is a significantly smaller subset of that of a typical Linux distro such as Debian.
That aside I'm sure others can suggest things they have used here. Or if you have something in mind check the forum, someone else may have tried it and posted about their experiences.
Of course our own hardware is guaranteed compatible. ;)
https://store.pfsense.org/Steve
-
There are no official brands other than Netgate. Those are supported and tested. Everything else may or may not work. That is why we have this forum.
On top of that: most 'good' hardware platforms (Intel CPU, Intel NIC, standard x86 or x86+IntelSPS, 32- and 64-bit, AES-NI) will work just fine. But they won't be verified.
Some hardware vendors list FreeBSD as a tested os. FreeBSD itself doesn't 'test' or 'verify' hardware from vendors.If you want something small: Netgate has that. If you don't have the money: Qotom has chinese boxes, but there is no support from the vendor.
-
Just to prevent any confusion for future readers the distro here very definitely does matter because FreeBSD is not a Linux distro. It has an entirely different hardware list, different drivers etc.
Generally speaking the compatibility of hardware in FreeBSD is a significantly smaller subset of that of a typical Linux distro such as Debian.
That aside I'm sure others can suggest things they have used here. Or if you have something in mind check the forum, someone else may have tried it and posted about their experiences.
Of course our own hardware is guaranteed compatible. ;)
https://store.pfsense.org/Steve
Right. I wasn't implying all the HCL's are the same, I was saying they all LOOK the same. Just chips and IC's. Never systems.
I'd love something form the official store, but I don't have an extra $3k for the Xeon. Anything less wouldn't take GigE with the plans I want from it.
:(
-
I'm actually a little surprised that the SG-2440 could not handle gigabit for you - what types of services do you run on your network that require more processing power?
I actually also have a Fios GigE connection and have been very happy with the Supermicro 5018D-F8NT 1U server, which even for what I need is still a bit overkill. In terms of compatibility, I've run both pfSense 2.3.x and now 2.4.0-RC on that box with no problems. If you need even more power than that, have a look at the Supermicro 5018D-F4NT, which has specs that are very similar to the XG-1541. Do note that both these are multi-core systems running at lower frequencies - depending on what you need, you might be better off with fewer cores operating at higher frequencies, it all depends on the intended use case.
Hope this helps.
-
I'm actually a little surprised that the SG-2440 could not handle gigabit for you - what types of services do you run on your network that require more processing power?
I actually also have a Fios GigE connection and have been very happy with the Supermicro 5018D-F8NT 1U server, which even for what I need is still a bit overkill. In terms of compatibility, I've run both pfSense 2.3.x and now 2.4.0-RC on that box with no problems. If you need even more power than that, have a look at the Supermicro 5018D-F4NT, which has specs that are very similar to the XG-1541. Do note that both these are multi-core systems running at lower frequencies - depending on what you need, you might be better off with fewer cores operating at higher frequencies, it all depends on the intended use case.
Hope this helps.
Awesome! Better. That was something I was looking for.
As for my requirements? Just a ton of VPN pretty much.
Pushing MPEG2/MPEG4 around. HDHomeRun, Plex, etc. but doing site to site OpenVPN.
-
If really site-to-site, you will move much more data around using IPsec instead if you don't need any of OpenVPNs "routed" characteristics.
-
If really site-to-site, you will move much more data around using IPsec instead if you don't need any of OpenVPNs "routed" characteristics.
Truth. That's where it gets a little hairy. Might just be many site to sites.
I'd LOVE to find a real MPEG4 encoder that just pumps out and my desired bitrate…
Anyway, the 5018D-F8NT looks awesome. I'm trying to find one with a higher core clock rate, willing to do less cores.
-
as has been said, netgate is the only official hardware for pfsense. Everything else you taking a punt, but generally intel i3/i5/i7/xeon chipsets, and intel network interfaces should be a safe bet.
Intel mobile chips do have a kernel panic issue on 2.4 but is fixed in 2.4.1. (upstream FreeBSD issue).
-
for doing gigabit OpenVPN - you cannot on a single instance, period. Even if you did have $3k to throw at it, not going to happen.
Gateway groups are currently the only way to do Gigabit over OpenVPN, that sounds like it might not work in your scenario.
I'd say try really hard to make ipsec work, otherwise just get the highest clocked modern Intel CPU you can get. If you are truly desperate for OpenVPN throughput, then overclock it but the benefits will not be linear and almost certainly not worth the price you will pay in stability.
-
for doing gigabit OpenVPN - you cannot on a single instance, period. Even if you did have $3k to throw at it, not going to happen.
Gateway groups are currently the only way to do Gigabit over OpenVPN, that sounds like it might not work in your scenario.
I'd say try really hard to make ipsec work, otherwise just get the highest clocked modern Intel CPU you can get. If you are truly desperate for OpenVPN throughput, then overclock it but the benefits will not be linear and almost certainly not worth the price you will pay in stability.
Interesting. Is there somewhere I can read more about how/why this happens?
From what you you understand, what is the OpenVPN "max"?
Thanks!
-
You can search around this OpenVPN subforum, it has been discussed quite a few times.
In short it's an OpenVPN limitation.I think about the highest single threaded speeds I've seen posted on here were in the 6xxMbps range? I think that was an i3. One of the new i3 K parts has I think one of if not the highest clock speeds of any consumer Intel CPU, that's the one you'd want for OpenVPN max speed!
-
Pushing MPEG2/MPEG4 around. HDHomeRun, Plex, etc. but doing site to site OpenVPN.
Unless you have a ton of HDHomeRuns you shouldn't need massive throughput for that. And yeah, everything that has been mentioned so far about OpenVPN is true. But how much bandwidth do you really need for that traffic? Honestly curious. I have 2 original HDHR devices so 4x tuners total and at max I can't imagine them generating more than about 70Mbps combined.
And, as others have said, gateway groups aggregating multiple OpenVPN tunnels do work.
As far as verified hardware goes, if you need support and to sell it upstream to management, buy from Netgate. I've been using pfSense for at least a decade now and have never run into incompatibility. I've run it on desktops, thin clients, servers, ESXi, and the official Netgate AWS AMI. Never a problem.
-
I mean, yes. It's not maxed all the time. I run 6 tuners from 2 different locations, and an additional 6 tuners locally.
OpenVPN seemed like the limiting factor, but it's really much more than that.
Pretty much, I run a massive LAN wherever I go, so 10+ computer are running TimeMachine, downloaders, etc.
The other issue is that I only get 500mbit on nntp servers, when the stupid fios router gets wire speed at about 950mbit.
Looking for a reliable platform that is small but powerfull.
The Supermicro 5018D-F8NT Looks amazing for the price.
-
make it smaller, lower power, quiet, 1U or Small-er form factor preferred.
This is really more crying for the brand new SG-3100 platform!
I actually also have a Fios GigE connection and have been very happy with the….
Ok that turns us back to another hardware section than. Do you use PPPoE?
….Supermicro 5018D-F8NT 1U server, which even for what I need is still a bit overkill.
The other issue is that I only get 500mbit on nntp servers, when the stupid fios router gets wire speed at about 950mbit.
It has a ASIC/FPGA inside that do the entire job, pfSense is a x86_64it based software firewall.
For sure you may need something higher, so sell your SG-2440 or turn it into a whatever platform such as a
a small Server, WLAN and or LTE router for the camping ground. The SG-4860 is owned by one of the developers
here, and he has 1 GBit/s symetric fibre line at home, he usually gets +900 MBit7s out of that line and ~470 MBit/s
over the IPSec VPN connection. His name is @gonzopancho and he was written that on reddit. So this would be the
key changer in your case I really think!
-
@BlueKobold
Thanks for the links.
Hrmm. Looking at the specs. It seems the D-1528 with the 2 extra cores really helps cpu marks.
It's a shame it doesn't have more ports, but thats OK. Switches are cheap enough.
-
I'm curiously following this thread as well, since I now have a 1G FiOS connection, use HDHomeRun extensively across VLANs (so it passes through pfsense for routing) and am taking on a site-to-site OpenVPN connection on a separate VLAN to handle some cloud orchestration tasks for my company. End result is low user count but extremely high bandwidth requirement on occasion. Somewhere in the following ranges when running batch jobs:
240 simultaneous connections (80 + 80 + 80)
30mB/s (upload) sustained throughput over two site-to-site IPSec IKEv2 AES-CBC-256 DH2048 SHA256
15mB/s (download) sustained throughput over site-to-site OpenVPN AES-256-CBC DH2048 SHA1I'm trying to move everything to IPsec but at the moment it's not my call to do so. I'm working on some compression and differencing changes that should reduce those limits eventually. At least using multipart uploads helps so far.
For the past 6 years, prior to the bandwidth upgrade and taking on a new project, I have been using a Jetway mini PC with an AMD G-T56N CPU. Now it's choking under the load even when not using VPN links, mostly due to interrupt overhead (Realtek NICs at fault here?):
Without snort it does relatively well (600mbps range) but that still leaves some available headroom.
Looks like SG-4860 may be a good option, but are there any planned updates to the line CPU wise coming up? I see a lot of threads about upcoming C3xxx options, but I know that's really new. Another small downside is I'll currently be financing this out of my personal budget, so I'm sensitive to making sure that what I buy will actually work to utilize the bandwidth I currently have access to.
Anyway, ignore the thread hijack but since I'm doing something kind of similar, I'll post if I find something that works well.
-
End result is low user count but extremely high bandwidth requirement on occasion.
To get any sort of answer you will probably have to be more specific.
-
Updating this thread as promised. I temporarily solved my issue by setting up a new tunnel just on the VM that I'm using to perform the high traffic work. My passmark 783 score CPU on Realtek drivers is still pushing 600mbps over the line when not dealing with encryption. Small bonus: fewer connections for snort to inspect since I'm tunneling through the router.
I'm pretty convinced that one of the newer QOTOM-Q355G4 units with an i5-5250U might be happily into "overkill" territory, but I'd have to get one and conduct tests to be positive.
As for the HdHomeRun situation, I'd like to clarify in case others are confused about issues with discovery:
HDHomeRun does NOT use mDNS. You can't use Avahi to forward discovery packets as far as I can tell (someone please prove me wrong!)This is a broadcast packet from HDHomeRun discovery:
This is an mDNS packet which uses multicast:
This is why I'm pretty sure you need to BRIDGE everyone onto the same subnet in order to use HDHomeRun discovery. I don't see a simple way out of this.
-
One of the new i3 K parts has I think one of if not the highest clock speeds of any consumer Intel CPU, that's the one you'd want for OpenVPN max speed!
Intel Core i3-7350K @ 4.20GHz
It's no longer number 3 on the Passmark Single Thread performance chart with the new coffee lake CPUs starting to trickle out, but it's still a price performance leader and then some!
It's a heck of a CPU for the money and the real sleeper of the Kaby Lake CPUs.
Also if you are a gamer, that's the CPU benchmark list to prioritize your CPU choice from. The vast majority of games are STILL heavily single thread dependent. In the off chance you have a beast of a video card like a GTX 1080ti so that your CPU will be more likely to bottleneck things, then you want something high on that single thread chart.
