Upgraded to Gigabit line, need to overhaul my network



  • So I recently upgraded my bandwidth, and unfortunately for me my old pfSense router couldn't keep up (Core 2 Duo w/6GB DDR2 Ram could only get about 250mbps).  I know that I am doing a total network overhaul in the next 6ish months and I want to have a permanent solution.

    I have 3 people using about 15 devices daily.  10 wireless devices, 5 hard-wired devices

    The things I need help figuring out:

    1.  I have 1 client that will be always using a VPN.  I am also considering using a VPN for the rest of the network.

    2.  I run a few packages, but nothing too extreme (yet).

    I am leaning towards using this piece of hardware: _https://www.amazon.com/dp/B071JBG8GL/_encoding=UTF8?coliid=I5HQGSU7KM2JB&colid=WAPSRWS1PPG9_EDIT:  I realized that this piece of hardware cannot use AES-NI encryption, and it will be incompatible with pfSense 2.5.

    If you were in my shoes, what would you set up for yourself?  I am debating on buying a dedicated firewall appliance such as: https://www.amazon.com/Firewall-Appliance-Gigabit-AES-NI-Barebone/dp/B072ZTCNLK/ref=sr_1_2?ie=UTF8&qid=1507640378&sr=8-2&keywords=protectli+aes-ni and then outfitting it.

    Celeron or Atom?  I have found some Celeron N3150 systems for very cheap ($150 barebones w/dual LAN) and I am curious if an Atom would be better
    4GB of ram or 8GB?
    60GB mSata SSD or more?  (I figure if I decide to cache with squid I should get 120GB+, but I am undecided if it would be worth it for my network)
    If I decide to VPN my whole network, I know there will be a bandwidth dropoff, but I want to make sure it is server side, not my router.

    Any input would be appreciated.



  • That processor won't be able to cary the load you're thinking off - especially the VPN traffic at a decent rate. I would recommend a mobile i5 chip (such as i5-5250U) with 4 gigs ram (min) - 8 gigs ram (future proof) and a 32gig SSD.

    There is limited benefit to using squid on a home network, let alone a home network that has a throughput of 1gbps to the outside world, unless you are on a metered connection. For reference squid would benefit you if:
    1. All of your users downloaded the same large files
    2. If your WAN bandwidth was less than your LAN bandwidth (as by installing squid you would just move the bottleneck from the WAN port to the LAN port with no overall increase in throughput)

    On the flip side you will get increased latency vs. going to the WAN directly.

    If you VPN your whole network expect a max throughput of about 600-700mbps max due to OpenVPN limitations. And that is using gateway groups (i.e. multiple OpenVPN connections, load balanced with each other) and for highly parallel traffic, such as torrents/browsing. For single large file downloads you would be limited to under 200-300mbps, again due to OpenVPN limitations, no matter what hardware you pick.



  • J1900 is going to suck for high speed OpenVPN. It's single threaded. Based on the following thread, you'll probably see sub 100Mbps throughput over the VPN.

    https://forum.pfsense.org/index.php?topic=115673.0

    You won't get Gigabit OpenVPN on any hardware, I think about the best you'll see is probably in the 6-700Mbps range if you go with something like an i3-7350K @ 4.2GHz.

    I would say for gigabit throughput if OpenVPN is involved, set your minimum CPU at a J3355 - you should get in the 300Mbps range with that over OpenVPN, full gigabit for just routing.
    Upper limit I would pay for is the 7350K @ ~$140.

    I would recommend a G4620 @~$90 as a good compromise.

    Pair that with some good NIC's - if your connection is PPPoE, use em - Intel PRO/1000, if not use igb - Intel i340 or i350.



  • @jgiannakas:

    I would recommend a mobile i5 chip (such as i5-5250U).

    For single large file downloads you would be limited to under 200-300mbps, again due to OpenVPN limitations, no matter what hardware you pick.

    Nope nope nope, all of this is bad/wrong.

    Mobile CPU's suck at OpenVPN, the 5250U is a 1.6GHz dual core from Broadwell.

    You can definitely break 2-300Mbps OpenVPN single thread with a modern high clock CPU. People get 300Mbps with J3355 Celerons single threaded, I'll grant you that there are diminishing returns, but 300Mbps is certainly not a ceiling.
    With gateway groups you can get full gigabit OpenVPN with even old quad cores - just use 4 OpenVPN gateways. Even an old i5-2400 will very likely hit Gigabit OpenVPN that way (~236Mbps x 4 cores)

    But - as mentioned, gateway groups have their own limitations. They are still a good idea for most people - just so long as you understand the limitations.



  • @belt9:

    @jgiannakas:

    I would recommend a mobile i5 chip (such as i5-5250U).

    For single large file downloads you would be limited to under 200-300mbps, again due to OpenVPN limitations, no matter what hardware you pick.

    Nope nope nope, all of this is bad/wrong.

    Mobile CPU's suck at OpenVPN, the 5250U is a 1.6GHz dual core from Broadwell.

    You can definitely break 2-300Mbps OpenVPN single thread with a modern high clock CPU. People get 300Mbps with J3355 Celerons single threaded, I'll grant you that there are diminishing returns, but 300Mbps is certainly not a ceiling.
    With gateway groups you can get full gigabit OpenVPN with even old quad cores - just use 4 OpenVPN gateways. Even an old i5-2400 will very likely hit Gigabit OpenVPN that way (~236Mbps x 4 cores)

    But - as mentioned, gateway groups have their own limitations. They are still a good idea for most people - just so long as you understand the limitations.

    Yes base clock is lower but it does turbo to 2.7ghz.

    https://forum.pfsense.org/index.php?topic=105238.msg709164#msg709164

    Also doing the above openvpn crypto benchmark on the i5 yields 320mbps throughput on aes-256-gcm and 296mbps on aes-256-cbc. Therefore its performance is on par with the J3355 and both will perform a tad bit slower than this in real life.

    Some performance comparisons are also here:
    https://forum.pfsense.org/index.php?topic=115673.0



  • Never rely on burst frequency for performance.

    Burst is just that - burst. You might get that frequency for a matter of seconds before it steps back down to base frequency.

    Burst is great for benchmarking, because the benchmark is often done before the CPU steps back down to base.
    Then 10 seconds later that performance goes away.

    Not to mention there's no guarantee you'll get it at all on FreeBSD.

    On top of all that, people typically try to keep the size of their routers to a minimum. This just means that the ambient case temps will be high enough that the CPU might not have the thermal headroom to burst at all, and if it does it will likely not be for long.



  • Don't get a protectli, but do check out Qotom (not the J1900). We have a thread for that: https://forum.pfsense.org/index.php?topic=132528.0

    You can get a cheap box with decent performance.


  • Rebel Alliance Global Moderator

    @DaddyNugget:

    So I recently upgraded my bandwidth, and unfortunately for me my old pfSense router couldn't keep up (Core 2 Duo w/6GB DDR2 Ram could only get about 250mbps).
    <snipped>If you were in my shoes, what would you set up for yourself?</snipped>

    I am in the same boat actually.. I just updated my line to 500/50 and my old pfsense VM just couldn't handle it..  To get instant access to the bandwidth I had to go with a temp install of a unifi usg 3p.. Only reason for this was it is cheap $100 and can route at speed… It handles the 500/50 without any issues..

    It only has to be handle the job until nov when get new pfsense hardware.  Which is more than likely going to be a sg-4860..  Go big or go home ;)  This is the most umph I can get from pfsense/netgate that aware of before go into rack units.  I don't have the place to put a rack system - even though would love too..

    Have to work out the details with the budget committee (wife), etc..  The $100 usg box was cheap enough to sneak through the budget without much grief.. hehehe

    Since your talking about redo of your network - do you have a budget in mind?  Have you looked at or have you considered hardware from pfsense/netgate?  Vs doing a DIY system?  Since you ask what I would do - while the price can be attractive, I personally would stay away from all the china boxes.. But that just me, there are many people that use them and are happy with them.



  • I agree with john on the chinese boxes. IMO official pfSense is supported, official, but you pay for that. DIY is community supported (pfSense has a pretty damn community support system on here, IRC and reddit), and unofficial - but you get amazing price/performance value.

    Then there's the chinese boxes. They aren't official, aren't supported, and they fall somewhere in between. You pay more for having the hardware pre-installed in a good looking SFF box, but you don't get any support if it doesn't work right. It is a better price/performance than official pfSense, but me personally I'd either save a lot of money and DIY or spend a little more and buy official.



  • @belt9:

    I agree with john on the chinese boxes. IMO official pfSense is supported, official, but you pay for that. DIY is community supported (pfSense has a pretty damn community support system on here, IRC and reddit), and unofficial - but you get amazing price/performance value.

    Then there's the chinese boxes. They aren't official, aren't supported, and they fall somewhere in between. You pay more for having the hardware pre-installed in a good looking SFF box, but you don't get any support if it doesn't work right. It is a better price/performance than official pfSense, but me personally I'd either save a lot of money and DIY or spend a little more and buy official.

    Yeah, most of the lesser known boxes or older boxes aren't a good choice. The few that work well have their own threads on the forums, but for other cases a DIY build is better. For cases where you want actual reliability and known vendors, the official hardware is the place to go.

    So far, our experiences and experiments with the more recent Qotom boxes (3rd gen Intel chips in the Celeron, Core i3 and i5 versions) as documented in the dedicated thread has been quite positive, plenty of guides to get a nice setup going. Other hardware solutions such as re-purposing other branded firewall boxes (like the Watchguard) have similar dedicated topics and information on setup, performance and quirks.


  • Rebel Alliance Global Moderator

    Another comment to these china boxes you see on amazon and such that mention pfsense.. They need to be real careful, I know pfsense has actively been going after them.. And they for sure can not have pfsense pre-installed on them.. Unless they have cleared that with pfsense.. Which I don't think any of them have.

    I know some threads have linked to some of these boxes, the links have been removed and pfsense has gone after them.. Just my understanding, not official in any way and I might be off base.

    But while you can say your hardware will "work" with pfsense.. You can not actually call it a pfsense firewall or hardware or have it installed on the box when shipped to the buyer, etc.  Unless you for sure have cleared that with pfsense, etc.

    To be honest, while you might save a few bucks doing DIY.. I think getting hardware from pfsense/netgate is better in the big picture.  It helps pfsense, it helps you know your hardware is going to be rock solid.. And you get gold to boot ;)  That being said if your wanting to build some rocket ship on a shoe string budget - there is that aspect of it too ;)

    I am curious to see what kind of info we get from people in the field once the sg-3100 start shipping.  This price point is pretty attractive for official hardware I think.. Take into account the gold and access to the book, etc.  And shoot your price point is right in line with some of these soho routers that don't do shit ;)



  • @belt9:

    J1900 is going to suck for high speed OpenVPN. It's single threaded. Based on the following thread, you'll probably see sub 100Mbps throughput over the VPN.

    https://forum.pfsense.org/index.php?topic=115673.0

    You won't get Gigabit OpenVPN on any hardware, I think about the best you'll see is probably in the 6-700Mbps range if you go with something like an i3-7350K @ 4.2GHz.

    I would say for gigabit throughput if OpenVPN is involved, set your minimum CPU at a J3355 - you should get in the 300Mbps range with that over OpenVPN, full gigabit for just routing.
    Upper limit I would pay for is the 7350K @ ~$140.

    I would recommend a G4620 @~$90 as a good compromise.

    Pair that with some good NIC's - if your connection is PPPoE, use em - Intel PRO/1000, if not use igb - Intel i340 or i350.

    An I3 7350K is not required for high OpenVPN speeds. I reach up to 800 Mbps with OpenVPN and PIA using a Intel G4400 and an Intel i350 NIC, the G4400 is almost $100 cheaper than a 7350K. The G4620 is probably a good choice as well, but might as well get a G4560 if you're after HT.

    @DaddyNugget: it's really the OpenVPN speed you need that determines which CPU suits you best. It's the single core speed that determines its OpenVPN capabilities. So for 300mbps OpenVPN a J3355B might be your best bet, if you need something faster a Pentium (G4400/G4560 etc.) or I3/I5(u) with a high single core speed is required. You can build a J3355B or Pentiums system yourself, same for a I3/I5 system but for a I3u or I5u you will probably need a prebuild system (QOTOM etc.).



  • Wow, 800Mbps single thread on OpenVPN is really impressive!

    Are you using fast io and increase buffers?



  • @johnpoz:

    Go big or go home

    I thought this was your home installation.  :P



  • Wow!  I was doing some homework before work, and figured maybe I would get a single reply before I got back, thanks for all the help guys!

    So because a lot of similar suggestions were made, I will rattle off my answers to everyone.

    1.  I am not looking at any specific manufacturer for hardware right now, but if I decide to go with some Chinese hardware firewall device I will most likely be going through aliexpress.  My reasoning here is that it costs $50-100 less than the exact same product on Amazon, and I might be able to score a better deal overall.

    2.  I have considered the already setup pfsense firewalls, but I haven't commited to these yet for a couple reasons.  The first is that I simply enjoy assembling my own pc's and devices.  The second is that they were suggesting me a firewall that is higher than my anticipated budget.  An SG-2440 w/32GB EMMC storage is $550.  If I understand correctly, the emmc storage is slower than a typical SSD and I was unsure if this could impact my performance.  A firewall device from AE w/a celeron (I can't pull up the specific one, but it was a 4 core w/AE-NI) w/8gb ram + 64GB mSATA SSD was $400.  I didn't realize that pfsense gold came with the hardware, which is making me re-evaluate what I want to buy.

    3.  Although I have plenty of components to DIY it myself (2port gigabit Intel NIC, Skylake Celeron I am not using, etc) the reason I want to put a bit more time, money, and effort into this is to both reduce my power bill and the size of the machine.  My previous build was in a smaller workstation, but was awkward to keep, not to mention it pulled much more than 20W.  My thought process is to get something small and capable, but that I won't need to replace for at least 4 years.

    4.  My network overhaul has a roomy budget because I won't be able to do everything all at once.  I am planning on running proper Cat6 or Cat7 throughout my home.  I will have a proper server cabinet, whether it is in a closet or mounted somewhere, with a patch panel etc.  I will also be running at least one PoE AP.  My end goal is to have a setup that isn't Jerry-rigged together and falling apart all the time, without paying for an enterprise solution.

    Ultimately I was originally under the impression that a DiY gigabit router would be $200 or less, but I came to the conclusion that if I am willing to spend $200 for a half-ass firewall, I should instead be willing to pay 2x-3x for a proper one.

    Thank you again to everyone for your quick and informative responses, the information about the VPN use was certainly helpful.  Also I came to the same conclusion about the usefulness of squid if my throughput goes all the way through to my WAN.  It just didn't dawn on me until I was in the car.



  • @DaddyNugget:

    Wow!  I was doing some homework before work, and figured maybe I would get a single reply before I got back, thanks for all the help guys!

    So because a lot of similar suggestions were made, I will rattle off my answers to everyone.

    1.  I am not looking at any specific manufacturer for hardware right now, but if I decide to go with some Chinese hardware firewall device I will most likely be going through aliexpress.  My reasoning here is that it costs $50-100 less than the exact same product on Amazon, and I might be able to score a better deal overall.

    2.  I have considered the already setup pfsense firewalls, but I haven't commited to these yet for a couple reasons.  The first is that I simply enjoy assembling my own pc's and devices.  The second is that they were suggesting me a firewall that is higher than my anticipated budget.  An SG-2440 w/32GB EMMC storage is $550.  If I understand correctly, the emmc storage is slower than a typical SSD and I was unsure if this could impact my performance.  A firewall device from AE w/a celeron (I can't pull up the specific one, but it was a 4 core w/AE-NI) w/8gb ram + 64GB mSATA SSD was $400.  I didn't realize that pfsense gold came with the hardware, which is making me re-evaluate what I want to buy.

    3.  Although I have plenty of components to DIY it myself (2port gigabit Intel NIC, Skylake Celeron I am not using, etc) the reason I want to put a bit more time, money, and effort into this is to both reduce my power bill and the size of the machine.  My previous build was in a smaller workstation, but was awkward to keep, not to mention it pulled much more than 20W.  My thought process is to get something small and capable, but that I won't need to replace for at least 4 years.

    4.  My network overhaul has a roomy budget because I won't be able to do everything all at once.  I am planning on running proper Cat6 or Cat7 throughout my home.  I will have a proper server cabinet, whether it is in a closet or mounted somewhere, with a patch panel etc.  I will also be running at least one PoE AP.  My end goal is to have a setup that isn't Jerry-rigged together and falling apart all the time, without paying for an enterprise solution.

    Ultimately I was originally under the impression that a DiY gigabit router would be $200 or less, but I came to the conclusion that if I am willing to spend $200 for a half-ass firewall, I should instead be willing to pay 2x-3x for a proper one.

    Thank you again to everyone for your quick and informative responses, the information about the VPN use was certainly helpful.  Also I came to the same conclusion about the usefulness of squid if my throughput goes all the way through to my WAN.  It just didn't dawn on me until I was in the car.

    Considering your points, the (so far) well tested Qotom is the way to go. Add pfSense gold to that (99,-) and you'll be at ~300 in total. It'll be small, not use a lot of power, and you'll be supporting the project. By the way, getting a $400 thing is a bit high for what you'd be getting. I have not found a PC or embedded system worth $400 on there ;-)



  • @johnkeates:

    @DaddyNugget:

    Wow!

    until I was in the car.

    Would you mind NOT quoting the total post, please? The info is already there to read for everyone, we don't need it twice. Right?



  • 4.  My network overhaul has a roomy budget because I won't be able to do everything all at once.

    Mostly here are playing more then one point together and it might be better to know all things you will be reaching.
    So if you are telling around that you will be later able to install more then one packet on top of this all, you should
    overthink that before buying your hardware. Increasing the mbuf size, squid, snort and pfBlockerNG will be fast
    eaten 4 GB!

    I am planning on running proper Cat6 or Cat7 throughout my home.  I will have a proper server cabinet, whether it is in a closet or mounted somewhere, with a patch panel etc.  I will also be running at least one PoE AP.  My end goal is to have a setup that isn't Jerry-rigged together and falling apart all the time, without paying for an enterprise solution.

    Perhaps you may think about a fast switch that will be able to route your network with wire speed can be relieve the
    firewall from some work to run one or more packets with ease. Cisco SG200/SG50 series SG300/SG350 series
    might be a really nice matching.

    Ultimately I was originally under the impression that a DiY gigabit router would be $200 or less, but I came to the conclusion that if I am willing to spend $200 for a half-ass firewall, I should instead be willing to pay 2x-3x for a proper one.

    The most peoples see only what they are running before changing to pfSense! Its mostly a consumer plastic router that is
    ASIC/FPGA based that will do then the entire job, and we are talking here then often over SPI (netfilter) and NAT (network
    address translation), but pfSense is a firewall that works with the BSD packet filter and can be turned into a fully featured
    UTM device, but without the whole license subscriptions and fees that came along with that UTM devices mostly too.

    And so the most users are thinking the best plastic router will be around -$200 till ~$300 and they are able to build
    a pfSense firewall also based on that budget or limit, it is truth but then often on top of this they have needs that will
    be not matching well to that budget as well, 1 GBit/s routing on the WAN, highest OpenVPN throughput given on earth
    and so on and so on.



  • Lots of headroom in the budget and already running cat6+ and Gb WAN? Go for 10GbE LAN!!! ;D

    Definitely get yourself a solid managed switch whether you go GbE or 10GbE.

    $200 DIY build (if you don't already have things to reuse) will get you a J3355B build with an eBay i340t2 & SO-DIMM's, picoPSU and small SSD - very power efficient and reasonably powerful. More than that will cost more $$.

    Since you're upgrading the whole network and jumping into pfSense definitely go for Gold no matter where you buy the hardware.


  • Rebel Alliance Global Moderator

    @jahonix:

    I thought this was your home installation.  :P

    It is! ;)  But the 6 ports are very attractive to me.. I don't like having to hairpin intervlan traffic..  This gives me the ability to break out vlans onto their own connection and just use a dumb switch I have on the shelf vs having to hairpin on the same physical interface traffic between vlans.

    I am a bit short on ports to do all of them currently..  But bigger switch will be next on the list.  Until that I will leverage 1 of them to hang my pi network off.. They are all on the same vlan, so I can break them off onto their own interface on pfsense and remove that traffic from the interface handling all my vlan traffic currently, etc.

    So of the 6 ports I will be using 4 of them right out of the gate.. Leaves 2 for future growth… Since won't be using wan interface into my esxi host now.. Could connect that direct to pfsense interface for another segment or vlans for vms, etc.

    Plus quad vs dual, more ram, etc. Plan on using this box or a while ;) And since my VM was a limitation with playing with other packages like ntopng, and the ips packages - this give me ample performance to play with about any sort of packages I want, etc..

    Funny thing is my home pfsense will be bigger than work.. But work is replacing juniper in 2 more branch locations here soon.  Prob get 3100 see how fairs before getting the 2nd one to decide if should stick with 2440 or 3100 save couple bucks, etc. Im hoping to move them in to more central locations and bigger load, etc.. But all I could get ahead for was the branch places for now.. So that should be about 10 I think when all said and done..



  • @belt9:

    Wow, 800Mbps single thread on OpenVPN is really impressive!

    Are you using fast io and increase buffers?

    Yes, both, really makes a difference.



  • @johnpoz:

    @jahonix:

    I thought this was your home installation.  :P

    Funny thing is my home pfsense will be bigger than work.. But work is replacing juniper in 2 more branch locations here soon.  Prob get 3100 see how fairs before getting the 2nd one to decide if should stick with 2440 or 3100 save couple bucks, etc. Im hoping to move them in to more central locations and bigger load, etc.. But all I could get ahead for was the branch places for now.. So that should be about 10 I think when all said and done..

    As FYI I think the 3100 is arm based. And my guess here is it will be walked all over performance wise by a decent x86 processor, be that Celeron or I series. Similarly for the 3100 - its an atom based processor so weaker than an I series and possibly the Celeron described above. In general the PFsense pre-built boxes dont really compete on bang for buck in terms of performance. You can get double+ the performance with half/ two thirds the cost. You do get official support and so forth but for a home install I would opt for self built / a china box as its not a mission critical application and you'll get better performance for your money. Buy the gold subscription directly and it is still better value for a home system.



  • Almost all Official pfSense hardware is intended for business, not home users. Just look at the "Best For" section.

    Home users paying $450 (not including cost of gold) for a C2358 and i350t4 paired with a 32GB SSD flash and picoPSU(SG2440), either just want to support the project or don't particularly care about price/performance. A $250 J3355B build will smoke the SG-2440.

    Conversely, most businesses would be ill advised to DIY their edge router just to save a couple hundred bucks. For them, buying official pfSense hardware is the obvious way to go.

    Netgate knows this, hence they correctly claim that most of their hardware is best for businesses.


  • Rebel Alliance Global Moderator

    Well I for sure love to support the project that is for sure.. I have been a very active member of the forum for 10 years ;)

    The SG-4860 I am looking to get states best for ;)

    Best For:

    SMB with Medium Sized Networks
        Small to Medium Sized Branch Office with heavy loads
        Managed Service Providers (MSP) / Managed Security Service Provider (MSSP) On Premise Appliance
      Anyone with High-Speed Gigabit Connections
        Many VPN Connections

    While I might not have gig currently.. Not sure what I might have next year ;)

    The sg-2440, sg-3100 lists
    Teleworkers needing an "Always-Up" network or VPN connections

    I am with you though all of their hardware is more designed for business use that is for sure.. I don't think they are pricing them with the home user in mind ;)

    If we are going to talk about pricing differences.. Your J3355B build for $250 draws how much power?  So in X number of months your up front cost savings could be eaten up by your extra $ per month powering it.. So sure I could put together something for cheaper now - but I am going to have it on for years..  So while I save few hundred now.. When do I start loosing money paying the electric bill?  Have to do the math, etc..  I would much rather pay that money up front and support pfsense.



  • I certainly won't argue with supporting the cause, I think that's awesome!

    J3355 is low power passively cooled Celeron, i340 is low power NIC, picoPSU 80W has high AC-DC efficiency (I think 88%, haven't looked at spec sheet in awhile?) SSD and SO-DIMM DDR3L also low power - the build has no moving parts. I never measured my J3355B at the wall when I was using it for pfSense.
    I currently use it for HTPC with LibreElec (Linux) and measured it with a killa Watt and it pulled I think between 11-14W during high bitrate HEVC 4K playback.

    So with pfSense shouldn't be all that different.

    All that aside, supporting the cause is a great reason to buy official for home! Just not everyone has the means to do so.



  • @johnpoz:

    When do I start loosing money paying the electric bill?

    Not before the equipment becomes obsolete.



  • @BlueKobold:

    …you should
    overthink that before buying your hardware. Increasing the mbuf size, squid, snort and pfBlockerNG will be fast
    eaten 4 GB!

    Perhaps you may think about a fast switch that will be able to route your network with wire speed can be relieve the
    firewall from some work to run one or more packets with ease. Cisco SG200/SG50 series SG300/SG350 series
    might be a really nice matching.

    To address your points, I agree that I should have an understanding of my network goals, which I do.  I know that my network will not exceed 1Gbps anytime in the next 5 years at least (and honestly, I doubt I will even need the connection I have at that point).  I figured I am just going to throw 8GB of RAM into whatever box I have just for the peace of mind (and I am able to upgrade my laptops RAM from 8GB to 16GB, and reuse the sticks from the laptop, getting a double benefit).

    @belt9:

    Lots of headroom in the budget and already running cat6+ and Gb WAN? Go for 10GbE LAN!!! ;D

    Definitely get yourself a solid managed switch whether you go GbE or 10GbE.

    $200 DIY build (if you don't already have things to reuse) will get you a J3355B build with an eBay i340t2 & SO-DIMM's, picoPSU and small SSD - very power efficient and reasonably powerful. More than that will cost more $$.

    Since you're upgrading the whole network and jumping into pfSense definitely go for Gold no matter where you buy the hardware.

    I have multiple gigabit switches, plus none of my devices are able to capitalize on a 10Gbps network.  Since 1000ft of In-wall Cat6 is roughly $100, but Cat7 is $350+, I can't justify laying down the cable without having any devices actually be able to use it.

    My goal is to run Cat6 to every room, 1-2 outlets w/a ceiling mounted Wireless AP.  I can probably get away with a 12-port patch panel, but will get a 24 port anyway just in case I decide to add more ports later.  I plan on getting a 4u+ cabinet that I can stuff the panel, my new pfsense router, a larger switch (using my current ones in each room), and potentially migrating my NAS into a rack-mount unit.

    So with that said, my budget is roomy, but it isn't unlimited.  It is hard to get approval from my wife for spending an extra $500-$1000 for a negligible, if any performance increase.  I plan on getting the best 'bang for my buck' as far as hardware.

    Thanks again for everyone's input, it has certainly helped a lot.



  • @DaddyNugget:

    @belt9:

    Lots of headroom in the budget and already running cat6+ and Gb WAN? Go for 10GbE LAN!!! ;D

    Definitely get yourself a solid managed switch whether you go GbE or 10GbE.

    $200 DIY build (if you don't already have things to reuse) will get you a J3355B build with an eBay i340t2 & SO-DIMM's, picoPSU and small SSD - very power efficient and reasonably powerful. More than that will cost more $$.

    Since you're upgrading the whole network and jumping into pfSense definitely go for Gold no matter where you buy the hardware.

    I have multiple gigabit switches, plus none of my devices are able to capitalize on a 10Gbps network.  Since 1000ft of In-wall Cat6 is roughly $100, but Cat7 is $350+, I can't justify laying down the cable without having any devices actually be able to use it.

    My goal is to run Cat6 to every room, 1-2 outlets w/a ceiling mounted Wireless AP.  I can probably get away with a 12-port patch panel, but will get a 24 port anyway just in case I decide to add more ports later.  I plan on getting a 4u+ cabinet that I can stuff the panel, my new pfsense router, a larger switch (using my current ones in each room), and potentially migrating my NAS into a rack-mount unit.

    So with that said, my budget is roomy, but it isn't unlimited.  It is hard to get approval from my wife for spending an extra $500-$1000 for a negligible, if any performance increase.  I plan on getting the best 'bang for my buck' as far as hardware.

    Thanks again for everyone's input, it has certainly helped a lot.

    I was just playing with 10Gb, it isn't terribly practical for home use yet other than client to client stuff.

    As far as cabling though, Cat6 is good for 10Gb on runs up to 180', so you're good there. Ethernet switches and NIC's are where you get killed on 10Gb though.

    You can certainly build out a very solid router though. Have fun!



  • I second the opinion not to go with the Protectli device on Amazon.  I bought one.  The E3845 4 port job, and I hit about ~500Mbps on a 1Gb X 35Mb cable provider.  With a laptop directly connected, about 840Mbps.  Plus I am pretty sure this device is a rebadged from, copy and paste here, YanLing Industrial Computer Technology (Shenzhen) Co.,Ltd. on Alibaba.

    I am pretty close to re-purposing this device as an upgraded workstation for my wife, and getting an SG-4860 as soon as I do some more checking on it's throughput with Snort, pfBlockerNG and other services turned on.  I might go with used i5/i7/Xeon hardware if I can keep the power consumption to a minimum.  I can't believe my router needs more power then my for VM, two container Proxmox server running a C2750.



  • What effect will QuickAssist support have on fast OpenVPN once QAT software support is enabled?  Might a Netgate SG-4860 or equivalent be a good choice when future-proofing is considered?

    10Gbase-T works but I don't think it should be taken into consideration yet.  I'm not sure the upstream providers have really settled on what connection/media to use from the modem to customer, and 10G switches are still expensive and noisy.  For gigabit connectivity I'd like some extra headroom on WAN and LAN (more than 1000base-T) but it's just not practical to spend money on that right now unless you already need 10G for other reasons.


  • Rebel Alliance Global Moderator

    "Not before the equipment becomes obsolete."

    That depends on the swing doesn't it.. Lets use the 20watt number thrown out there somewhere..  Yeah that is not very much.. about 25$ year..  If the device is 100$ cheaper, that gives you 4 years until you break even on the cost difference..  Don't know about you.. But I would hope to get 4+ years out of the thing ;)  It sure not going to be obsolete in 4 years.  Unless maybe they put in new internet in your area and you can get 10Ge for cheap ;)

    If its a 40w swing.. That is 2 years..  So while it might be nice to throw that rocketship of a CPU at it and sure can do all kinds of cool things with it.. Do you really need that?

    So lets take device more priced for home, the sg3100.. So since your wanting to support the company you would get gold either way right ;)  So throw that out.. So now you get that hardware for $250.. Can you build your box comparable to the sg3100 for $150?  Does it use 20w or more per hour more?  If so then your savings are gone in 4 years… If you run it for six years.. Then that device actually cost you 50$ more than if you would of just gotten the sg3100..

    I get you how a few watts here a few there make no real difference.. I wouldn't drop $200 bucks to save 20w an hour..  But if your needing to buy hardware anyway for a project.. I would for sure take into account the difference in electric cost of the device.. Don't forget that that 20watt swing also means that device can run on ups for just that much longer than the power hungry device, etc.



  • @johnpoz:

    "Not before the equipment becomes obsolete."

    That depends on the swing doesn't it.. Lets use the 20watt number thrown out there somewhere..  Yeah that is not very much.. about 25$ year..  If the device is 100$ cheaper, that gives you 4 years until you break even on the cost difference..  Don't know about you.. But I would hope to get 4+ years out of the thing ;)  It sure not going to be obsolete in 4 years.  Unless maybe they put in new internet in your area and you can get 10Ge for cheap ;)

    If its a 40w swing.. That is 2 years..  So while it might be nice to throw that rocketship of a CPU at it and sure can do all kinds of cool things with it.. Do you really need that?

    So lets take device more priced for home, the sg3100.. So since your wanting to support the company you would get gold either way right ;)  So throw that out.. So now you get that hardware for $250.. Can you build your box comparable to the sg3100 for $150?  Does it use 20w or more per hour more?  If so then your savings are gone in 4 years… If you run it for six years.. Then that device actually cost you 50$ more than if you would of just gotten the sg3100..

    And if the price difference is $0 and the power difference is 10kW, then it pays for itself instantly! Of course, those aren't the real numbers so why bring them up? You started this by comparing an SG-4860 to a J3355, so you're looking at a $500 premium to save less than 20 watts. I'll stand by the assertion that the hardware will be obsolete before that investment pays for itself.

    Then you tried to change the rules by pushing the sg3100 instead. That certainly makes the price difference lower and the shaves a couple more watts off the power consumption, but we don't actually have any idea how it performs. Is it a reasonable alternative to a J3355 for OpenVPN? No idea. (Probably not.) Excluding OpenVPN also makes the J3355 overpowered, so now you're comparing the SG3100 to an APU2. The value there basically comes down to how important you consider the integrated switch & the gold subscription. For some people/applications it's a slam dunk, for others it's a meh.



  • It really just comes down to what the user is comfortable buying or building. But generally speaking you will come out on top diy.

    Official hardware (when compared to today's COTS hardware) is shockingly anemic. But this is very normal for this type of hardware. I would also argue that 95% of the DIY boxes for home use pfSense are WAY overspecced for routing packets.

    For $250 to beat out the SG-3100? Easy, J3355B build with a Pico PSU in an m300 case with an i340t4. J3355 is apples and oranges to the ARM A9. The power difference will probably be sub 10W as well.

    Another option is a business laptop, for example my travel laptop is a T430. It cost me I think $150 used and comes with an i2xx Intel NIC and an i5 with AES-NI. Pair that to a decent switch for $100, install pfSense and now you have a very powerful home network that is also low power and has a UPS.
    But not everyone can or will repurpose a laptop.

    There are reasons to buy official pfSense for the home, I'm just saying price/performance isn't one of them, even when power consumption is considered.



  • @jrv:

    What effect will QuickAssist support have on fast OpenVPN once QAT software support is enabled?  Might a Netgate SG-4860 or equivalent be a good choice when future-proofing is considered?

    No, there's very little chance that the QAT will ever do anything useful on those boxes. You certainly should not base purchasing decisions on something that might happen at some undefined point in the future.



  • The power consumption decision involves a lot more than just savings on the electric bill.

    My machine closet is sound insulated with an air duct & return (the guy I bought the house from did it, not me!)  Few people have that luxury: my previous "machine closet"  was the back wall of coat closet with no air flow & surrounded by thermal insulation; before that the equipment was in the office with me, exposing me to the noise any fans might make.

    My guess is that a fan is out of the question for most people.  I can see paying a little more for a lower-power solution if the goal is to minimize noise and thermal issues while approaching gigabit speeds with OpenVPN



  • You will not approach Gigabit Speeds on OpenVPN with any cheap fanless solution. The exception to this would be via Gateway groups, then you can do it with something like a J3455 - I think each of it's four core does something like 200Mbps OpenVPN, and that's another cheap fanless board.

    But if you want it on a single thread, and fanless? I'm not aware of anything that will do that without some extreme cooling solutions. Certainly no official products if that's what you're alluding to.
    But again, it's worth mentioning that High Throughput OpenVPN isn't really an enterprise level solution (most users trying to do this are home users) and most of the official products are aimed towards enterprise. So you can still get great VPN performance out of the official hardware using IPSec, it'll just take a little more effort to setup.



  • @jrv:

    The power consumption decision involves a lot more than just savings on the electric bill.

    My machine closet is sound insulated with an air duct & return (the guy I bought the house from did it, not me!)  Few people have that luxury: my previous "machine closet"  was the back wall of coat closet with no air flow & surrounded by thermal insulation; before that the equipment was in the office with me, exposing me to the noise any fans might make.

    My guess is that a fan is out of the question for most people.  I can see paying a little more for a lower-power solution if the goal is to minimize noise and thermal issues while approaching gigabit speeds with OpenVPN

    An APU2 is fanless. A J3355 is typically fanless. A kaby lake celeron can be fanless, or depend on a large low RPM case fan. Even an i3 or i5 has a variable speed fan which isn't tremendously loud unless you're really hitting the CPU–in which case you're doing something you wouldn't be able to do with any low power fanless system, anyway. The worst offenders with fan noise are rackmount systems, which generally aren't designed with noise as a concern because they're intended to be in a rack in a data center. Just avoid those and you're good. So while you're right that there's more to low power than the electric bill, it's not particularly relevant to this thread.



  • What effect will QuickAssist support have on fast OpenVPN once QAT software support is enabled?  Might a Netgate SG-4860 or equivalent be a good choice when future-proofing is considered?

    At this time, today when I drop you this could of lines here, to speed up VPN tunnels and pointed directly to
    speeding up OpenVPN it will be having no impact and/or benefit! But, if you read between the lines here and there
    this feature or option is even actually in the game play or better said it is even on the road map of the pfSense
    developers that are not inserting it into the code for nothing. Perhaps it´ll be not really important for any user
    and many customers or plain all who are using pfSense, I am pretty sure, but the ones who want it, need it or
    use it, they will be happy with it.

    10Gbase-T works but I don't think it should be taken into consideration yet.  I'm not sure the upstream providers have really settled on what connection/media to use from the modem to customer, and 10G switches are still expensive and noisy.  For gigabit connectivity I'd like some extra headroom on WAN and LAN (more than 1000base-T) but it's just not practical to spend money on that right now unless you already need 10G for other reasons.

    Netgear GS110MX ~200 € - unmanaged Layer2
    Netgear GS110EMX ~250 € - WebGui Layer2
    D-Link DGS1510-20 ~230 € - CLI, WebGui Layer3

    Excluding OpenVPN also makes the J3355 overpowered, so now you're comparing the SG3100 to an APU2. The value there basically comes down to how important you consider the integrated switch & the gold subscription. For some people/applications it's a slam dunk, for others it's a meh.

    SG-3100 VPN and WAN throughput, the first numbers (lab tests)

    • the device has up to a gigabit throughput with pfSense

    • up to 300Mbps throughput with IPsec AES128-CBC SHA1.

    • up to 95Mbps throughput with OpenVPN AES128-CBC SHA1

    Thread on reddit with the same numbers
    SG-3100 is doing 300mbps IPsec in the lab, but we just found that only 1/2 the crypto unit is enabled

    If the VPN is for mobile clients from the road to home network it should also be running well over IPSec,
    if not or for an VPN provider connection it will be better to go with another hardware here in that case.

    Intel Atom C2558 V Intel Atom C3558 AES
    (besides of all)

    No, there's very little chance that the QAT will ever do anything useful on those boxes. You certainly should not base purchasing decisions on something that might happen at some undefined point in the future.

    OK I will consider to this statement for sure, during one or more development phase´s all can be changing
    fast as no one was able expect it before.

    Intel QAT small talk:
    9 month ago
    one moth ago
    another one month ago
    2017 Userspace summit



  • @BlueKobold:

    Intel QAT small talk:
    9 month ago
    one moth ago
    another one month ago
    2017 Userspace summit

    You do understand that the QAT in the C3xxx series is incompatible with the QAT in the C2xxx series? The more talk there is about the QAT in the newer series, the less likely that the QAT in the C2xxx will ever be utilized. (And, in fact, you can find the pfsense developers directly aying that it's unlikely that they'll ever bother with the QAT in the C2xxx.)



  • Let us imagine some other points, I said only imagine, not that this will be coming or passing through!

    You do understand that the QAT in the C3xxx series is incompatible with the QAT in the C2xxx series?

    Yes I am understanding that! But you should be thinking more positive please.

    If the QAT driver version 1.6 from pfSense team is not compatible with the Intel Atom C2000 but perhaps with
    the newer negate hardware based on Intel Atom C3000 called Denverton and the QAT driver version 1.5 from
    the NetBSD team is supporting also the Intel Atom C2000 called Rangeley, they only have to exchange this
    drivers and porting them to each of their OS, so the developers will not have any more to bother with that
    driver and all is fine for them and us!

    So it could be happen, that at November 2017 the newer hardware from netgate will be launched and fine for
    using QAT and perhaps in Dezember 2017 or later it could be happen that the older customers and clients
    of them get their "Christmas parcel" too and will be able to use QAT also. Its more cutting half the entire
    work time on that drivers that must only be exchanged then as the results.

    For sure that can be running very different each from another, or never becomes true but it will be a real chance
    for and us too as I see it right.

    And being very open talking over that point, perhaps many users will be very impressed if they know that peoples
    from pfSense and/or were talking with employees from the VyprVPN company about the one or other thing, who
    knows it really….....

    The more talk there is about the QAT in the newer series, the less likely that the QAT in the C2xxx will ever
    be utilized.

    But with this words you are talking that it will be not utilized only and not it is not finding its way into the system, right?  ;)
    Like on Rangely, the QAT scales by the number of cores. Unlike on Rangeley, the QAT has good support. Link

    And, in fact, you can find the pfsense developers directly aying that it's unlikely that they'll ever bother with the QAT in the (C2xxx.)

    I don´t know if that driver from the NetBSD project is able to exchange only, or if this will be easy or able to realize,
    but if so I think this might be nice for both parties as well as for us.