Hardware Advice



  • Hello everyone.  I am looking for some recommendations for new hardware.  I just got AT&T gigabit fiber and when I plug their gateway into my switch I get the speeds shown in the attached screenshot but when I put my pfsense firewall in front of everything my down and up speed dropped to around 380Mb/s.  I did switch interfaces to use the two on-board nics rather than the daughter card and after that I was getting 450 Mb/s down and 500 Mb/s up so that helped a little.  I feel like my hardware is limiting my speeds.  I have tuned and tweaked my pfsense configuration to the best of my knowledge and using guides from pfsense community users.

    Currently I am using a Dell r210 1u I inherited from a company I used to work for. I have had it for about 5 years and it was already used.  It has a standard 7200 rpm dive along with 1GB ram and Intel(R) Atom(TM) CPU D525 @ 1.80GHz 4 CPUs: 1 package(s) x 2 core(s) x 2 HTT threads.

    I am looking for something small form factor that uses a flash card or that I can put a small ssd into but has enough cpu and ram power to push 1Gb throughput.  It needs to have at least 2 nics obviously. My budget is going to be around $300 and am willing to build something I am just not familiar with small form factor components and am looking for advice.

    Would either of these devices push 1Gb?  I am looking at both trying to decide which would be better.  Please any advice is appreciated.

    https://www.amazon.com/gp/product/B01GIVQI3M/ref=ox_sc_act_title_1?ie=UTF8&psc=1&smid=AZEYJ27R4YB41

    https://www.amazon.com/gp/product/B01N6MDE01/ref=ox_sc_act_title_2?ie=UTF8&psc=1&smid=A2UOC8V31KFPZU

    One thing to add that I have noticed today.  When I download something and my firewall cpu pegs I start losing connectivity until the processor can calm down then it ramps back up and craps out again.

    I read the pfsense hardware sizing guide and it states for 501+ mbps you need at least 3.0Ghz.

    I also read that pfsense 2.2 or later will use multi core cpu's but mine does not seem to be and I am on the latest version.




  • Those 2 links are showing a J1900 box.

    In this thread, they talked a lot with J1900:
    https://forum.pfsense.org/index.php?topic=114202.0

    If you need more power, you may consider Qotom q355g4 in amazon or aliexpress which is with a i5-5250u. There are few models under $300.

    But it seems no way to push gigabit intel NICs to achieve 1 gigabit NAT(like 980Mbps or 990Mbps). The best score I saw in this board is 950Mbps.



  • Thank you for that info.  I have been doing additional reading and it seems that 2.3.3 should be using multi core cpu's properly.  I have upgraded since the 1.7 version and have never reinstalled completely so I am going to try that and hopefully the new kernel will utilize the quad core cpu correctly.



  • @authenticx:

    Thank you for that info.  I have been doing additional reading and it seems that 2.3.3 should be using multi core cpu's properly.  I have upgraded since the 1.7 version and have never reinstalled completely so I am going to try that and hopefully the new kernel will utilize the quad core cpu correctly.

    A D525 is slow, you'll need a faster CPU for gigabit.



  • You've got what I consider a first world problem, my friend.  A 1Gbps connection and looking for pfSense hardware to handle it.  A lot depends on what else you want to do besides basic NAT and firewalling, but I'd skip the J1900 solutions if I were in your shoes.

    The pfSense hardware guide is a bit outdated, but the basics remain true for a 1Gbps connection.  You need a fast CPU.  You need enough HDD (or SSD) space to run the OS.  Don't consider CF or any other flash storage for new pfSense builds.  RAM amounts aren't dependent on the speed of the connection, more so on the packages you run, and of course the number of connections, which for a home environment are negligible.

    I have a bit of a maxim regarding pfSense:  Choose two of three:  Low power, high performance, low cost.

    If I was choosing hardware for a home 1Gbps connection on a $300 budget, I'd be looking at ITX socketed boards, a Kaby Lake CPU (the cheapest will do) and a PCIe 2 or 4 port Intel chipset NIC (that doesn't mean Intel branded; there are plenty of used server NICs on the market from Dell, HP, IBM, Intel, etc that all use Intel chipsets and are indistinguishable to pfSense). For storage, I guess a 60GB SSD is probably the cheapest out there now.  RAM is up to you. pfSense itself will run just fine with 512MB of RAM.  I'd start with a 4GB stick of whatever works in your board, since a 4GB stick seems to be the lowest common denominator now.

    That is my recommendation for choosing high performance and low cost.


  • Banned

    I've read threads on here where people claim to get gigabit speeds with N3150's. It doesn't seem like you need as much CPU as some make it out for gigabit WAN. Maybe you do though, I've never tried.

    It doesn't make much sense to me though. If you aren't using packages, why would you need a mid level desktop CPU to do what some embedded SOHO routers can do?
    Again, I don't speak from experience but most recommendations seem to aim to buy you a system that will handle what you need without trying hard. Seeing as to how you're last system was a hand-me-down you used for five years you probably actually want to pay for a system that does just what you want without hiccups and nothing more, otherwise you're paying for nothing.

    If you're going to overprovision anything, overprovision your NICs if getting full gigabit speed is essential to you. If you must have 1000Mbps on your WAN, then you probably want a >1GbE NIC.
    If you're OK with ~950Mbps on your WAN, then just about any good intel NIC will probably work for you (PRO/1000, i340, i350).

    I'll bet you can get what you asked for with a passively cooled modern celeron.

    I don't know why you wouldn't be able to use the compact flash card you already have as install media? You probably don't reboot this thing very often so who cares if it's slow to boot? You can greatly mitigate writes to it with a RAM disk. It's true that nanoBSD is being slowly phased out, but behind it comes ZFS. You should be fine on the CF card you already have or just about any other version of flash media you can install to.



  • The AT&T gateway I got the speeds in the screenshot is a Pace  5268AC and I am trying to find out what hardware is in it but haven't been able to find any white paper on it.  Logic tells me if my router hardware was comparable to what it has then I should be fine. The only I do I can find is it has a "dual core processor with hardware accelerated routing"



  • @pfBasic:

    If you must have 1000Mbps on your WAN, then you probably want a >1GbE NIC.

    What good would an expensive >1 GbE NIC do when the ISP end is a 1 GbE port?

    Are there any ISPs that give you > 1 GbE CPEs?


  • Banned

    I don't know why, but due to ambiguous "headroom" issues, users often can't achieve full gigabit connections with gigabit NIC on WAN.

    I don't think it's an important difference, just don't be surprised if you can only achieve something like ~950Mbps on a gigabit WAN  NIC even if you get ~980-1000Mbps on your modem.



  • @pfBasic, unfortunately you didn't answer my questions.

    As far as I know even a 10 GbE NIC will connect at 1 GbE when that is what's at the modem (as in this case) so I still don't understand your advice to "overprovision your NICs". In my opinion that would be a waste of money that could be better used elsewhere.


  • Banned

    I didn't advise to overprovision the NIC, in fact my advice was to make a point of not overprovisioning anything.

    I was only suggesting that if you must overprovision something, do it with your NIC, not your CPU, because I think you can get a very cheap CPU that will do gigabit  WAN without packages to a home sized network.  butstill don't think overprovision anything is a good idea.

    My recommendation would be a modern low end passively cooled Celeron and a used i340-tx, install media didn't really matter so use whatever you have, use whatever RAM you have, use whatever PSU you have, if you don't have the above then install to cheap flash or a cheap ssd, pico PSU is nice because there's no fan, but probably not worth it if you already have a PSU.



  • @pfBasic:

    I was only suggesting that if you must overprovision something, do it with your NIC…

    And that's the reason I asked you what benefit that overprovisioning would give in the situation of the OP. I also asked if there are any ISPs at all that's offering a CPE where overprovisioning the NIC would be relevant. So far you've avoided answering those questions.

    My theories on the subject may be wrong but then please educate me.

    Many people come to an official forum like this to learn and get advice and if in hindsight a suggestion wasn't very wise, in my opinion it's best if the person that originally made the suggestion have the courage to admit that and change the suggestion…


  • Banned

    First off, I'm not avoiding your questions. The first thing I wrote in response to your questions was your answer, "I don't know why".

    @pfBasic:

    I don't know why

    Second, once again I do not recommend overprovisioning anything to include the NIC.

    I already told you why I even mentioned the NIC, then told you that in my opinion it isn't worthwhile and that a decent used Intel gigabit NIC is what I recommend.

    I'm not an expert by any stretch of the imagination, these are just my opinions based on things I've read in this forum. It's totally fine if you disagree, I wouldn't be the least bit surprised if you know better than me.

    But you can't disagree with me that the OP should overprovision his NIC, because I don't think he should either.  ;)

    I apologize that my comments are confusing to you. Let's just let this exchange die so the OP can get some more useful commentary on his question.



  • I appreciate all the discussion and find it interesting and helpful.  For the record I would be happy to reach 900's with a new firewall hardware build sine my issue at the moment is my firewall CPU pegging at 450Mb/s.  I tried to order a qotom with a core i5 from Amazon then the seller in Hong Kong messaged me saying there was a mistake with the price and he revised it and asked me to cancel and reorder. I refused and demanded they fulfill my order since it's crap to say there was a mistake and then raise the price $70 and expect me to reorder. If they do that process it I'm going to file a complaint against them with amazon and just build my own itx box.



  • @authenticx:

    The AT&T gateway I got the speeds in the screenshot is a Pace  5268AC and I am trying to find out what hardware is in it but haven't been able to find any white paper on it.  Logic tells me if my router hardware was comparable to what it has then I should be fine. The only I do I can find is it has a "dual core processor with hardware accelerated routing"

    I have that same Pace gateway.  There's really not much you can do about it, sadly. AT&T has us locked down hard to their hardware.  You can't truly bridge it, but the DMZ pinhole mode seems to be the next best thing.  Other than that there's really not much point comparing the hardware in it to what you have or may build in a pfSense box since it's not running a general purpose OS like FreeBSD.  I do know that it apparently doesn't suffer from the same low total connections in the state table as some of the previous AT&T hardware (due to RAM limitations, I gather).  That's important because of the lack of bridge mode.  Yeah, your pfSense box gets the public IP, but the Pace is still doing NAT and therefore every connection in the state table in pfSense (at least those that are destined for the WAN) has to also exist in the state table of the Pace. Yuck.



  • Would either of these devices push 1Gb?  I am looking at both trying to decide which would be better.  Please any advice is appreciated.

    Do you need PPPoE? If not that hardware named in the next line is able to archive nearly 1 GBit/s (~936 MBit/s) and plus
    the TCP/IP overhead it is "nearly" 1 real GBit/s. Y guy from hongkong reported back that his FTTH connection is running
    fine with 1 GBit/s over that units and others where doing measuring (~920 & 908 MBit/s) nearly his reported ~936 MBit/s
    he was archiving.

    • Jetway NF9HG-2930 ~$200
    • 32 GB mSATA ~$40
    • M350 case ~$45
    • 8 GB RAM ~$50
    • PSU ~$15
      = ~$360 and this is nearly your budget

  • Banned

    @BlueKobold:

    Would either of these devices push 1Gb?  I am looking at both trying to decide which would be better.  Please any advice is appreciated.

    • Jetway NF9HG-2930 ~$200

    Whoa! $200 for a 3 year old celeron and 4 NICs?! What on earth justifies that!? Small form factor? That's absurd. If you must have embedded and are willing to pay hundreds more for a little bit smaller case, then supermicro is rolling out current generation celeron boards for about $20 more.



  • @BlueKobold:

    Do you need PPPoE?

    No PPPoE required with AT&T service.



  • Whoa! $200 for a 3 year old celeron and 4 NICs?!

    Stop kidding me please, it is a industrial board from Jetway with support to 2019 and no consumer grade hardware
    it is also able to deliver the asked speed for and come ith two miniCPie and one SIM slot and 4 Intel based NICs!
    Absolutely fan less and silent! What is interesting me how old it is! Where was someone asking for the newest CPU?

    What on earth justifies that!?

    • the quality of the board (industrial)
    • fan less, quiet and silent
    • it delivers the right and wished power
    • 100% surely running pfSense without problems (reported)
    • It is reported as a board that is able to achieve the asked speed for!

    Small form factor? That's absurd.

    What!? Because you said, or why?

    If you must have embedded and are willing to pay hundreds more for a little bit smaller case, then supermicro is rolling out current generation celeron boards for about $20 more.

    The best one and cheapest should be in my eyes the APU2C4, but only with;

    • max. 4 GB RAM
    • only 3 LAN Ports
    • only at 1,2GHz CPU
    • not powerful enough to route 1 GBit/s

    And this is able to buy here in Germany for around ~220 Euros as a whole bundle from the Varia-Store.
    Case & PSU & 32 GB mSATA but not strong enough for 1 Gbit/s as said!

    No PPPoE required with AT&T service.

    Thanks this was what I was not knowing here.



  • @BlueKobold:

    The best one and cheapest should be in my eyes the APU2C4, but only with;

    • max. 4 GB RAM
    • only 3 LAN Ports
    • only at 1,2GHz CPU
    • not powerful enough to route 1 GBit/s

    Who needs more than 4GB RAM on a firewall? Anyway, the elephant in the room here is that linux can route 1gbps on that hardware just fine–the issue is scalability limits in freebsd/pf...


  • Banned

    @BlueKobold:

    …it is a industrial board from Jetway... ...and no consumer grade hardware...
    ...two miniCPie and one SIM slot and 4 Intel based NICs...

    ...- the quality of the board (industrial)...

    ...- 100% surely running pfSense without problems (reported)...
    ...- It is reported as a board that is able to achieve the asked speed for!...

    • Industrial is just a buzz word (and an expensive one too) unless you are using it in an industrial environment, maybe the OP is, but I doubt it since he didn't mention it

    • It sounds like the OP is looking for consumer, not industrial. Maybe not though?

    • Why is miniPCIe important? Unless you have an intended purpose for it you're just paying for something you don't need (like industrial hardware)

    • SIM slot is almost certainly useless to the OP, one more thing to pay for and not use

    • Integrated NIC's are too expensive with so many high quality server pulls online,why use them unless you have a specific need to do so?

    • You would have a harder time finding hardware that pfSense won't run on than hardware it will run on…

    • If that board is reported to do gigabit speeds, all that means is that just about anything >Celeron N2930 w/ gigabit intel NICs can do gigabit

    @BlueKobold:

    If you must have embedded and are willing to pay hundreds more for a little bit smaller case, then supermicro is rolling out current generation celeron boards for about $20 more.

    The best one and cheapest should be in my eyes the APU2C4, but only with;

    They now also have N4200 & E3940 boards, all of which are cheaper than the jetway.
    I wouldn't recommend any of them either unless the OP really is looking for industrial hardware.

    Too many people get years of good use out of old consumer grade hardware that already had years of use on it before it ever saw pfSense to justify recommending expensive industrial quality products.

    If you are getting paid to setup pfSense for a customer, sure buy industrial but in that case just buy the supported stuff that pfSense sells! In just about all other cases, consumer grade will last longer than you need it to for a fraction of the cost.

    Your recommendations are great for someone who is looking to use all of the features you touted, but not for someone who isn't.

    A lot of people (probably most) look to pfSense because they can get industrial grade performance without the price. Only to be recommended to buy industrial grade hardware for non-industrial purposes, over and over again.  :o

    If price were no factor then there would not be much room on the market for pfSense to have moved in with giants like Cisco already in place.



  • In my memory in college in China around 1999, the teachers in network center use a FreeBSD machine with pentium 166 as a BGP router for the whole campus. At that time, FreeBSD is perfect for network already. pfSense is based on FreeBSD.



  • @newabc:

    In my memory in college in China around 1999, the teachers in network center use a FreeBSD machine with pentium 166 as a BGP router for the whole campus. At that time, FreeBSD is perfect for network already. pfSense is based on FreeBSD.

    Since that's a single core system almost certainly not running at gigabit speeds, it's not particularly relevant: scaling efficiently across multiple cores is hard.


  • Banned

    @VAMike:

    the elephant in the room here is that linux can route 1gbps on that hardware just fine–the issue is scalability limits in freebsd/pf...

    @newabc:

    In my memory in college in China around 1999, the teachers in network center use a FreeBSD machine with pentium 166 as a BGP router for the whole campus. At that time, FreeBSD is perfect for network already. pfSense is based on FreeBSD.

    I think he was commenting on that? Which btw, are there any long term plans to upgrade PF in FreeBSD to address this?

    EDIT: answered my own question

    https://www.netgate.com/blog/further-a-roadmap-for-pfsense.html

    pfSense software version 3.0 is a longer-term project. pfSense 3.0 is a major re-write consisting of 4 major components…

    ...Third, the core of pfSense (pf, packet forwarding, shaping, link bonding/sharing, IPsec, etc) will be re-written using Intel’s DPDK...

    ...We have a goal of being able to forward, with packet filtering at rates of at least 14.88Mpps. This is “line rate” on a 10Gbps interface. There is simply no way to use today’s FreeBSD (or linux) in-kernel stacks for this type of load. Since this work is only available on certain, select Ethernet cards (mostly 1Gbps/10Gbps/40Gbps Intel interfaces as well as various VMware and Xeon ‘virtualization’ NICs. Other vendors, including Broadcom, Myrianet, Chelsio and Cisco have shown interest. This also means that the underlying kernel and system will be 64-bit only...

    https://www.netgate.com/blog/pfsense-around-the-world-better-ipsec-tryforward-and-netmap-fwd.html

    Back in February, I wrote a blog post that discussed our plans for pfSense software version 2.3, which is now in alpha, and our plans for pfSense 3.0. While I promoted DPDK then, we’ve since found that netmap provides a simpler API, and substantially better safety, as the device drivers remain in the kernel, rather than running in userspace with DPDK. Still, DPDK provides a set of libraries, such as longest-prefix match, which uses a variation of the DIR-24-8 algorithm for routing lookups, which we should find useful in our pursuit of the ultimate open source software router.