PfSense box questions
-
Hello all,
Seems my router left me last night and may be looking to file for divorce :(. After all the abuse I have put the router through I can understand why it would to go this route.
I am looking to build a pfSense box, gain performance, gain control, and save money in the long run. I have done a bit of research but am still unsure on the best way to approach building this box, please post any suggestion or opinion you may have that could help me, I will appreciate it!
Speeds - 100/35 going 24/7 and VPN ( I need to prepare for speeds increased to 300+ down and 50+ up )
Users - Dozen'ish
Ports - Using about 8 with two switches ( Switches in separate locations )
WiFi - Will need some wifi, nothing of insane quality just N speeds/range with some decent antennas ( Not sure if a a cheap AP might be more convenient )
Budget - Up to $150ish Can add if needed, or consider this the budget without shipping costs ( I looked at the Zotac zbox ci321 and a smart/managed switch but I felt for close to $200 I could build something better, maybe not )Items
Case - http://www.mini-box.com/M350-enclosure-with-picoPSU-80-and-60W-adapter ( $70 ) - I am assuming I should avoid the $10 Pico psu's from China on ebay so this seemed like a good way to save on shipping costs from buying the enclosure and psu separately.
Wifi - Must research
Storage - Might have an SSD around or use USB if possible
Ports - Looking to get a quad port nic for the pci/pci-e expansion and wire 2servers and 2switches onto the nic
Cpu-I've singled down to: Atom D525, Atom D2500 / D2700 , Celeron J1900 ( Not sure if Bay trail still has issues, can't recall what I read at the moment ), Celeron N2930 , Celeron N3150
Motherboard- Whichever the best deal is price wise or feature wise. I've chosen the CPU above because my plan is to use the pci/pci-e to install a quad port nic. The jetway boards I've seen all are close to $200 and I can save a lot of money buying a board and quad port nic seperately.I understand I can benefit from AES instructions on the CPU. Are all these chips overkill? Should I focus power consumption first or look for highest frequency single threaded chip with largest amount of cache. I've been using cpuboss to compare some of the chips specs but I am unaware if the other CPU instructions will benefit any features in pfSense. I may be able to snag a D525 or D2500/D2700 for really cheap(I lose AES instructions, otherwise I ditch the Atom and look for Celerons. Should I ignore memory bandwidth speeds or will there be any benefit from something such as the N2930 12,800MB/s vs N3150 24,600MB/s ?
Looking to squeeze the most performance out of my budget, soon there will be more devices on the network and the speeds will be more than double. For this reason I'd like to avoid building around a 100mbit box for basic NAT routing. Really would like to have room to use some features and not worry about any hardware constraints. I've seen some pre-built systems on several m itx stores but they are pricey. Not to mention it takes the fun out of building the box myself!
I'm definitely in need of some help as you can see, I'll be reading around more on this forum and hopefully through some of your advice. Thanks in advance to anyone willing to help me.
-
You're probably going to have to search hard for some used equipment or make some hardware compromises (Realtek) to achieve your budget goal.
If form factor isn't an issue and if power consumption is secondary I'd try to find a used desktop that someone is pitching and add a 4 port NIC and a cheap router set up as an access point. That would meet your stated needs and financial goal. I'm using 13 year old desktop components for my setup, but my bandwidth needs are modest.
-
Take a look at this thread: https://forum.pfsense.org/index.php?topic=109121.0
You can get just the board for around $175, but you need to then put a case on it.
Doing tests, I could get ~1Gb with VPN, this was tested internally.
Does around 10Watts as far as power consumption. With lower end speeds, I have seen near in the 90% range over VPN, so low speed loss with encryption.
-
You're probably going to have to search hard for some used equipment or make some hardware compromises (Realtek) to achieve your budget goal.
If form factor isn't an issue and if power consumption is secondary I'd try to find a used desktop that someone is pitching and add a 4 port NIC and a cheap router set up as an access point. That would meet your stated needs and financial goal. I'm using 13 year old desktop components for my setup, but my bandwidth needs are modest.
I'm definitely looking to score on used equipment. Found some very cheap atom boards but some listings are questionable and I have to contact seller in attempt to try and save time and money dealing with possible DOA parts. I understand I could do wonders with old hardware, but I'm set on keeping m-itx as form factor and considering power consumption after. I thought about getting rid of the m-itx enclosure and going DIY with a tupperware case to save the $40 ( will buy enclosure at a more convenient time ).
Finding a decent PCI quad port nic to allow me to take advantage of super cheap atom boards is becoming more frustrating. I'll have to go through that pfSense hardware list and see if anything decent is on ebay at the moment. An example of that frustration is how I can buy a used $35 D525 easily and struggle to find a PCI nic but I can buy a new ASRock N3150B-ITX for $72 and easily buy a $30-60 PCIe nic while scraping by on the rest. This leaves me with cheapest pico, cheapest ram, and a salvaged AP. The lack of AES on the atom isn't helping and something I need to look into. If throughput will be greatly increased with AES instructions I may have to avoid atom all together.
Take a look at this thread: https://forum.pfsense.org/index.php?topic=109121.0
You can get just the board for around $175, but you need to then put a case on it.
Doing tests, I could get ~1Gb with VPN, this was tested internally.
Does around 10Watts as far as power consumption. With lower end speeds, I have seen near in the 90% range over VPN, so low speed loss with encryption.
Will be checking this out!
Thanks for taking time to guide me, I know these builds can be simple, super cheap, and get the job done. I'm probably more of a complication than obtaining the actual 'required' hardware. I'll be searching ebay and reading forums more when I'm home from work.
-
Hello all,
Seems my router left me last night and may be looking to file for divorce :(. After all the abuse I have put the router through I can understand why it would to go this route.
I am looking to build a pfSense box, gain performance, gain control, and save money in the long run. I have done a bit of research but am still unsure on the best way to approach building this box, please post any suggestion or opinion you may have that could help me, I will appreciate it!
Speeds - 100/35 going 24/7 and VPN ( I need to prepare for speeds increased to 300+ down and 50+ up )
Users - Dozen'ish
Ports - Using about 8 with two switches ( Switches in separate locations )
WiFi - Will need some wifi, nothing of insane quality just N speeds/range with some decent antennas ( Not sure if a a cheap AP might be more convenient )
Budget - Up to $150ish Can add if needed, or consider this the budget without shipping costs ( I looked at the Zotac zbox ci321 and a smart/managed switch but I felt for close to $200 I could build something better, maybe not )Items
Case - http://www.mini-box.com/M350-enclosure-with-picoPSU-80-and-60W-adapter ( $70 ) - I am assuming I should avoid the $10 Pico psu's from China on ebay so this seemed like a good way to save on shipping costs from buying the enclosure and psu separately.
Wifi - Must research
Storage - Might have an SSD around or use USB if possible
Ports - Looking to get a quad port nic for the pci/pci-e expansion and wire 2servers and 2switches onto the nic
Cpu-I've singled down to: Atom D525, Atom D2500 / D2700 , Celeron J1900 ( Not sure if Bay trail still has issues, can't recall what I read at the moment ), Celeron N2930 , Celeron N3150
Motherboard- Whichever the best deal is price wise or feature wise. I've chosen the CPU above because my plan is to use the pci/pci-e to install a quad port nic. The jetway boards I've seen all are close to $200 and I can save a lot of money buying a board and quad port nic seperately.I understand I can benefit from AES instructions on the CPU. Are all these chips overkill? Should I focus power consumption first or look for highest frequency single threaded chip with largest amount of cache. I've been using cpuboss to compare some of the chips specs but I am unaware if the other CPU instructions will benefit any features in pfSense. I may be able to snag a D525 or D2500/D2700 for really cheap(I lose AES instructions, otherwise I ditch the Atom and look for Celerons. Should I ignore memory bandwidth speeds or will there be any benefit from something such as the N2930 12,800MB/s vs N3150 24,600MB/s ?
Looking to squeeze the most performance out of my budget, soon there will be more devices on the network and the speeds will be more than double. For this reason I'd like to avoid building around a 100mbit box for basic NAT routing. Really would like to have room to use some features and not worry about any hardware constraints. I've seen some pre-built systems on several m itx stores but they are pricey. Not to mention it takes the fun out of building the box myself!
I'm definitely in need of some help as you can see, I'll be reading around more on this forum and hopefully through some of your advice. Thanks in advance to anyone willing to help me.
I've just built a N2930 with 4 x Intel NIC onboard platform with budget slightly < US$250, you might take a look here (with lots of pics)
Not sure how many NICs you need, if you only need 2-3, you might consider this Intel 2500CCE, Atom D2500 with 2 x Intel NIC onboard, with Mini PCI-e slot for a WiFi connectivity, and if you want 3rd NIC you can definitely add a PCI (not PCI-e) Intel NIC. According to some forum reports about the Intel D525 performance (D2500 is kind of similar thing), 500-600Mbps NAT throughput shall not be difficult to achieve.
AES instructions is useful if you want a decent VPN throughput, but from what I see from the throughput requirement, even without hardware acceleration there shouldn't be any issue with the above mentioned board.
-
As @edwardwong was reporting the Board and the CPU (SoC) is for around ~$200 but it will be only
sorted with an front power jack that allows you to save the money for the PicoPSU and go alone with
the M350, 2 x 4 GB and mSATA drive. But for many users and more throughput go with a Intel Core
i3 or Core i5 with 4 cores @3,0GHz, a SSD and 8 GB RAM too. It lets you archive more throughput
for less money and you can run more packets on top of this. -
Thanks again to those who helped me, I have a few more questions and concerns that are keeping me from finalizing the list and ordering.
For AES instructions on a 300mbps connection, without am I looking at 10-20% throughput less/more? For example, with AES 50% of speeds and without AES 10-20% of speeds. I'm curious on the difference so I can decide between atom and celeron.
Another question, if I choose a PCI based atom bored am I going to shoot myself in the foot when I expect a quad-port gigabit nic to provide maximum LAN throughput on a 133MB/s (1064mbps) PCI bus or am I missing something here? I can find some great deals on d525/d2500/d2700 if that is not the case. I did find a x4 PCI-e atom based board, not the cheapest but it would be a solution to LAN throughput if PCI is a problem. Ultimately I can only choose the atom if the lack of AES instructions isn't a massive loss.
I'm encountering the same problem for celeron, asus n3150 board has a PCI-e x4 slot in x1. Will I still be limited, just not as bad as with PCI?
I've been looking around, I have many item lists based on atom/celeron but I need to understand this then choose the route. Everything is within the $100-180 range, parts pieced together, buying new from stores like newegg, and even adding $20-30 starts including various jetway boards. Came across this supermicro board SUPERMICRO MBD-X7SPA-H-O, will I see a noticeable performance loss within features running DDR2?
Ended up re-writing my post because all the other questions I had asked are somewhat based around AES/PCI bus.
-
The SUPERMICRO MBD-X7SPA-H-O is here able to get for ~220 € and a Supermicro A1SRi-2558 is able to
get for something about ~250 € so I would be really deciding then, build this appliance step by step and
go with the more powerful and better sorted Supermicro board if I am would be in your situation.For AES instructions on a 300mbps connection, without am I looking at 10-20% throughput less/more? For example, with AES 50% of speeds and without AES 10-20% of speeds. I'm curious on the difference so I can decide between atom and celeron.
Together with IPSec the SG-4860 will be able to hit 500+ MBit/s over IPSec and this device is similar to
A1SRi-2558 board. So if you are using IPSec here are the numbers for that action using AES-NI.Another question, if I choose a PCI based atom bored am I going to shoot myself in the foot when I expect a quad-port gigabit nic to provide maximum LAN throughput on a 133MB/s (1064mbps) PCI bus or am I missing something here? I can find some great deals on d525/d2500/d2700 if that is not the case. I did find a x4 PCI-e atom based board, not the cheapest but it would be a solution to LAN throughput if PCI is a problem. Ultimately I can only choose the atom if the lack of AES instructions isn't a massive loss.
As today I would not go together with PCI twi or quad port cards and GB LAN. Go with the PCIe slot and you
will be on the safe side. The ASRi-2558 is offering a real PCIe 2.0 x8 slot and so there will be not problem
with it.I'm encountering the same problem for celeron, asus n3150 board has a PCI-e x4 slot in x1. Will I still be limited, just not as bad as with PCI?
If you want to walk the "budget" way you could go with an Intel Celeron G3260 @3,2GHz and there are mostly
also PCIe 2.0 x4 or x8 slots on that boards and fast memory on top of this.I've been looking around, I have many item lists based on atom/celeron but I need to understand this then choose the route. Everything is within the $100-180 range, parts pieced together, buying new from stores like newegg, and even adding $20-30 starts including various jetway boards. Came across this supermicro board SUPERMICRO MBD-X7SPA-H-O, will I see a noticeable performance loss within features running DDR2?
The Jetway board is delivering perhaps enough speed for the WAN part but with each other installed packet or
enabled or offered service it will be going much more down in performance. If the use case above from the
opening post is right, it should be a really strong sorted and powerful appliance and not from the lower end
based Atom SoCs from Intel, with PCI and slower RAM, it should be DDR3 RAM with 1333MHz or better DDR3
RAM with 1600MHz or 1866MHz and not under 8 GB better more then less with an looking eyes towards to
the version 2.3.x then you would not running in any trap, some head room is also given and some spare
horse power will not be lets you in some month find out that you go underperformed. If at this time not
enough money is there please save money for 6 month and perhaps you will be able to go with a used or
refurbished Intel Core i3 or i5 with 4 CPU cores and much cheap RAM and a sufficient board. -
From what I see from OP's need, "future" means around 300Mbps, for the D510, or N2930, both are fine with the throughput value.
Yes other package loading does matter, so D510 might have some trouble if you want to run something like Squid+ClamAV (decompression speed depends on CPU speed), but for N2930, as a 700Mbps up/down WAN-LAN NAT only eats up about 50-60% CPU, considering to your bandwidth requirement, most likely it will be less than 40% for WAN-LAN NAT, and then the remaining processing power will be good for other usage.For large amount user environment, most likely Squid is a must (I did once many years ago, with Squid cache I can serve 150-200 office users with 5M + 2M ADSL), and the NAT is not really getting too much load because of the local cache. So I do recommend to try out with Silvermont Celeron, or Rangeley Atoms.
-
@BlueKobold:
The Jetway board is delivering perhaps enough speed for the WAN part but with each other installed packet or
enabled or offered service it will be going much more down in performance. If the use case above from the
opening post is right, it should be a really strong sorted and powerful appliance and not from the lower end
based Atom SoCs from Intel, with PCI and slower RAM, it should be DDR3 RAM with 1333MHz or better DDR3
RAM with 1600MHz or 1866MHz and not under 8 GB better more then less with an looking eyes towards to
the version 2.3.x then you would not running in any trap, some head room is also given and some spare
horse power will not be lets you in some month find out that you go underperformed. If at this time not
enough money is there please save money for 6 month and perhaps you will be able to go with a used or
refurbished Intel Core i3 or i5 with 4 CPU cores and much cheap RAM and a sufficient board.I can see the flaws in my original plan and realize that I may as well spend a little bit more now instead of build an outdated box for this network. The performance trade off for power consumption is too large. Form factor also was a limiting factor.
Looking for a deal on either i3-3xxx / t (ivy bridge AES), i3-4xxx / t ( haswell AES & ECC ), and pentium g4400 ( AES & ECC ). While I don't have a thorough understanding of many things, but IIRC pfsense can utilize multiple cores/threads. I am still aiming for highest frequency and largest cache. The power consumption on these chips is not too high, especially when I'm expecting every drop of performance and previously thought I could achieve that at 6w.
i3 ivy bridge/haswell or pentium g3/g4 $50-120
motherboard $50
ram $15 / ecc ram $30 ( $20 more for a haswell i3 vs ivy bridge i3 so I figured why not ECC )
case - have
power supply - have
storage - live usb, ssd laterStill close to budget, just closer to 10x the power consumption of that 6w celeron chip ;D. Not sure if I'll go with a full atx or micro atx motherboard but I will just make use of the case and power supply I do have. I can boot off a live usb if I'm not mistaken and that will be fine until I get an SSD for the box. Thanks again to all who have helped, I think I won't be re-considering atom/celeron SOC boards and going with a socket motherboard. It just seems better this way, especially if I re-purpose the machine in the future, sell, or upgrade parts.
From what I see from OP's need, "future" means around 300Mbps, for the D510, or N2930, both are fine with the throughput value.
Yes other package loading does matter, so D510 might have some trouble if you want to run something like Squid+ClamAV (decompression speed depends on CPU speed), but for N2930, as a 700Mbps up/down WAN-LAN NAT only eats up about 50-60% CPU, considering to your bandwidth requirement, most likely it will be less than 40% for WAN-LAN NAT, and then the remaining processing power will be good for other usage.For large amount user environment, most likely Squid is a must (I did once many years ago, with Squid cache I can serve 150-200 office users with 5M + 2M ADSL), and the NAT is not really getting too much load because of the local cache. So I do recommend to try out with Silvermont Celeron, or Rangeley Atoms.
Am I not simply shooting myself in the foot buying a PCI motherboard and expecting full duplex gigabit LAN on 4ports on a 1064mbps PCI bus? Seemed like I ran into similar problem with PCIe 2.0 x1 slot on the n3150 SOC.
Thanks again edwardwong and BlueKobold, If you or anyone have any last suggestions based on the list above please let me know. Open to suggestions, but even more if I am misunderstanding some of the things I will appreciate being corrected/informed.
-
i3 ivy bridge/haswell or pentium g3/g4 $50-120
motherboard $50
ram $15 / ecc ram $30 ( $20 more for a haswell i3 vs ivy bridge i3 so I figured why not ECC )
case - have
power supply - have
storage - live usb, ssd laterStill close to budget, just closer to 10x the power consumption of that 6w celeron chip ;D. Not sure if I'll go with a full atx or micro atx motherboard but I will just make use of the case and power supply I do have. I can boot off a live usb if I'm not mistaken and that will be fine until I get an SSD for the box. Thanks again to all who have helped, I think I won't be re-considering atom/celeron SOC boards and going with a socket motherboard. It just seems better this way, especially if I re-purpose the machine in the future, sell, or upgrade parts.
Where do you find a $50 motherboard which supports ECC memory? Even your CPU supports ECC, your motherboard has to support this as well.
And you forget to add in the Quad Port Ethernet controller.
There are certain benchmark showing that Atom C2758/C2750 is faster than some of the i3 processor, and the motherboard is definitely supporting ECC memory, and….if you choose Supermicro, you already have decent Intel LAN chip x 4 onboard, you don't need to buy anymore! (For your application, I would say the little brother C2558/C2550 is already doing a great job) And you can enjoy a much lower power consumption pattform.From what I see from OP's need, "future" means around 300Mbps, for the D510, or N2930, both are fine with the throughput value.
Yes other package loading does matter, so D510 might have some trouble if you want to run something like Squid+ClamAV (decompression speed depends on CPU speed), but for N2930, as a 700Mbps up/down WAN-LAN NAT only eats up about 50-60% CPU, considering to your bandwidth requirement, most likely it will be less than 40% for WAN-LAN NAT, and then the remaining processing power will be good for other usage.For large amount user environment, most likely Squid is a must (I did once many years ago, with Squid cache I can serve 150-200 office users with 5M + 2M ADSL), and the NAT is not really getting too much load because of the local cache. So I do recommend to try out with Silvermont Celeron, or Rangeley Atoms.
Am I not simply shooting myself in the foot buying a PCI motherboard and expecting full duplex gigabit LAN on 4ports on a 1064mbps PCI bus? Seemed like I ran into similar problem with PCIe 2.0 x1 slot on the n3150 SOC.
Thanks again edwardwong and BlueKobold, If you or anyone have any last suggestions based on the list above please let me know. Open to suggestions, but even more if I am misunderstanding some of the things I will appreciate being corrected/informed.
4 x Gigabit LAN on single PCI slot of course not an option, that's why we need to know how many you need.
For the Intel D2500CCE, or Asrock AD2550R/U3S3, they already have 2 x Intel LAN onboard, plus 1 single port PCI GbE (for Intel 2500CCE), or a dual port pci-e GbE card (for Asrock AD2550R/U3S3), then you'll get 3 or 4 NIC for your system. Try to look into their specification and you might have some surprise. -
i3 ivy bridge/haswell or pentium g3/g4 $50-120
motherboard $50
ram $15 / ecc ram $30 ( $20 more for a haswell i3 vs ivy bridge i3 so I figured why not ECC )
case - have
power supply - have
storage - live usb, ssd laterStill close to budget, just closer to 10x the power consumption of that 6w celeron chip ;D. Not sure if I'll go with a full atx or micro atx motherboard but I will just make use of the case and power supply I do have. I can boot off a live usb if I'm not mistaken and that will be fine until I get an SSD for the box. Thanks again to all who have helped, I think I won't be re-considering atom/celeron SOC boards and going with a socket motherboard. It just seems better this way, especially if I re-purpose the machine in the future, sell, or upgrade parts.
Where do you find a $50 motherboard which supports ECC memory? Even your CPU supports ECC, your motherboard has to support this as well.
And you forget to add in the Quad Port Ethernet controller.
There are certain benchmark showing that Atom C2758/C2750 is faster than some of the i3 processor, and the motherboard is definitely supporting ECC memory, and….if you choose Supermicro, you already have decent Intel LAN chip x 4 onboard, you don't need to buy anymore! (For your application, I would say the little brother C2558/C2550 is already doing a great job) And you can enjoy a much lower power consumption pattform.From what I see from OP's need, "future" means around 300Mbps, for the D510, or N2930, both are fine with the throughput value.
Yes other package loading does matter, so D510 might have some trouble if you want to run something like Squid+ClamAV (decompression speed depends on CPU speed), but for N2930, as a 700Mbps up/down WAN-LAN NAT only eats up about 50-60% CPU, considering to your bandwidth requirement, most likely it will be less than 40% for WAN-LAN NAT, and then the remaining processing power will be good for other usage.For large amount user environment, most likely Squid is a must (I did once many years ago, with Squid cache I can serve 150-200 office users with 5M + 2M ADSL), and the NAT is not really getting too much load because of the local cache. So I do recommend to try out with Silvermont Celeron, or Rangeley Atoms.
Am I not simply shooting myself in the foot buying a PCI motherboard and expecting full duplex gigabit LAN on 4ports on a 1064mbps PCI bus? Seemed like I ran into similar problem with PCIe 2.0 x1 slot on the n3150 SOC.
Thanks again edwardwong and BlueKobold, If you or anyone have any last suggestions based on the list above please let me know. Open to suggestions, but even more if I am misunderstanding some of the things I will appreciate being corrected/informed.
4 x Gigabit LAN on single PCI slot of course not an option, that's why we need to know how many you need.
For the Intel D2500CCE, or Asrock AD2550R/U3S3, they already have 2 x Intel LAN onboard, plus 1 single port PCI GbE (for Intel 2500CCE), or a dual port pci-e GbE card (for Asrock AD2550R/U3S3), then you'll get 3 or 4 NIC for your system. Try to look into their specification and you might have some surprise.I will double back on everything.
I did forget the NIC, have to add $15-30 on that list. Honestly depends on how much patience I have to find the best deal, sometimes I'll pay more to avoid the hassle of searching, same with shipping charges.
I'm aware the motherboard needs to support ECC. I'm not crazy ;D but I did find some crazy deals that I will not link for obvious reasons. If you're actually interested I can private message you them. Seems our pricing may vary due to region. When you referred to the MBD-X7SPA-H-O being around ~220€ I can find used and working for under ~$100 USD.The layout for NIC would be the following:
1x WAN ( onboard )
1x NIC ( server 24/7 use of gigabit )
1x NIC ( server2 24/7 use of gigabit )
1x NIC ( switch location1 up to half dozen users, up to gigabit speeds, not 24/7 )
1x NIC ( switch location2 up to half dozen users, up to gigabit speeds, not 24/7 )I have to guarantee both servers have the full speeds, users on switch might not need/use 24/7 but it's the instance they do which will be my problem. If this doesn't make sense or you want to know specifics past I can share through private message.
Also thanks again for helping along the way, great community here with much information. Unfortunately time is always a constraint and when I do get a few hours to search parts or performance specs the hardware changes. At this point I'm pretty sure it's either what you or I have listed but it's shaving off the want vs need. I definitely don't need ECC but for the $20 more on processor and finding working boards at a fraction the cost it becomes too easy to justify the "need".
Hope you all had a good weekend, also if anyone who is reading this has been in my situation and can share experiences I would be very interested to hear from you.
-
Oh, I forgot there is another board, X10SBA-L which might also suit your need, this board is using J1900 quad-core processor, running much faster than D525/D2500/D2550, definitely no issue on 1G NAT throughput, Amazon US is selling around US$150-170 only.
This board also has 2 x i210 LAN, with a PCI-E x2 (mechanically x8) slot you can add in a quad port ethernet card without any issue (then you can get 6 ports)
