Hardware questions & recommendations - 100Mbps and beyond
-
Hello all –
I'm currently running a Snapgear SG560. It's a nice little box, but I've got a itch to build a DIY router with pfSense. The connection here is a 50/50 fiber but will be upgraded to a 100/100 soon and 1G/1G is available for a very reasonable price so it's in my scopes as well. My current setup is fiber -> Snapgear LAN0 -> LAN1 -> Switch. On average I have ~10 active devices on the LAN and 4-5 on the WAN (it will continue to serve wireless itself, though is connected to the switch with the router handling IPs)
With that said, I have some questions I'd like to ask:
-
Is there anything in the Snapgear SG560 features list that I'd likely miss when switching over to pfSense? I'm not seeing anything, but perhaps I'm missing something.
-
I've read that the integrated Intel 82574L based NICs may leave something to be desired especially when thinking of scaling beyond 100Mbps. Is a PCIe NIC such as the PWLA8492MT something I should be looking at using? At 100/100? At 1Gbps/1Gbps?
-
From what I've gathered on the forums here a Intel Atom based solution can push up to ~550Mbps while I'll need at least a i3/1155 based system to support 1Gbps/1Gbps.
-
Can a 1Gbps system be built at a reasonable price and low power consumption more in alignment to that of the 100Mbps system? So far, I'm not seeing it. If not, I will initially build a 100Mbps supporting device and will build a new one from scratch when/if I go to 1Gbps connection
-
With all of that said, here is a rough idea I have for hardware breakdowns (note that I may be way off still; looking for feeback!).
Up to 1Gbps System:
$89 Antec ISK 300-150 Black 0.8mm cold rolled steel Mini-ITX Desktop Computer Case 150W Power Supply $74 Intel BOXDH61DLB3 LGA 1155 Intel H61 USB 3.0 Mini ITX Intel Motherboard $69 Intel Pentium G620 Sandy Bridge 2.6GHz LGA 1155 65W Dual-Core Desktop Processor Intel HD Graphics BX80623G620 $22 G.SKILL Value Series 4GB 240-Pin DDR3 SDRAM DDR3 1333 (PC3 10600) Desktop Memory Model F3-10600CL9S-4GBNT $00 Existing 80GB HDD $61 Intel PWLA8492MT PRO/1000 MT PCI/PCI-X Dual Port Server Adapter $14 Akasa Low Profile Intel CPU Cooler - Mini-ITX and Micro-ATX ( AK-CC7122BP01) (Sockets 775 / 1155 / 1156)100Mbps System (Up to ~550Mbps?):
$96 Intel D2500CCE Mini-ITX $24 G.SKILL Ripjaws Series 4GB 240-Pin DDR3 SDRAM DDR3 1066 (PC3 8500) Desktop Memory Model F3-8500CL7S-4GBRL $69 M350 enclosure with picoPSU-80 and 60W adapter $00 Existing 80GB HDDSome other notes:
-
I'd like to save as much power as possible. I calculated ~126W required for the 1Gbps system.
-
It doesn't have to be perfectly silent, but should be quiet as possible.
Lastly, sorry for the long post and thanks for reading! Any feedback whatsoever is very welcome and appreciated.
-
-
-
First, your "65 watt" Sandy Bridge CPU is the top Thermal Design, meaning whatever you decide to use for heatsink/fan should be able to dissipate at least that much, it's a maximum that should be designed for. Meaning, it doesn't run at 65 watts all the time.
Here's a nice graph of a few random full systems at idle, the G620 happens to be in that list: http://www.xbitlabs.com/articles/cpu/display/pentium-g850-g840-g620_7.html
Second, the PWLA8492MT NIC is PCI-X, not PCI-Express. Very different bus. PCI-X is an extended 64bit PCI Bus, generally only found in servers and some high end workstations. It may work in a standard PCI Bus, but then you've effectively got a standard PCI card. If you really want to push Gb, do not get that card (oh, and the board you chose doesn't have standard PCI slots, anyway.)
Current versions of pfSense (from 1.2.3 up) should support the 82574L just fine, can you point to what you were reading about speed problems? The link I see there seems to mention the 82574 as "relatively low end", but no data to back that up, just an empirical label. Otherwise there is a comment about Intel Desktop Gb Nics as a whole as a lack of "optimizations", which may or may not be relevant anyway. Not that I read through all 7 pages of that thread.
If you're really worried about it, get a PCI-Express server NIC (but you'll need a different board, which I'll get to later.) They aren't all that expensive on Ebay. Or, you could start with standard desktop NICs and upgrade if it seems like it'll make a difference.
If you're looking for a cheap and low power machine that should be able to do Gb, a used Core2Duo desktop would also work. But, at the same time, realize that if you're looking to save cash, why are you spending $90 on an Antec case? Or going mini-itx at all? Just because it's small doesn't mean it's taking less power, there's plenty of micro-atx motherboards and cases that would be of the same power draw. Plus, that board only has a single PCI-Express x1 slot, if you're going to skip the onboard NIC and go "server" NIC you need a dual port card, and most, if not all dual port cards are going to be x4 or larger.
A decent Core2Duo machine usually runs between 40 and 60 watts at idle, but "Server" grade NICs offload a lot of processing from the processor, hence can often draw a considerable amount of power on their own, possibly up to 25 watts, but probably only 5-15watts.
-
Thanks for your feeback matguy –
That's embarrassing, I linked the wrong board. Given your input however, I think I'll poke around more at integrated NIC's (I do not have any raw data about performance issues with the integrated Intel's, this is why I'm asking here).
I think at this point, I'll also go for a solution for a 100Mbps connection. When/if I upgrade to 1G/1G, I'll cross that bridge.
Given that, is the Intel D2500CCE Mini-ITX and the build listed sufficient for 100Mps with QoS+Firewall?
I'm not too worried about saving cash, I just don't like to blow money where it's not needed. I'd like a MiniITX due to size. Looks are semi-important to me (this will be visible in my office… call me crazy). The M350 looks visibly nice for a router / Atom solution though.
Is there benefit of going SSD besides perhaps boot up time?
-
Intel NUC
http://forum.pfsense.org/index.php/topic,56452.0.html -
The 82574L, per Intel's marketing, is ideal for low-power consumption work like telecommunications (http://www.intel.com/content/www/us/en/ethernet-controllers/82574l-82574it-gbe-controller-brief.html)
I'm using a Intel BOXDQ77MK LGA 1155 board with an i3-2100. I threw in 4GB RAM and two Intel EXPI9301CTBLK Network Adapters. It has a boring IN WIN BL641.300TBL Black Steel MicroATX Slim Case with a built-in 300w power supply. Because the board and the CPU are designed for some energy efficiency, and it's somewhat overkill in general, the CPU churns at 20%, it's not a bad affordable, low-power build. I think it cost me around $400 with shipping to build it, and it's been churning away for the past five months with very little issue.
All four connections to it are Gb Ethernet, and I have two 50Mbit WAN connections coming into it. The LAN connections are Gbit. Real quiet and it might be energy efficient. ;D
However, I don't have any issues at all with the 82574 NIC. I use this motherboard in my pfSense box and my VMWare box. Very happy with it.
-
Is there anything in the Snapgear SG560 features list that I'd likely miss when switching over to pfSense?
PfSense does not include either web filtering or spam blocking by default. Both these are possible by adding packages. The spam blocking package is relatively new though, I've not tried it.
Steve
-
…I think at this point, I'll also go for a solution for a 100Mbps connection. When/if I upgrade to 1G/1G, I'll cross that bridge.
Given that, is the Intel D2500CCE Mini-ITX and the build listed sufficient for 100Mps with QoS+Firewall?
I would think so.
Is there benefit of going SSD besides perhaps boot up time?
Mainly just if you're doing Squid or anything that's going to write to the drive a lot. Otherwise, it's functionally similar to a CF card (other than the translation from SATA to PATA for the CF card, unless it's an onboard CF slot, and/or onboard PATA.) Really, a CF card is just a SSD in a different format; basically a predecessor to modern SSD's. For the most part, you probably could do Squid or other frequently writing package to a sufficiently large CF card. Depending on the wear leveling capabilities it might be fine, especially for something like a 32GB card, but at that point, you're not benefiting from any cost savings.
Now, if you're looking at the benefit of an SSD over a spinning disk, aside from the obvious access time differences, spinning drives fail. Not that SSD's don't, but if you're not writing to an SSD that much, they can last damn near forever. I had a 64MB card in my m0n0wall box for 7 some odd years, it's still fine. Oh, and it wasn't a new card to begin with, it was already a few years old, probably 4 or 5. A spinning drive doesn't particularly care how much you read/write to it, as long as it's spinning it's wearing itself out. If you're not writing to the drives you could set ataidle (is that enabled by default now?) and hope they don't continually cycle between spin up and down.
-
Intel NUC
http://forum.pfsense.org/index.php/topic,56452.0.htmlI've looked at this a bit, but seems a bit too "cutting edge" for my purposes right now. I'd like most of it to just work :)
All four connections to it are Gb Ethernet, and I have two 50Mbit WAN connections coming into it. The LAN connections are Gbit. Real quiet and it might be energy efficient. ;D
Thanks for your feedback. What is the total amount of bandwidth you're pushing through this thing?
PfSense does not include either web filtering or spam blocking by default. Both these are possible by adding packages. The spam blocking package is relatively new though, I've not tried it.
Thanks for pointing that out. I don't currently use the spam or web filtering features of the Snapgear, so this is a non-issue for me.
Mainly just if you're doing Squid or anything that's going to write to the drive a lot. Otherwise, it's functionally similar to a CF card (other than the translation from SATA to PATA for the CF card, unless it's an onboard CF slot, and/or onboard PATA.) Really, a CF card is just a SSD in a different format; basically a predecessor to modern SSD's. For the most part, you probably could do Squid or other frequently writing package to a sufficiently large CF card. Depending on the wear leveling capabilities it might be fine, especially for something like a 32GB card, but at that point, you're not benefiting from any cost savings.
Now, if you're looking at the benefit of an SSD over a spinning disk, aside from the obvious access time differences, spinning drives fail. Not that SSD's don't, but if you're not writing to an SSD that much, they can last damn near forever. I had a 64MB card in my m0n0wall box for 7 some odd years, it's still fine. Oh, and it wasn't a new card to begin with, it was already a few years old, probably 4 or 5. A spinning drive doesn't particularly care how much you read/write to it, as long as it's spinning it's wearing itself out. If you're not writing to the drives you could set ataidle (is that enabled by default now?) and hope they don't continually cycle between spin up and down.
I don't think I'll be using Squid. Good point on the "spinning drives fail" portion. I'd feel much better with a SSD due to that and that alone.
-
All four connections to it are Gb Ethernet, and I have two 50Mbit WAN connections coming into it. The LAN connections are Gbit. Real quiet and it might be energy efficient. ;D
Thanks for your feedback. What is the total amount of bandwidth you're pushing through this thing?
I have 1Gb going to each LAN (10.0.1.x/24, 10.0.2.x/24), and that sees a decent amount of activity. I have two 24-port switches connected to each LAN port.
On the WANs I'll peak out at 55Mbit each down. I only get 8Mbit up on each, so those two aren't as busy as the LANs. The ISP claims the lines are 50/8.
The Q77 chipset on the motherboard provides a PCI 2.0 x1 controller for the Ethernet ports. Each Gb ethernet port has it's own PCIe 2.0 x1 lane to the PCH, and of course the PCIe Gbit boards also have a dedicated x1 and x4 lane (depending on the slot it's in, and each board is only an x1 interface). I think it has a total of 8 x PCIe 2.0 lanes.
Because I'm not using a newer CPU, I don't get PCIe 3.0 from the subsystems. And honestly I wouldn't have any use for it. However, if you wanted to use a PCIe 3.0 x16 multiport board with this motherboard, you'd want to get an Ivy Bridge i3 to take advantage of that technology, but a 4-port Gbit E card should do well in a PCIe 2.0 x16 slot.
-
After reading this thread I've had some second thoughts about a SSD? There seems to be a lot of debate. Should I go for a CF instead?
Anyway, I'm close to setting on a build. Here are a couple options:
Option 1 - with SSD:
$96 Intel D2500CCE Mini-ITX $24 G.SKILL Ripjaws Series 4GB 240-Pin DDR3 SDRAM DDR3 1066 (PC3 8500) Desktop Memory Model F3-8500CL7S-4GBRL $69 M350 enclosure with picoPSU-80 and 60W adapter $59 Corsair Nova Series 2 CSSD-V30GB2A 2.5" 30GB SATA II Internal Solid State Drive (SSD)Option 2 - with CF:
$96 Intel D2500CCE Mini-ITX $24 G.SKILL Ripjaws Series 4GB 240-Pin DDR3 SDRAM DDR3 1066 (PC3 8500) Desktop Memory Model F3-8500CL7S-4GBRL $69 M350 enclosure with picoPSU-80 and 60W adapter $16 SYBA SD-ADA50024 2.5" SATA/USB To Compact Flash Adapter $24 Transcend 16GB Compact Flash (CF) Flash Card Model TS16GCF133Any thoughts?
-
There's a lot of misinformation in that thread. IMHO. ::)
There were a lot of early failures with the Kingston S100 8GB SSD that seemed popular with pfSense users. This was due to bad firmware causing data corruption but was interpreted by many as SSD failure. This is not representative of SSDs in general which should be good for many years.
If you use a CF card you should use the NanoBSD image (because a CF card will be damaged by excessive writes) and there is no point using a card bigger than 2GB at this point. There are some packages you can't use and some that are restricted. You can run Squid for example but only as a filter with no cache.Steve
-
There's a lot of misinformation in that thread. IMHO. ::)
There were a lot of early failures with the Kingston S100 8GB SSD that seemed popular with pfSense users. This was due to bad firmware causing data corruption but was interpreted by many as SSD failure. This is not representative of SSDs in general which should be good for many years.
If you use a CF card you should use the NanoBSD image (because a CF card will be damaged by excessive writes) and there is no point using a card bigger than 2GB at this point. There are some packages you can't use and some that are restricted. You can run Squid for example but only as a filter with no cache.Steve
Exactly, and some OCZ drives gave a lot of people… frustration. It's not so much of an issue at this point. Also, you're going to hear the horror stories much louder than the perfectly reliable instances.
-
I run the full install on $5 4GB USB flash drive, a keep a duplicate on hand for when it fails.
-
I run the full install on $5 4GB USB flash drive, a keep a duplicate on hand for when it fails.
Which may be OK if you run in such a state that there's not a lot of writes to "disk". Some people do, at which point a USB stick will not only wear out fairly quickly, but it also may be counter-productive to run it on something that may have less than optimal read/write performance.
I run mine off a 2GB USB flash drive as well, although the nano install.
-
Good, I got the impression that some of the information in that thread may have been blown out of proportion or just plain FUD.
I think at this point I've pretty much settled on "Option 1" above. I don't plan on having much disk I/O so unless I get faulty parts, it should be solid for quite some time.
-
Good, I got the impression that some of the information in that thread may have been blown out of proportion or just plain FUD.
I think at this point I've pretty much settled on "Option 1" above. I don't plan on having much disk I/O so unless I get faulty parts, it should be solid for quite some time.
Yes, a modern 30GB SSD should last a long time on a standard pfSense install. That's an early SandForce drive, which should be fine. I would, however, make sure to update the firmware before you start. If I recall correctly, that era of SandForce reserved some space for wear leveling and helped with performance, which is good (more recent versions are releasing that space back to the user for more capacity, since you don't exactly need that much space for your purpose, that extra wear leveling slack can help with longevity, faster is a side bonus.)
