Recommend hardware with gig ports?



  • Yes I've searched and read. I just want to know what the latest recommendation on available hardware is.

    Looking to replace a fullblown dell server with something small and less power hungry. I need 3 or 4 gig ports. My sustained traffic doesnt saturate gig on a regular basis but it does occur now and then and I need to be able to keep up with it. Most of the small bits of hardware I've been seeing talked about have 10/100 interfaces which rule them out unfortunately.

    Input is appreciated and in the meantime I'll be doing more searching :)



  • I haven't yet deployed this configuration, but I've done a lot of research and come up with the build below. It should be able to pass 1Gbps without too much trouble, is fairly low cost and fairly low power, and built from quality parts. You can save a few bucks and get a bit more flexibility building it in a non-rackmount (or more RU) case, but I need a 1U unit:

    SUPERMICRO CSE-502-200B Black 1U Rackmount Server Case
    SUPERMICRO MBD-PDSBM-LN2+-O - 2 onboard Intel PCIe NICs
    Intel Core 2 Duo E5300 2.6GHz - could probably get away with a Celeron 400 or E1xxx/E2xxx series, but the Core should have no trouble at 1Gbps
    Kingston 1GB 240-Pin DDR2 SDRAM ECC
    Dynatron P199 - Stock fan won't fit in a 1U

    If you need more than 2 NICs, add either a single or dual Intel PCIe server adapter with a Supermicro CSE-RR1U-E8 riser (not available at NewEgg, but fairly inexpensive elsewhere - or just use a larger case).

    I am planning on running the build from an industrial CF card in a SATA->CF adapter, but you could add a proper disk if you wanted.

    With 3xGigE this setup will cost you ~$500, less if you don't need a rackmount solution.



  • Okay so I have 3 spare machines laying around that I could repurpose for pfsense.

    1 of them has a 2.8g pentium D, but the NICs onboard are VIA Rhine II nics. VIA P4M800-M7A motherboard

    The other system is a Pentium4 2.66 and has Intel 1000 nics onboard.

    Also have an old Dell PE1550. It has a P3-1ghz cpu. Seems slow by todays standards but what its capable of I dont really know.

    Which do you think would be better suited? I'm inclined towards the intel nic board (supermicro) but dont know if the extra cpu core will be that important or not.

    Any input?

    KTIMS : any recommendation on CF card and adapter?



  • I'd definitely go with the machine with better NICs if you're not prepared to upgrade them. Besides, the Pentium-D is very power hungry and I'm not sure the extra core is worth that, but if you want to do gig I'm not sure either is fast enough, you'd have to experiment. Then again, the Rhine II is a 100mbps card, so maybe that's not a priority anyway? You'll save quite a bit of money on your power bill if you go with a modern Core architecture CPU, those NetBurst processors just soak up the juice.

    As for flash, I've just found InnoDisk disk-on-modules which seem a better (and cheaper) solution than using a CF->IDE adapter. You can buy them here. Prior to finding this product I was using Transcend industrial CF with a CF->SATA enclosure by SANS Digital.



  • Yeah I am actually testing both these machines with my Kill-a-watt unit. The PentiumD box uses more than twice the power. Both systems have 1 disk powered for the test. That alone decides it. Its been a long time since I've pushed more than 150meg so gig isnt imperative. Good point about the Rhine being 100mbps though. Everything points the way to the Supermicro system.

    thanks for the input. I'll check out innodisk.



  • Out of curiosity… what capacity size innodisk did you decide on? Seems to me that a 4gb module is more than sufficient.

    @ktims:

    I'd definitely go with the machine with better NICs if you're not prepared to upgrade them. Besides, the Pentium-D is very power hungry and I'm not sure the extra core is worth that, but if you want to do gig I'm not sure either is fast enough, you'd have to experiment. Then again, the Rhine II is a 100mbps card, so maybe that's not a priority anyway? You'll save quite a bit of money on your power bill if you go with a modern Core architecture CPU, those NetBurst processors just soak up the juice.

    As for flash, I've just found InnoDisk disk-on-modules which seem a better (and cheaper) solution than using a CF->IDE adapter. You can buy them here. Prior to finding this product I was using Transcend industrial CF with a CF->SATA enclosure by SANS Digital.



  • I've used 1GB CF cards in all my builds to date, haven't actually used any of those InnoDisk modules yet, but they're on the list for my next build. For my needs 1GB is more than sufficient, but I don't use Squid or any other modules that use much disk.



  • I wanted to point out that I have the same board that ktims has up there. You need to know something though, that board does not support Q9000, E8000, E7000 or E5000 series processors! I know becuase I found out the hard way. So that Proc you listed will not work. The board won't even post with that Proc.



  • @tommyboy180:

    I wanted to point out that I have the same board that ktims has up there. You need to know something though, that board does not support Q9000, E8000, E7000 or E5000 series processors! I know becuase I found out the hard way. So that Proc you listed will not work. The board won't even post with that Proc.

    Good catch  :-X

    Probably want to go with the E2220 instead. Oops!



  • Ordered a 2gb solid state module. Will report back when I've received it and installed it.



  • Any reports on this configuration?    ;D



  • That Supermicro setup interests me.  I'm interested in something more than an ALIX and an Atom would probably do fine, but lightweight C2D would be better.  What kind of power does it take?  Have you measured it with a Killawatt?



  • We've been building some walls with the Jetway NC92 board, with the 3 x GB NIC daughterboard, and it has an additional 2 x PCI if you use a suitable case.  Cheap, fast, low power, and 4 x GB NIC!!



  • @valnar:

    That Supermicro setup interests me.  I'm interested in something more than an ALIX and an Atom would probably do fine, but lightweight C2D would be better.  What kind of power does it take?  Have you measured it with a Killawatt?

    I haven't measured it, and the UPS it's on powers a bunch of other equipment as well, so even those rough measurements aren't going to be useful.

    It works very well though, basically the exact setup described above but with a Celeron 430. No trouble doing 100mbit, but that's as fast a WAN link as I've got and it's in production so I can't push it further :P



  • Hey, guys.

    First time poster/first time pfsense builder here.  Please forgive me if I should have started a new post, but I specifically want to inquire about nexusone's and tommyboy180's hardware implementations.

    I've been digging through the forums and across the interwebs for workable hardware configs and this supermicro setup you guys are running has me very intrigued.  But I'd really like to get you guys' input before I build because this is uncharted territory for me and I've spent more time than I care to admit researching and spec'ing this thing out.

    ktims: is your CF/SATA adpater setup for embedded builds, or are you using it for full installs?  Assuming you're doing a full install, how do the industrial CF cards hold up to numerous writes?  Is this a better way to go than, say, a sata hdd?  Because of the solid state/no-moving-parts, or is it more of a power-saving issue?

    I would be grateful if any of you that are using this PDSBM-based setup would share your complete specs.

    Here's my planned build (most of which I stole from ktims):

    MBD-PDSBM-LN2+
    Supermicro CSE-502-200B
    Celeron E1500 http://www.newegg.com/Product/Product.aspx?Item=N82E16819116075
    Kingston 2x1GB DDR2-667 http://www.newegg.com/Product/Product.aspx?Item=N82E16820134046
    Dual Port Intel Gigabit Server NIC PCIe http://www.newegg.com/Product/Product.aspx?Item=N82E16833106014
    Hitachi 80GB SATA http://www.newegg.com/Product/Product.aspx?Item=N82E16822145238
    Supermicro 1U Active Heatsink http://www.provantage.com/supermicro-snk-p0032a4~7SUP9016.htm
    CSE-RR1U-E8  Riser
    2.5" HDD retention bracket http://www.provantage.com/supermicro-mcp-220-00044-0n~7SUP9019.htm

    According to the CSE-502-200B manual, you can't really fit both an expansion card and a hard drive, so I was going to use the retention bracket and dremel most of it off to make two rails for the laptop hard drive.  I'm thinking/hoping that with this setup, the NIC should have enough clearance over the hard drive to accommodate both NIC and HDD in the chassis.  Has anyone tried anything similar?

    Nexusone and tommyboy180: what type of storage device are you installing pfsense to?

    Is anybody using the supermicro board setup with a SATA hdd?  Any caveats?  From what I've read, some folks are having a hard time installing pfsense to sata hdds on ICH7, which is the controller on the PDSBM (well, ICH7R, anyway), like this person here: http://forum.pfsense.org/index.php/topic,7172.0.html

    But by the looks of it, you guys aren't having a problem installing to CF cards emulating SATA devices…  ::)
    I suppose I could always install an 2.5" IDE drive instead; has anyone tried this with this board?

    Thank you all for taking the time to read this.

    EDIT:  ktims: after reading some of your other posts, I see the answer to my question about the preference for CF cards seems to be due to the solid stateness of them.

    EDIT:  And what release are you guys running with this board?



  • Hello Bok Bok,
    I think you will be very pleased with the preformace with your supermicro hardware. This is a very powerfull system that can support 1000's of users.

    I just wanted to ask you why you have decided to purchase the Intel EXPI9402PT. Do you need 4 NICs, or will the onboard NICs not do 'it' for you? The onboard NICs are very powerfull and reliable. I have never had any problems.

    Also double check you RAM agaist the tested RAM for the board on the SuperMicro page. Your RAM will work, I don't doubt that, but if you are looking for Max preformance and Max Stablility then go with a chip that was tested on the board.

    I am using the board with a SATA drive, 160GB. The preformance is unbeatable. The only thing I would recommend with if you decide to go with the onboard SATA is to use a very short SATA cable.

    My full specs are
    Board - http://www.newegg.com/Product/Product.aspx?Item=N82E16813182126
    Cooler - http://www.newegg.com/Product/Product.aspx?Item=N82E16835114075
    Case - http://www.newegg.com/Product/Product.aspx?Item=N82E16811152106
    Ram - http://www.newegg.com/Product/Product.aspx?Item=N82E16820134192


  • Banned

    If I should settle for 1U rackmounts, i would definetely look at Ebay for IBM 335/336 hardware….

    Lot cheaper used than Supermicro, and it can handle heavy loads. On embedded CF, it uses 120W....And is not so noisy.....



  • problem could be the depth of the ibm's? some people only want minimal depth for a small wall mount cab.
    was just looking at another mobo with intel gig on and seen this.
    http://www.icp-uk.com/index.php?act=viewProd&productId=225
    looks a promising little board that would easily fit in the supermicro case



  • tommyboy180: Thanks for the reply!

    To answer your about the NIC, I need four ports.  I have absolute confidence in the onboard Intel NICS (kind of building the system around them  ;) - and the form factor), but I just need more ports.

    However, as has been discussed many times on this forum, a popular way to go seems to be to just set up vlans, run them all through a single LAN interface, and pair the setup with an 802.1q-friendly switch.

    To be honest, I'm a little reluctant to do it that way.  While the security aspect is debateable, I keep thinking about what will happen when someone else inherits this setup.  Will the VLANs make them like, "huh"?  Pretty much any technician (I hope) gets port=interface, but I'm not so sure about port=3-to-12 interfaces.

    OTOH, while right this second three LAN interfaces is exactly what I need, at some point down the road (I don't know how far) this location is probably going to outgrow three interfaces, and so then it's either more routers for all the subnets, or it's VLANs.

    With regards to RAM,  I will definitely take your suggestion, tommyboy180.  May I ask who manufactures the 160GB SATA drive you are using? And short like 6" or short like 10"?

    If I could pick your brain just a little more:

    1. Are you using 1.2.2 or a 1.2.3 snapshot?  It looks like the 1.2.3s alleviate a lot of SATA hassles, and a lot of people seem comfortable using 1.2.3 in production.

    2. Do you do your installs off IDE or SATA optical drive?

    Thanks again for the useful info!

    @louis-m: that looks like a great board - I think a number of folks are using it, tho I'm not sure it would fit in this supermicro chassis: http://www.newegg.com/Product/ProductReview.aspx?Item=N82E16811152106 - read Camarofleet's review.  Of course, this is just speculation on my part.

    Right on point about the wallmount action.  "Being able to mount your bad-ass firewall on your relay rack… priceless."



  • I'm actually using an IDE DOM from Innodisk in my builds now; these are about 1" tall and plug directly into the IDE header on the motherboard - very convenient. I use their CF in my embedded installs as well. It's been reliable, though performance is quite a bit poorer than a hard disk. I've got 3 or 4 of them out there now, but only for 4 months or so when I discovered these. They're running full installs and I've had no issues to speak of, but I don't do any write-heavy operations on them either (Squid etc.). If you're planning to do lots of writes I'd go with a hard disk, otherwise I'd go with one of these DOMs. I'm currently running 1.2.3-RC1 on the Supermicro build I have in production.

    WRT. 802.1Q & VLANs - while they're not really commonly understood among low-rent IT 'consultants' that might do work for a small business, anyone that's likely to understand a setup with 4 subnets and various rules in pfSense should have at least a peripheral understanding. Certainly anyone that should be doing this kind of work should have a good understanding. And as you wisely notice, the 1 interface per subnet paradigm just doesn't scale. It's going to be a matter of do it right right now or try and migrate in the future which is going to be a lot more painful for you. Using VLANs also gains you a lot more flexibility over the network in general if you can justify replacing all your presumably unmanaged existing switches.

    A compromise might be to use one of the onboard NICs for your WAN connection and only do VLANs on the LAN side and connect that to the switch. That should be a more intuitive setup for other folks. And of course documentation, but I wouldn't stoop to the 'next guy's' level if it compromises your work.



  • @Bok:

    However, as has been discussed many times on this forum, a popular way to go seems to be to just set up vlans, run them all through a single LAN interface, and pair the setup with an 802.1q-friendly switch.

    To be honest, I'm a little reluctant to do it that way.  While the security aspect is debateable, I keep thinking about what will happen when someone else inherits this setup.  Will the VLANs make them like, "huh"?  Pretty much any technician (I hope) gets port=interface, but I'm not so sure about port=3-to-12 interfaces.

    I don't think you're giving network folks enough credit.  VLAN's are very common.  Any consultant worth their salt knows how they work.  A regular PC tech?  Maybe not.

    @ktims:

    A compromise might be to use one of the onboard NICs for your WAN connection and only do VLANs on the LAN side and connect that to the switch.

    That would be the right way to do it.  You just need a 802.1Q aware switch



  • ktims: Thanks for getting back to me and for sharing the details.

    I'm familiar with these DOMs; we actually currently use them on our linux DVRs for our surveillance, but as you can imagine, the video data is recorded/written to a separate IDE disk.

    As for limiting the vlans to the physical lan interface, absolutely.  I can't say I'll ever have enough balls to run my WAN through the same physical interface as my LAN - but who knows; necessity is the mother of invention. ;)

    @ktims:

    WRT. 802.1Q & VLANs - while they're not really commonly understood among low-rent IT 'consultants' that might do work for a small business, anyone that's likely to understand a setup with 4 subnets and various rules in pfSense should have at least a peripheral understanding.

    Well you nailed that one.

    @ktims:

    Using VLANs also gains you a lot more flexibility over the network in general if you can justify replacing all your presumably unmanaged existing switches.

    No worries there, as it's a new install;  since I was already on the fence about the VLANs, I went ahead and bought some ProCurves.  Besides, in this day and age, I can't see buying a switch that isn't at least "smart".  And beautiful.  ;D

    @valnar:

    I don't think you're giving network folks enough credit.  VLAN's are very common.  Any consultant worth their salt knows how they work.

    Valnar, you're right. I think I tend to let my perception get a little skewed by some of the people I've gone in behind.  I'm not even what I would consider "good", but I've still seen some "professional" work that made me cringe. :'(

    Anyway, thanks guys!  I'll let you know how it turns out.



  • Bok Bok,
    The HDD is http://www.newegg.com/Product/Product.aspx?Item=N82E16822136075, a WD.

    When I say short for the SATA cable I mean 3 inches. Thats all you need since the SATA ports are right next to where you will house the HDD.

    I am running the latest 1.2.3 snapshop. Working just fine.

    My DVD Drive is actually USB external and is only connected when needed for installs.

    No problem, if you are interested in pictures check my blog post on the hardware: http://www.tomschaefer.org/web/wordpress/?p=255 There is a link at the bottom for the Gallery or click http://www.tomschaefer.org/web/Slideshow/SuperMicro_Router_28Jan2009/



  • http://www.lannerinc.com/Network_Application_Platforms/x86_Network_Appliance/1U_Network_Appliances/FW-7560

    I'm not sure what the price would be, but this is the 1U version of the system that I installed ~4 months ago.  The FW-7520 is the fanless "desktop" version that I installed in my in-wall wiring cabinet.  The chipset includes the 1gig Intel NICs.  I've managed to push 300Mbps (802.11N on one port to wired gigabit on another port) with no problems.  Building a system will probably be cheaper, but there is something to be said for a finished product.



  • kc8apf, Any idea of how much power that takes?  I'm looking for a step up from an ALIX board, so something that like would be good, but I have my eye on an Atom based appliance too.


  • Banned

    Power
    1U ATX SPS /150W
    AC 100~240V @ 50~60Hz

    Based on that, I would say around 40-50w normal use….



  • @valnar:

    kc8apf, Any idea of how much power that takes?  I'm looking for a step up from an ALIX board, so something that like would be good, but I have my eye on an Atom based appliance too.

    The 7520 runs about 20W at full load.  I don't know about the 7560, but the Lanner rep has been very willing to provide any details via email.



  • @tommyboy180:

    When I say short for the SATA cable I mean 3 inches. Thats all you need since the SATA ports are right next to where you will house the HDD.

    I am running the latest 1.2.3 snapshop. Working just fine.

    My DVD Drive is actually USB external and is only connected when needed for installs.

    No problem, if you are interested in pictures check my blog post on the hardware: http://www.tomschaefer.org/web/wordpress/?p=255 There is a link at the bottom for the Gallery or click http://www.tomschaefer.org/web/Slideshow/SuperMicro_Router_28Jan2009/

    You just need to mount the HDD the other way around (so that the connection ports face the rear of the chassis).  The typical SATA cable supplied with motherboards (30cm to 50cm) aren't that long as with server boards (1m) so those will work just fine.
    If the cable is still too long, just fold it lightly and zip tie in the middle (don't let the ends kink).



  • OK, gang.

    I'm burning in my setup as we speak.  I basically went with tommyboy180's exact setup except I used the supermicro 1u cooler, which is like as heavy as the whole rest of the setup.  Also, using the celeron 430 for the lower wattage (plus anything more is probably overkill for the target environment).

    I love the form factor of this supermicro 1u. It's so little and just drops right in with your switches.  Fan noise isn't too bad if you tone down the rpms in BIOS.

    I'm glad you guys convinced me to ditch the extra interfaces and just VLAN everything (well, LAN-wise, anyway).  It's made everything so much easier in my test environment, and it'll make provisioning new networks a breeze in production.

    Anyway, haven't installed pfsense on the new 1u setup - still burning in - but I'll let you all know how it goes.

    Ktims, tommyboy180, others, thanks for all the guidance.



  • Glad to hear you like it, sounds like your going to be a SuperMicro fan from now on.

    Let us know if you have any problems.



  • I've been a SuperMicro fan ever since my first dual socket (slot) PIII SuperMicro board.  ;)

    Thanks again!


  • Banned

    How long does spareparts take to arrive from Supermicro???

    I live in Europe and need day to day in a production environment…..

    Are they able to do that??



  • When I was in Japan it took about a week and a half to get a riser card from them, but I didn't pay for over night.



  • @ktims:

    I haven't yet deployed this configuration, but I've done a lot of research and come up with the build below. It should be able to pass 1Gbps without too much trouble, is fairly low cost and fairly low power, and built from quality parts. You can save a few bucks and get a bit more flexibility building it in a non-rackmount (or more RU) case, but I need a 1U unit:

    SUPERMICRO CSE-502-200B Black 1U Rackmount Server Case
    SUPERMICRO MBD-PDSBM-LN2+-O - 2 onboard Intel PCIe NICs
    Intel Core 2 Duo E5300 2.6GHz - could probably get away with a Celeron 400 or E1xxx/E2xxx series, but the Core should have no trouble at 1Gbps
    Kingston 1GB 240-Pin DDR2 SDRAM ECC
    Dynatron P199 - Stock fan won't fit in a 1U

    If you need more than 2 NICs, add either a single or dual Intel PCIe server adapter with a Supermicro CSE-RR1U-E8 riser (not available at NewEgg, but fairly inexpensive elsewhere - or just use a larger case).

    I am planning on running the build from an industrial CF card in a SATA->CF adapter, but you could add a proper disk if you wanted.

    With 3xGigE this setup will cost you ~$500, less if you don't need a rackmount solution.

    I just deployed a Supermicro PDSBM-LN2+-0 with an E2200 and 2GB inside a SUPERMICRO CSE-512L-260B case. The system Is humming along. Out of the box the system can be loud but the case fan, and the cpu fan both have 4 pin connectors, so you can throttle down the speed in the bios.



  • Still liking it?

    I still have yet to have an issue with my supermicro pfsense rtr.



  • Still humming along. It doesn't have a huge load on it. A T1 for the internet, two ipsec vpn's and 5 openvpn clients. 19 internal users. Perhaps overkill I don't know.

    In the next month or so, I hope to get the Supermicro Atom mobo (I forget the model, but it has dual intel nics) and use it as a test


Log in to reply