My 1U mini-ITX server home firewall
-
As somebody who had too much money (Christmas presents) vs common sense, I built myself a perfect 1U rackmount firewall server for home. I had to go to three different vendors, but I created my own Frankenstein which is fairly powerful, yet low powered. The idea was to use a mini-ITX MoDT board with a laptop CPU since the Atom was not powerful enough, and a desktop C2D would not fit in a 1U enclosure.
I started with the 1U case here:
http://www.idotpc.com/TheStore/pc/viewPrd.asp?idcategory=121&idproduct=410
I bought the PCI riser board with it, but not any of their power supplies. I didn't want an external DC power brick, and all their internal choices were 200W+ or inefficient at lower use.
So I found this: http://www.power-on.com/flexatx120g.html
Even though Idotpc.com didn't show it on their web site, they said they could get the same power supply. So I let them.I tried their MSI MS-9803 once before but it had a nasty BIOS which does not allow for any Speedstep or power saving features of the mobile CPU's. In other words, it ran full tilt 100% of the time. That went back. The search continued.
I then picked up the Gigabyte GA-6KIEH-RH instead from Logic Supply, along with a heatsink/fan.
http://www.logicsupply.com/products/ga_6kieh_rh
This works much better. It comes with dual Intel NIC's built-in, and I will use another dual Gigabit Intel PCI-X NIC in the PCI slot. I need four ethernet ports total, and they must be gigabit. See below**The laptop CPU was bought from eBay. I got the Intel Merom L7700 to conserve a bit of power. I could go up to the T8300 if I want to later, but the 1.8Ghz L7700 is already about twice as fast as an Atom. The TDP is 17W, but like all Intel CPU's, it's a conservative number only when pushed hard. http://ark.intel.com/Product.aspx?id=32243
The I/O shield has to be modified to fit in the 1U case, and Idotpc.com said they would be willing to do it. Since it was Christmas season and I felt trusting, I sent them my motherboard and I/O shield to fit in their case.
I got back the whole enchilada a few days ago. Put in a 2GB RAM stick, 160GB 2.5" HDD, my L7700 CPU and fired it up. All runs well.
I wanted something more powerful than an ALIX or ATOM because I want to run Untangle on it for some filtering. pfSense will still be my main firewall. So I loaded up the free VMware ESXi which requires Gigabit NIC's** and then installed both pfSense and Untangle in VM's. It works great! Two NIC's are for pfSense and two for Untangle in transparent mode. The "wan" side of Untangle has a short patch cable going directly into the "lan" side of pfSense. All that comes out of this 1U box to the world is the Untangle LAN NIC to my internal switch and the pfSense WAN NIC to my cable modem.
For home use, this works out great. Everything is fully ESXi compatible and I can change firewall distros (if needed) at will. Since ESXi 4.0 supports dynamic power throttling now (Speedstep, etc), my laptop CPU powers down a bit when not in use. At idle, the whole 1U server runs at 23W according to a Kill-a-Watt, and up to 30-35W when powering up VMs or at 100% (both CPU and HDD have to be chugging to get up to 35W).
Mobile-on-desktop motherboards aren't so common now and in about a year I expect the Atom's to be caught up to this power-vs-performance ratio (until I upgrade to a T8300). But for now, this is the best I could come up with. If I didn't need ESXi or Untangle, I could easily get by with the MSI IM-945GSE-A, which is another board I highly recommend for those who need to step up from an ALIX.
Remember…Intel NIC's all the way. Friends don't let friends run Realtek. Thanks for reading.
-
Sounds very nice, but why are you wasting two NIC's just to connect to VM's?
You can make virtual internal LAN / DMZ / WAN switches in the ESXi.I've created a very nice setup on my vmbox where servers are connected directly to the LAN port of the pfSense, and a NIC is also connected to the internal LAN network, so that I can put my normal switch there.
I've created a test-environment also, where I'm going to experiment with the 2.0 firmware, still only utilizing the WAN NIC, and a internal virtual switch, where I have a virtual WXP running.
-
Sounds very nice, but why are you wasting two NIC's just to connect to VM's?
You can make virtual internal LAN / DMZ / WAN switches in the ESXi.That might be my next step, but since I just built this box I wanted to test it with physical NIC's. Untangle is also very new to me, so I wanted to be able to remove it (physically) from the mix and just use pfSense by itself just in case. The reason for the physical cabling requirement is so if something goes haywire when I'm away, I can talk my spouse into changing cables rather than navigating the vSphere Client. :o
-
Ahhh, yes the WAF (Wife Acceptency Factor) is of course in the mix in some situations.
-
I am now using a virtual switch in the middle and just using two NIC's for the in & out. Works fine.
If VMware takes a dump (which it never has and I don't anticipate) while I'm away, I have my ALIX box running pfSense standing by and my wife can just swap the cables into it.
-
Valnar, how is your setup working out? I'm looking to build an almost identical setup (1U rack system with ESXi for pfSense + Untangle).
-
It's working great. I have no problems.
However, the total amount of bandwidth that goes through it is less than 8Mb (my broadband cap) and total number of simultaneous users tops out at 5. Although I do use bittorrent on one PC and it doesn't hiccup at all.
-
Thanks! I'm currently running pfSense + Untangle as Hyper-V virtual using legacy network drivers at around 20 Mbs so I'm not worried about the speed. It has been stable but I wanted to setup a dedicated 1U box for pfSense since I wanted to start doing some development for it.
-
Sorry one additional question. How is the processor load? I currently have a dual Xeon Quad Cores in my server and have 4 cores assigned to pfSense and have never had the utilization spike running running HAProxy, tinyDNS, Snort, squid, squidGuard, spamd and a couple other packages.
Thanks again.
-
My processor is a low voltage 1.8Ghz L7700 laptop CPU but it performs fine for my needs. On full throttle for my connection, which is downloading an Ubuntu ISO, copying a file from work over an IPSEC VPN (conected to the pfSense virtual) and bittorrent running on another internal box, my ESXi CPU is around 55%. Normally it is more Untangle than pfSense, but in this instance it's about 50/50 due to the IPSEC crunching on pfSense. pfSense otherwise runs very low. The CPU hits 100% of course while things are booting up.
To compare your Xeon vs my L7700, you can extrapolate the difference by searching the CPU score here:
http://www.cpubenchmark.net/cpu_list.phpNow mind you, I do NOT run the spam or antivirus modules in UT, but I do run web filtering, spyware, attack blocker, etc. I have no idea how the av/spam modules would hit my CPU since I've never loaded them.
With general web usage, ESXi CPU is usually less than 10% total. I also set the affinity so each virtual gets its own core. No concrete reason – just somebody on the UT forum scared me into thinking I shouldn't time slice UT.
I use the E1000 NIC for pfSense and VMXNET3 for UT. See the UT forums for details. I believe somebody prebuilt a VMtools ready UT appliance on v7.1.1.
-
Thanks valnar, I appreciate taking the extra info and taking the time to respond.
I have a spare Intel Core 2 Quad Q8400S laying around. I think I'll try to find a mini-ITX board for it rather than the mobile CPU route.
Thanks again.
-
Thanks valnar, I appreciate taking the extra info and taking the time to respond.
I have a spare Intel Core 2 Quad Q8400S laying around. I think I'll try to find a mini-ITX board for it rather than the mobile CPU route.
Thanks again.
Because of the cooling needed, you might have to spring for a 2U server box.
-
Not too worried about it anymore, a client just today offered to give me a SuperChassis 808LT-780B. I've used them before w/ Xeons and haven't had any cooling problems.
Overkill for a home setup but for free I'll take it!
-
Please see my thread here. http://forums.untangle.com/installation/15276-i-no-longer-run-untangle-vm.html
I no longer run Untangle in a VM. It was not such a good idea in practice.
-
I'm not registered to post over on Untangle, it was interesting to read your VM experience.
In my evaluating UT, it was a very very slow program, even with dedicated hardware. I even tried a bunch of systems I had onhand, even a decent Core 2 duo system that were painfully slow with UT, when configuring the UI. I never did get it installed in a user environment.I've run lots of varied things in Virtual Machines, normally on Microsoft Hyper V as thats what I'm familiar with. I've NOT found that anything virtualized is that much affected by moving from real iron to virtual. On a 4 processor machine, one or 2 processors and sufficient memory to handle the app is plenty. But then again, I'm running a good Xeon HP servers with multiple Raid drives. Were you actually seeing the processor pegged in your VM? Or could it have been IDE disk overload?
Do you know what might have been the bottleneck? -
It wasn't the CPU, memory or disk. All was fine according to VMware. Part of the problem may have been Untangle itself, but since it works fine now my guess was VMware is the culprit. I classify a UTM like Untangle as a "real time" application. And as such, it's susceptible to even microsopic jitter which is something that shows up easily in a G.711 VoIP testing app. You wouldn't want to run an Audio recording studio application in a virtual for the same reason. I think because VMware does CPU time slicing, even with only one VM (since ESXi needs some time too), that is enough to cause the jitter. The extra latency happened when I ran pfSense at the same time. That mostly went away when I moved down to just Untangle.
In any case, when all my Internet packets need to process through four NIC's and two firewalls, every millisecond counts, especially with Voice.
-
understood.
thnx.
