Newbie, please comment on my (planned) setup.
-
Hello all,
This is my first post here, I've been looking at PfSense for some years and now ready to make the switch.
After reading here and numerous other sites, this is what I come up as far as hardware.
Sorry if it's a bit long, perhaps others are researching the same stuff so I may as well share my thoughts.Situation:
Currently using another all in one server distro (with some issues), dated machine is using way to much power. Providing Internet access to a number of people, currently on cablemodem D3 but
fiber is coming soon. Gigabit routing needed. Snort planned, Squid absolutely.
Vlan core switch is present, 4 Gbit ports and 24 port base 100. Other small 8 port unmanaged switches and some wifi AP (Ubnt Unifi).
On the server (Intel E8400, 4GB ram, 9TB storage) Zoneminder, Samba file server, Unifi controller, Radius, Mail and other stuff.Plan:
Electricity cost is way to high. Also, every time the server has issues, the whole network goes down. Looked at virtualization, but it won't change that.
So finally decided to separate the router/squid/Radius part from the server and build two separate low power systems instead.
First I thought: Why not just buy the Edgerouter Lite and be done with it?
But what's the fun? I like something more "custom" and don't mind fiddling.
So let's start with the PfSense box, to assure Internet access at all time.Hardware:
I really like to have some headroom and be prepared for the future. Hardware may need to be repurposed later, so something standard would fit.
Budget is low, current location is Latin America.
After checking Intel's offerings, Baytrail is what I want. 10W TDP and more juice than previous Atom solutions.
That said, I looked at this motherboard:GIGABYTE GA-J1900N-D3V.
Yes, I know there are issues, I've read the topic here. But it seems PfSense alpha 2.2 runs on it, so let's just try.
The quad core might not be used at this point in time, PfSense may support it later.
Some doubts however, I've seen people mentioning those embedded solutions lately, but all of them seem to have just one PCI-e x1 slot or one PCI slot.
If you guys wanted to drop in a multi port network card, won't that be difficult?
I mean, most quad port Intel cards I've seen are x16, and even the dual cards are x4. So it won't fit in a x1 slot.
As for the PCI slot, it won't be able to keep up with dual port gbit cards.
On Ebay there are some Broadcom dual port PCI cards, that may be worth trying.
How would you guys solve this issue?
There seem to be microATX baytrail boards from Asrock with a PCIe x16 slot.
Obviously that needed an extra chip, but will accommodate 4 port PCI-e cards.As for the rest:
An Intel 530 120GB SSD, hoping it will last (Squid has me worried, see below)
8GB DDR3, can be SO-DIMM or normal size ram, I don't mind. I do have 2 sticks of Samsung low power dimms.
Pico PSU 120W. Definitely worth it. Has anyone tried the Chinese imitations from Ebay?
Case: I have some mATX spare cases, can adapt for itx if needed.That's for the parts.
As said, I plan to run 2.2 alpha but Squid proxy has me worried. Is it possible to create a ramdisk for /var (read it here somewhere) and put the whole Squid cache on the ramdisk?
I wouldn't mind putting in more ram, as it will be faster and may relieve the SSD.
Power outages will be minimum, we can't do without UPS here :-)Last but not least, something I couldn't find in the docs, does PfSense route between vlans out of the box?
I'm using vlans to separate users, it would be neat not have to setup blocking rules.Cheers.
-
Yes, pfSense routes between VLAN interfaces exactly like any other interface.
I have a Chinese dc-dc power supply that's worked just fine. It hasn't done that many hours though.
Steve
-
Thanks stephenw10 for your comments.
So I'll just have to set up rules to block trafic between vlans. No big deal.
As far as the Chinese "pico psu", they are cheap, so will buy two of them to have one as spare.It takes a while for the parts to get here, about 20 days (it has to be imported from USA and China) but
once I have it up and running I'll post back with the results.Cheers.
-
Missed one of your questions. Yes you can move /var to a ramdisk and choose its size via the webgui.
Steve
-
The ASrock microATX baytrail board does indeed have a physical x16 slot, however keep in mind it only runs at 4x max. This isn't a problem for most dual/quad port NICs, you should be able to pick up a HP branded Intel card off ebay for £25-£50.
-
Thanks stephenw10 and paulsnoop for the tips.
Putting /var on the ramdisk will limit my Squid cache somewhat, at the moment I've dedicated 10GB for Squid on a raptor disk.
If I go the "ramdisk" way, there will be at most 5GB of space for Squid, taking into account the (memory) space needed for PfSense and other packages.
It may be even less.
Still, for me it's worth going with ramdisk simply for the speed. Reading from ram is faster than reading from disk. However, the difference may be not that big when using SSD drives.
Sticking in another 8GB of memory (total 16GB) seems not an option, all J1900 boards can accommodate a maximum of 8GB (2x4GB).
Perhaps someone has experimented with the ramdisk size, if so please share your findings.As for the expansion slot, in another topic here we found a solution to populate the mini-pci-e slot with a dual port Intel gbit card.
Here two links:http://linitx.com/product/jetway-dual-gigabit-lan-mini-pci-express-card/13534
http://www.logicsupply.com/components/expansion-cards/admpeidla/
Not expensive at all and it will expand connectivity on the Gigabyte board with extra two gigabit ports, making a total of four ports.
This can be extra rewarding for mini-itx cases where a normal expansion card wouldn't fit.Cheers.
-
It has been a few weeks, and the parts are slowly being delivered here in Colombia.
I would like to report on what I actually bought.Supermicro seemed the way to go. Gonzopancho has been talking about the 2000 SoC and how future versions of PfSense will take advantage of its features.
So I ended up getting this:http://www.supermicro.com/products/motherboard/atom/x10/a1srm-2558f.cfmFor the PSU I got a PicoPsu. Accidentally bought the 20 pin version, but it worked just fine in the 24 pin onboard ATX connector.
The power brick is an ADP 180AB Gateway power adapter. One brick (180W) should power both my PfSense box and the storage server.The memory is Samsung "green" memory, low voltage (1.35v) non-ECC. Two sticks 4GB each for a total of 8GB.
I wasn't sure it would work because the mem is not on SM's list, but it did work flawlessly.Apart from the onboard 4 port Intel nic, I installed an HP dual port Intel PCI-e x8 network card.
I was pleased to see that the bios listed it as a PXE boot option.Storage will be a SLC DOM that's still underway, it will take another week or so. In any case I've got an Intel 2500 SSD ready if the "itch" to build becomes unbearable.
Measured AC power consumption (kill-a-watt) with Intel SSD drive: 22 watts. Can't comment on load consumption yet, no PfSense was installed.
Anyway for my uses the SoC won't be pushed at all, so with PfSense running I expect to stay under 30 watts.I also tried to measure power consumption with a Seagate 500GB conventional harddrive connected. At once it jumped to 30 watt, so the spinner took more than 8 watts.
That's a good reason to get rid of old spinners when possible. Sadly I have 3 harddisks 4TB each for my storage server, so those disks alone would consume together 24 watt.In the end I was a little bit worried about the flashing led at mid-board. I read some comments here about bad sensors on SM boards.
It was a false alarm however, the led indicates the IPMI heartbeat, flashing is normal.Perhaps those first impressions can be useful for people looking at some similar setup.
Cheers.
-
"Storage will be a SLC DOM that's still underway"
Which DOM? How big?
-
We talked about that. You don't remember that?
It's the two channel SLC DOM 32GB, when it arrives I'll put PfSense on it.http://www.ebay.com/itm/Kingspec-Industrial-Disk-on-Module-SATA-DOM-7Pins-32GB-SLC-2CH-for-POS-Machines-/161279269555?talgo=origal&tfrom=151281943955&ttype=price&tpos=unknowSeller specifies that the module will last for 80 years, writing 10GB a day. It's impossible to verify that claim at the moment, but if this proves to be true, it's a hell of a deal.
No worries letting squid and logging do it's thing.Cheers.
Edit: First it says write endurance "8 years at 100GB/day", further down it says 80 years at 10GB/day.
Can't take this seriously, but let's wait and see. -
Yes - I do remember talking about it and thought it was you, but didn't see my comment in the thread so thought maybe I was confused. Good - I'm anxious to get feedback on the entire build, but especially that DOM (-:
-
If you were testing the power consumption without any OS installed you will probably find it actually decreases. The bios setup usually doesn't (I've never seen it) have any power saving code or features enabled.
Steve
-
I have a similar setup. Can you report your operational temperature? What case are you using?
-
@stephenw10: You're right, I didn't think of that. Anyway it may be necessary to tweak the firmware settings a bit also once the OS is installed.
@Wolf666: Yes, when the SSD module comes in I'll test something more and will also take temperature readings.
Keep in mind that the board is mATX, I've got enough space. The case is a cheapo desktop mATX.
The SoC heatsink is only a bit warmer than ambient.Cheers.
-
Ok, I've got it set up :-)
For Wolf666: Temperature is at 53 celsius. That's in a desktop mATX case, pico psu 90, no fan at all.
Ambient temp is a bit high also, this is a warm country.I had some difficulties setting it up, mainly because I'm a n00b. But not ashamed to elaborate on it.
My SSD SLC module still has not arrived from China, so I just went ahead and put in the Intel 2500 SSD.
First on the UTP cables. No idea if Intel LAN cards are more picky than other brands, but for the life of me I couldn't get them to negotiate at 1000baseT.
Had to change the cables for others, some cat6 cable here is from a very bad quality it seems.
After swapping two cables, the ports negotiated at 1000baseT full duplex.After that, I had quite some trouble setting up vlans.
It was not clear at the beginning that the LAN interface had to be removed after setting up the vlans. Found that comment elsewhere here on the forum.
In my stupidity, I used a /32 mask for the interface and could not get to the webGUI for some hours. Finally figured it out.Some trouble with Squid install, not relevant for this section however.
After testing, it is quite a bit faster than my old server. Considering I'm on 2.2 beta, it may even get better (Multi threading perhaps, quick assist).
Cheers.
-
WOW - They didn't deliver your SSD… STILL!? Do you live in a remote jungle or are they just that bad at delivering?
-
Remote jungle.
Any somewhat fancy tech has to be imported, but I can't do it directly, it would be way to expensive.
Merchandise has to be send to Miami, FL first.
There it will be consolidated and send to Colombia.We are paying approximately 30% more for any tech, because of this.
Cheers.
-
I understand. Hope you get to test that soon. Glad your machine is working well.
