Embedded vs full hard drive install

tranquilnet

After much research, we have decided to use pfsense for our two perimeter firewalls in a colocated environment.

We will be using the following hardware:

Motherboard SuperServer 5015B-MRB
http://www.supermicro.com/products/system/1U/5015/SYS-5015B-MR.cfm

Processor Xeon Dual Core 3065 2.33 4M 1333fsb Boxed

Memory 4GB (4 x 1GB) 667MHz DDRII Unbuffered ECC Memory

We initially were going to run pfsense from a 1GB Dual embedded disk module (SLC).

http://www.innodisk.com/production.jsp?flashid=81

After reading numerous forums, wikis, etc, I am not convinced this is the best route to go.

This is why I request your support and advice.

Ultimately we want:

Reliability and performance without failure
VGA support if possible
The ability to update/upgrade pfsense easily
Possible package addition to the standard install

Taking the above into account, my questions are:

Would you recommend we use an embedded disk module (or some other compact flash solution) or a SATA hard drive for the pfsense instlall?
If embedded, what type and does this make adding vga support, adding packages and maintaining upgrades more difficult of a process?
If SATA hard drive, what brand/speed drive would you recommend?
Why would I choose one or the other (what is the hype over embedded installs)?

We appreciate everyones support in advance.

Thanks,

-Will

tranquilnet

UDPDATE:

I spoke to the manufacturer of the embedded disk module we chose to initially perform the pfsense install on. According to him, due to recent technology, there are not issues with many writes to the module and a full OS can be installed on it.

If this is the case:

Can I simply do a full install on an embedded disk module that can withstand many writes?
What would be the minimum size module I would need?
If partitioning is needed, what would the layout be?

Thanks,

-Will

jasonlitka

I just rolled out a few boxes with 8GB SLC CompactFlash cards and don't anticipate any issues with write-wear (I wouldn't trust an MLC card though). Anyway, I allocated a 2GB swap partition as the installer was complaining that if I omitted one or used something smaller than the amount of physical RAM then it wouldn't be able to save a memory dump in the event of a crash. The rest of the space was given to / and thus far I've used a grand total of 87MB of it.

tranquilnet

Jason - Thanks for the info. I appreciate the support and the feedback.

The embedded modules that I have are 1GB SLC. I have been told that read/writes will not be an issue by the manufacturer as well.

Does anyone know if I can get away with the pfsense install without creating a swap partition since I only have a 1GB module to work with? Note that I have 4GB of memory on my server. I have done so in the past with other applications (openfiler) and there seems to be no issues. I assume pfsense would be the same. Note that this install is for a production environment. Can anyone confirm and or give me the pros/cons of having/not having a swap partition (I am sure this is unix/linux 101 so I apologize in advance)?

Thanks,

-Will

tranquilnet

UPDATE:

I did a full live CD ISO install and was getting a boot error on pfsense 1.2.2 after the install:

hptrr: no controller found
ad4: FAILURE - SET_MULTI status=51 <ready,dsc,error>error=4 <aborted>My Hardware:

Motherboard SuperServer 5015B-MRB
http://www.supermicro.com/products/system/1U/5015/SYS-5015B-MR.cfm

Processor Xeon Dual Core 3065 2.33 4M 1333fsb Boxed

Memory 4GB (4 x 1GB) 667MHz DDRII Unbuffered ECC Memory

I installed pfsense with no swap on a 40 Pin IDE 1GB Dual embedded disk module (SLC).

http://www.innodisk.com/production.jsp?flashid=81

I ended up changing the BIOS setting for this IDE ad4 device by entering the BIOS ~~and selecting:

Ext. Primary Master [1048MB]

and changing the settings from [auto] to [user]

and changing the Transfer Mode to [FPIO 4 / DMA 2]

and changing the Ultra DMA Mode to [Mode 4]

After doing this, everything booted fine.

-Will http://www.tranquilnet.com~~</aborted></ready,dsc,error>

jasonlitka

The "FAILURE - SET_MULTI status=51 <ready,dsc,error>error=4 <aborted>" you can ignore.

As to your DMA/PIO issues, yeah, I had the same problem with the card reader for my test system (though not the ones I'm using in production). Unfortunately, the BIOS on the systems I'm using doesn't support forcing PIO, so for the system with the cheap reader the alternative solution is to boot into safe mode (where PIO is forced) and add the line "hw.ata.ata_dma=0" to the bottom of /boot/loader.conf.</aborted></ready,dsc,error>

osopolis

Hi.

I also want to use pfsense in a production enviroment but as firewall and bgp router.
I have read the specs about the embedded flash storage and saw the read/write thruput.

Could this be a bottle neck when using pfsense + openbgpd?
If pfsense swaps a lot than it definately would. I dont have any real life experiences with pfsense.

The box will have 4GB ram and a pentium 3,8Ghz sigle core CPU.
I need to push 400+ mbit/s.

Cheers.
Victor

tranquilnet

Victor - I would LOVE to know the answer to your question as well.

Anyone out there that could shed some light here???

I wish I had more experience. Even with my current setup, I do not know if I am going to run into any issues. I installed pfsense 1.2.2 from the full ISO installer on an 1GB IDE embedded disk module with read/write of 40/20 MB/sec with no swap specified. I actually just guessed this would be the best solution because I wanted VGA support as well, I could have went with the embedded image but it seems pfsense shys away from totally supporting it.

Let me, and this forum, know what you find out.

Thanks,

-Will http://www.tranquilnet.com

jasonlitka

Read/Write throughput isn't the end-all-be-all of performance. Access time is VERY important when working with tiny bits of data and on that front an SLC SSD (be it Compact Flash, DoM, 2.5" SATA) will destroy a normal disk. I'd say you'll be fine as long as your device supports DMA (PIO4 is still 20MB/s but it comes with high CPU usage).

To osopolis: I'm not sure that a single core Pentium 4 will be able to deal with 400Mbit/s, though I'll admit that I've never tried to route that much traffic through anything but an actual hardware router (not to mention that that chip is going to run hot as hell, what is that, TDP of 120W?). You'd probably be better off with something newer like a Intel E7400 or the Xeon equivalent. Also, make sure you get Server network cards (or at least Intel Desktop cards) as cheap Realtek parts (or anything similar) aren't going to be able to keep up.