Install on a Wear-Leveling CF Drive ?



  • I'd like to install the standard pfSense distribution on an Intel Atom Mobo with a 2GB CF EIDE drive as a boot device.  The Atom board has VGA, keyboard and mouse so I'd rather not go with one of the embedded images.  I lost my appetite for RS232 consoles a while back and doubt I even have a null modem cable lying about any longer.

    It appears as if the log files are stored on a RAM drive even in the standard install of pfSense, so I was wondering what other R/W activity would occur to the CF device with the standard image.  The CF device I have is wear leveling, so that should help with longevity.

    Are there other options I could pursue, such as going through the standard install and then configuring the CF device as read-only post install (would this even work) - or is the I/O to the storage device so low that I could get by for years before I run out of CF writes?

    I've been running pfSense for about a year on an old AMD mobo with a HD install and am thrilled with it.

    Thanks,

    Stephan



  • I suspect that if the wear leveling is any good you might get by for a good while on it, provided you don't install any packages with a propensity for disk writes. I killed a CF card in about 3 months using the freeswitch package. I don't recall if I was running squid at that time. The CF card had no wear leveling that I'm aware of, an I wasn't too surprised when it died.

    I suppose there's a way to install the full version and modify it to run read-only, but I don't know how complex that might be. You're probably aware that the nanoBSD-based embedded version is designed and normally recommended for read-only setups, with a limited set of available packages.



  • I'd suggest an SLC-based CF card over the much cheaper MLC.  I was running pfSense + squid on one (8GB) for a while (~6 months) with no issues.  The cards below are the ones I use in all my pfSense boxes (with full installs).  Make sure you get a CF adapter that supports DMA though, otherwise it will be SLOW.

    4GB: http://www.newegg.com/Product/Product.aspx?Item=N82E16820208418
    8GB: http://www.newegg.com/Product/Product.aspx?Item=N82E16820208417



  • I'm about to do the same thing, and was about to ask the same question.  I have an 8GB Sandisk Ultra card on a CF-to-SATA adapter, on an Atom-based barebones box.  There's no serial port (maybe one internally, but nothing on the outside), I'm going to have this hooked up to a KVM anyway, and I don't want to have to go through the hassle of installing the 4GB embedded image and resizing it on the CF card to get all 8GB.  Then again, having the CF card die prematurely on me would be more of a hassle… so, is there a way to see/log disk activity, so I know exactly what is being written to the CF, and can appropriately tweak the offending process?



  • Not sure how often they are written on the full image, but the nano image specifically keeps the RRD graph data in memory, and only writes it to disk on a shutdown.  I'd suspect that the full image is writing this directly to disk, which is every minute.

    Cheers.



  • it does write to disk, how often not exactly sure but i know its enough to kill a CF card (hence why there are nanno BSD version that are read only specifically ment for SSD media), but if you want to try it on a large enough card and put a full install on it no one will stop you but we are at least gonna let you know just be prepared for when it dies on you…



  • @jaime:

    it does write to disk, how often not exactly sure but i know its enough to kill a CF card (hence why there are nanno BSD version that are read only specifically ment for SSD media), but if you want to try it on a large enough card and put a full install on it no one will stop you but we are at least gonna let you know just be prepared for when it dies on you…

    It's not going to kill an SLC CF card, they're designed for that kind of work.  As I've already mentioned, I use them on all my machines (with full installs) and have never had one fail.



  • right, as long as its an SLC card you should be good to go, MLC cards are what most "common" cards seem to be "MLC" cards which from my understanding and past experience are the ones that tend to die a bit more often due to the "write failure" issues. and they are the ones you really would rather not use with full systems so I agree there with you SLC cards are more up to the task (the military uses that type of card anyways lol)…and Out of curiosity wear-leveling cards are generally going to be SLC arn't they or are there MLC cards that could also do this?



  • I guess I see too many conflicting comments on the real risks to make an intelligent decision on using full install on a CF card of whatever type.

    From here:
    http://www.edn.com/article-partner/CA6319917.html
    it says MLC cards are 10,000 cycles and SLC are 100,000 cycles.

    Sure, one picture per day is OK on a MLC camera card and it'll last 27 years.

    But if our firewalls are writing stuff every second (very feasible), there are 86,400 seconds in a day.
    Wouldn't that mean even an SLC card would begin to fail only after a few days?



  • Only if there is no wearleveling mechanism.

    A good manufacturer of wearleveling CF is http://swissbit.ch
    here is their official datasheets with more information to it:
    http://swissbit.ch/images/stories/pdfs/C-100_SFCFxxxxHxBI_data_sheet_v121.pdf

    If you need more information, i think i could give you a contact person who works at swissbit.



  • In further poking around on the forums, I see the unsupported option of setting 'platform' to 'embedded' in etc/platform after a full install.  What I haven't seen anywhere is what that configuration setting actually does.  Does it just reduce writes to the storage device or are there other behavioral changes to pfSense that I would see if I made this configuration change.  I'm not pathological about wanting to eliminate writes to my CF device, rather if there are simple things I can do to safely reduce writes, then I'd be thrilled to learn about them.

    Given the increasing availability of cheap, low-power Atom based systems with keyboard and VGA support, it could be valuable if someone knowledgable of the internals of pfSense could outline an embedded-like configuration that limits CF writes after the full install.

    I use pfSense at home and whilst one might argue it isn't mission critical - I would prefer avoid the calls from my wife if her internet connectivity drops (as the result of a dead CF drive).  I can't ask her to log a ticket and wait for resolution.  Well, I guess I could go the ticket route - but I suspect it would not improve the situation.



  • Since a lot of the functionality of pfSense is taken verbatim from FreeBSD and the Open Source community, I don't believe much rewrite of the software is taking place.  If a package uses code that does XYZ, and XYZ likes to log to disk every second, they are unlikely to re-code that.  So it really does depend on the packages you install.

    Now, that being said, it would be nice if the package owners disclosed what we could expect from installing their packages in regards to disk writing.  And if they had the ability to make them write to memory, then please do!



  • I suppose nearly everyone knows this already…  The little "micro drive" CF cards (which use a hard disk the size of a quarter) are very reliable as pfSense boot disks.  In fact, I've only killed one while using pfSense in three locations over three years.  (It was an infant mortality so the card was probably defective to begin with.)  There is no inherent write limit to disk technology.

    The speed of the little hard drive is pretty good, with decent sustained read and write times.  There is a small delay as the tiny disk spins up but it doesn't affect the firewall adversely.

    One point to watch out for -- not all micro-disk CF cards are bootable under FreeBSD.  I have found the Hitach Microdrive brand to be very reliable and it works well with pfSense.  It is commonly found on eBay in 4- and 6-GB sizes.  A lot of them are "pulls" from Apple iPods and the like.

    On the other hand, I have been unable to make a Seagate branded micro-drive CF card work in a pfSense firewall.



  • @joebarnhart:

    On the other hand, I have been unable to make a Seagate branded micro-drive CF card work in a pfSense firewall.

    I tried a 4GB Seagate micro drive and I couldn't get it to work with pfSense either. If I recall correctly it did work in a USB-CD reader.  At the time I could get a Transcend 1GB DOM (solid state disk) for quite a bit cheaper than the Seagate drive (let alone the USB CF adapter) so I went with the Transcend which has given reliable and quiet service for about 2 years now.



  • @stephanfr:

    In further poking around on the forums, I see the unsupported option of setting 'platform' to 'embedded' in etc/platform after a full install.  What I haven't seen anywhere is what that configuration setting actually does.  Does it just reduce writes to the storage device or are there other behavioral changes to pfSense that I would see if I made this configuration change.  I'm not pathological about wanting to eliminate writes to my CF device, rather if there are simple things I can do to safely reduce writes, then I'd be thrilled to learn about them.

    Given the increasing availability of cheap, low-power Atom based systems with keyboard and VGA support, it could be valuable if someone knowledgable of the internals of pfSense could outline an embedded-like configuration that limits CF writes after the full install.

    I've got a similar situation here. I've got an Atom D510 based soon-to-be firewall with a 2GB SATA DOM from Transcend. While the need for SMP support is probably up for debate, having a VGA console considerably eases management of this device since it also has IPMI 2.0.

    Because of this I'm in the process of finding out what the differences are between a true embedded install and switching from pfSense/SMP to embedded. So far it seems both ways mount the root filesystem read-only by default. But there are other subtle differences, like the way the filesystems are created and the way the system boots. For example, embedded uses UFS1 with a block size of 4k, while a default full install will use UFS2 with a block size of 16k. Another difference is that an embedded install will have two partitions containing a root filesystem and uses labels for all filesystems: "pfsense0" for the active root fs, "pfsense1" for the backup root fs and it also has a separate filesystem for storing the config labeled "cf" and mounted on /cf. The idea behind this is probably to facilitate firmware upgrades on the embedded platform, but I could be wrong. I'm still trying to find out though if configuration changes etc. are handled differently. Also, RRD graphs are stored in a small memory filesystem for both situations.

    But I'm still not sure which way to go… The SATA DOM does have wear leveling, but I rather go the safe route I think.



  • @scoop:

    So far it seems both ways mount the root filesystem read-only by default

    I thought that was a "feature" of the scripts, or /etc/fstab, not the kernel.

    But, even if it is a kernel feature, the scripts still have to be aware of that, and "write" to a suitable location, which is available as rw.

    Cheers.



  • @EddieA:

    I thought that was a "feature" of the scripts, or /etc/fstab, not the kernel.

    But, even if it is a kernel feature, the scripts still have to be aware of that, and "write" to a suitable location, which is available as rw.

    Cheers.

    Yes, that's true indeed. I just want the "full install" kernel for SMP and VGA console support. The rc-scripts /etc/rc and /etc/rc.embedded particularly act upon the contents of /etc/platform and create a memory filesystem for both nanobsd and embedded platform to hold RRD graphs and logfiles etc. I also found that the PHP GUI config takes care of remounting the root filesystem and - if present - the /cf filesystem to/from read/write access for both scenarios. So AFAIK there is no real difference in how the whole configuration part is handled between a true embedded install and a full install that's been altered to embedded.

    The only thing left to figure out is whether or not the upgrade process would still work after such change. Maybe that's just a matter of temporarily changing the platform back to pfSense/SMP and changing the fstab to mount the filesystems read/write by default. I guess there's only one way to find out… ;)



  • ok Im looking at this and I am  going to try installing on a CF drive (ok its actually a DOM) but I looked in the GUI and couldn't find a option to allow me to set the firewall to "read only" is there something Im missing or is this more something I have to do via SSH/Shell?



  • @jaime:

    ok Im looking at this and I am  going to try installing on a CF drive (ok its actually a DOM) but I looked in the GUI and couldn't find a option to allow me to set the firewall to "read only" is there something Im missing or is this more something I have to do via SSH/Shell?

    It's not an officially supported option, so you'll have to modify this manually through either Diagnostics -> Edit file -> /etc/platform -> Load -> Change pfSense/SMP to embedded -> Save or using SSH and vi. After this change you'll have to reboot and verify if your root filesystem is mounted read-only. If you prefer to check this using the GUI, you can do that with Diagnostics -> Command -> Command: mount -> Execute and verify if / is mounted read-only.



  • ok, ill give that a go once I get to that point, thanks!



  • I could have sworn that I posted this yesterday, but I guess not.

    @scoop:

    After this change you'll have to reboot and verify if your root filesystem is mounted read-only. If you prefer to check this using the GUI, you can do that with Diagnostics -> Command -> Command: mount -> Execute and verify if / is mounted read-only.

    How do I tell what's ro or rw, because here's what mount shows me:

    $ mount
    /dev/ufs/pfsense0 on / (ufs, local)
    devfs on /dev (devfs, local)
    /dev/md0 on /var/tmp (ufs, local)
    /dev/md1 on /var (ufs, local)
    /dev/ufs/cf on /cf (ufs, local)
    devfs on /var/dhcpd/dev (devfs, local)
    
    

    Cheers.



  • @EddieA:

    How do I tell what's ro or rw, because here's what mount shows me:

    $ mount
    /dev/ufs/pfsense0 on / (ufs, local)
    devfs on /dev (devfs, local)
    /dev/md0 on /var/tmp (ufs, local)
    /dev/md1 on /var (ufs, local)
    /dev/ufs/cf on /cf (ufs, local)
    devfs on /var/dhcpd/dev (devfs, local)
    
    

    Cheers.

    It should show "read-only" next to "ufs, local" when mounted read only, and nothing (other then "ufs, local") when mounted read/write. But judging on the output I reckon you're running the nanobsd embedded version, correct? Since I don't recall the full install having a separate filesystem for /cf. I'm doing all this from my memory, but I figure I'd better written down all the steps taken… I'll rerun the install here on a VM and take notes about the exact changes. I could very well have modified the /etc/fstab to mount the filesystems read-only by default.

    EDIT: Ok, checked with a reinstall and changing a full install to embedded through /etc/platform automatically mounts the root filesystem read-only. I have changed /etc/fstab however so that it immediately gets mounted read-only since this is also done on a nanobsd install.



  • @scoop:

    But judging on the output I reckon you're running the nanobsd embedded version, correct?

    Yes, correct.

    Now, here's an interesting observation.  If I issue the mount command, via the Diagnostics GUI, I get the result above.  However, if I log on to the box, and issue it at a command prompt, I get this:

    [root@roadblock.bogolinux.net]/root(1): mount
    /dev/ufs/pfsense0 on / (ufs, local, read-only)
    devfs on /dev (devfs, local)
    /dev/md0 on /var/tmp (ufs, local)
    /dev/md1 on /var (ufs, local)
    /dev/ufs/cf on /cf (ufs, local, read-only)
    devfs on /var/dhcpd/dev (devfs, local)
    [1.2.3-RELEASE]
    

    Hmmmmmmmmm.

    Cheers.



  • I think that the GUI config remounts the filesystems read-write during command execution. Sounds logical of course, since the command might be trying to modify the filesystem. So the only way to correctly check it is through SSH.



  • I've been running 1.2.3 on the Intel D94GCLF Atom 330 motherboard for a couple weeks now with an Innodisk 2GB Wear Leveling embedded IDE drive.  I've modified the platform setting from pfsense to embedded and also set the drives to RO in /etc/fstab.  I've seen the same behavior discussed below of the drives being listed as RO from the console but RW from the GUI.

    Thus far, everything seems to be perfectly fine - the only effect of setting the CF drive RO I have seen is that all the RRD graphs get reset after a reboot, which is no problem for me.  Operationally, I've noticed no difference between this install and my prior full install deployment on an older Athlon with a conventional hard drive.

    For what it is worth, the Atom 330 runs pfSense without even starting to break a sweat.  I've got a 20MBps line into the house and without running any VPN sessions, I can't get the CPU loading over a couple percent.  The board with 1GB of memory, an extra GB NIC and the CF drive draws right around 30W.  Based on power savings alone, the board and the Mini-ITX case will be paid for in right around 3 years when compared to my prior Athlon setup.  Given the performance of the system, I can't imagine a reason why I'd touch it indefinitely - short of something failing.

    The only thing I'd consider changing if I were to do this again would be to get a board with one of the 5xx series Atom processors - they have even lower power draw - but are also a little more expensive as well so I'm not sure if the ROI would change much.

    At this point, I think it would be helpful if we could get an officially supported embedded install with VGA and keyboard support.  I'm using this at home so I don't mind going slightly off the beaten path, but if I were doing this for my job I might think twice.

    Thanks to everyone who chimed in on this thread - I appreciate the comments and information.

    Best Wishes,

    Stephan

    http://www.intel.com/Products/Desktop/Motherboards/D945GCLF/D945GCL
    http://www.mini-box.com/2GB-40-pin-Embedded-Disk-Card-4000



  • I've got a couple installs on InnoDisk IDE DOMs running full installs, and my home testing install which gets reinstalled frequently from 2.0 snapshots. All of these have been stable running for at least 18 months, the oldest is about 24 months with no issues. Industrial flash is designed to be used this way, I doubt you'll have any problems with proper industrial flash modules, be it CF, IDE/SATA DOM or SSD.


Log in to reply