Install pfsense to an usb drive



  • Hi

    Everytime i google for information on installation of pfsense to a usb pendrive all i get i how to install from a pendrive.

    The disk in my pfsense install just went rouge so at the moment i run on a live usb, but i am considering using the usb drive as the main drive because of the flash part/non mecanical parts.

    The question is in this incredibly bad written post.

    Is it possible to install pfsense to a usb drive. Is there any downsides to it? And is any of the images made for something like this, withput the need to actually install it on the drive?

    Best Regards
    Kristoffer


  • Netgate Administrator

    Yes, you can do this using the NanoBSD images.
    Just write the image to the usb drive as is documented for nanobsd installs.
    On the first boot it will fail at the mountroot> prompt because it expects to find root on the primary master IDE drive, ad0sa1.
    Type '?' at the prompt to see what devices are found and enter the appropriate mount point.
    After it has successfully booted edit /etc/fstab to point at the correct mount points for your USB drive.

    Downsides are; because you're using the NanoBSD image you won't have all packages available.

    Steve



  • Interesting

    Is there a way to get a nanobsd system to become "full version" so the packages are not missing? And why is the reason that it is not possible from the start?


  • Netgate Administrator

    No. There are only a few packages that are unavailable though.
    Nanobsd was created to make Freebsd suitable for installing on embedded systems, specifically on flash media. Flash media has a limited number of write cycles and a standard install can burn through them quickly resulting in drive failure. Nanobsd avoids this by logging to ram only and mounting the file system read only (among other tweaks). Because of this some packages cannot work. Most have been adapted to work, for example it's possible to run squid without the cache just as a web filter.

    Steve



  • Ok great:) thanks for the answer, but how will this affect the config of the install on reboots?


  • Netgate Administrator

    Could be that I haven't had enough coffee yet but I don't understand the question.  :-\

    Steve



  • If the filesystem is readonly, will that have the effect that the config can not be saved and that it has to be reconfigured everytime it boots?


  • Netgate Administrator

    Ah!
    No.
    The NanoBSD system is clever. It consists of three partitions (slices). A root partition, an alternative root partition and a config partition. The root partition and the config partition are mounted read only by default. /tmp and /var are created at boot as ram disks. The pfSense config file, config.xml, which contains every setting in pfSense is stored in the config partition. When you make a change it is re-mounted as RW temporarily. If you add packages or in some other way alter the root file system in the approved manner it too is temporarily re-mounted RW. Doing this prevents anything that you may have installed writing continuously to your flash drive potentially killing it in short order.
    You can choose to boot from the alternate partiton at boot time. This means that if you install some package (and I'm referring to FreeBSD packages here not pfSense) that overwrites some critical file you can just switch to the other partition. You can then choose to duplicate the working partition on to the broken one. The same applies when you update pfSense, the new version is written to the alternate partition and then set to boot from that. If for some reason it fails to boot you can simply choose to boot the old version.

    To be honest all this is explained better in the link I gave above, even after two cups of coffee!  ;)

    Steve

    Edit: spelling



  • I think it was perfect :)

    Then its settled.

    Have a merry christmas :)



  • I plugged a usb drive into my linux pc and ran
    sudo dd if=pfSense-2.0.2-RELEASE-4g-amd64-nanobsd_vga.img of=/dev/sdg

    (nanobsd)

    then i booted another pc with that usb drive and the first time it had the root mount error (actually the 2nd iteration of this process it didn't get that error), which was easy to resolve.
    i went through the initial set up, assigned my wan/lan, and was able to connect to internet via laptop->switch->lan port of pfsense->wan port of pfsense->modem
    then i went through the webconfigurator wizard, and everything was hunky dory.

    then i rebooted, and the initial config started all over again as if it had not written the settings to the drive, and this time it no longer was able to recognize one of my nics (onboard nic of a p4m800pro-m mobo).  it had no issue recognizing my intel pro/set pci nic

    why didn't it remember the config?
    why did it lose the driver module for my nic?
    should i boot a livecd and install the non-embedded version on this usb flash drive (corsair voyager mini 16GB) instead?


  • Netgate Administrator

    If it sees that the system config has changed, specifically if one the NICs has been removed, then it will trigger the initial setup to reconfigure the interfaces. This is the intended behaviour. Quite why it no longer saw your on board NIC I'm not sure.  :-\ Was there any clue in the logs? Is it auto disabled by something? (inserting another NIC?)

    Steve



  • i rebooted it remotely via the web interface, didn't touch the pc at all and after reboot it hates me
    i'm re-writing the usb stick right now and i'll try again and try to look through logs before i reboot it.

    i don't know how to get to a shell during the install to see the logs after the reboot, any idea?



  • as stephenw10 said, my problem was certainly due to pfsense detecting that my config had changed because after reboot freebsd showed no indication of the onboard nic at all in the logs.
    i put in an additional pci nic from my box of banished hardware and by using just the 2 pci nics and not the onboard, everything is cool now.

    i still don't know why freebsd/pfsense can't handle the onboard nic vr0 after initial config + reboot (works 100% of the time on "fresh" image before configuring), but i have lost all sympathy for the onboard nic and i'm going to bed

    thanks



  • Er…. Uh-oh??!  :o

    What I am reading here is that my install of the "LiveCD version" (AMD Architecture) direct to my 32GB USB key drive... rather than the "NanoBSD" version... may not be fully supported.  It's just that I am up and running just fine, with no issues.  I very much regard my USB drive as a "regular" drive.

    S'funny.  I guess I am used to running VMWare's ESXi product ("Apples and Oranges", I know) off of a USB key drive, I thought I could pull the same stunt with pfSense.

    So, is the concern about my method similar to what is expressed to those installing to CF?  ... a finite amount of read/writes... maybe I/Ops?  Overall performance?

    In the link given, I read  that the NanoBSD version is "suitable for use on a Compact Flash card (or other mass storage medium)" ...  "other mass storage medium" could possibly include USB keys, but until now, I thought they were quite distinct from CF-type drives...

    Maybe not so much?


  • Netgate Administrator

    Yes the finite write/erase cycles on flash media is the worry here. Many people are running a full install from flash with no issues though. The amount of writing to the card/stick is largely dependant on what your running.
    New in 2.1 is the ability to setup /tmp and /var as ram disks the same way it's done in Nano. That will reduce writes to the flash substantially.
    If you do run a full install from flash just be aware of the writes issue and keep a spare stick handy. Can you get SMART info from it?

    Steve



  • @stephenw10:

    Yes the finite write/erase cycles on flash media is the worry here. Many people are running a full install from flash with no issues though. The amount of writing to the card/stick is largely dependant on what your running.
    New in 2.1 is the ability to setup /tmp and /var as ram disks the same way it's done in Nano. That will reduce writes to the flash substantially.
    If you do run a full install from flash just be aware of the writes issue and keep a spare stick handy. Can you get SMART info from it?

    Steve

    Yup - the SMART widget in my dashboard seems to see my USB key drive fine (the status is "Passed").  But, now I am worried.  Gonna switch to HDD ASAP.

    Thanks for the insight!



  • I apologize for digging up an old post, but…

    I thought that most or all recent USB sticks had controller level wear leveling like newer SSD disks?

    I am wondering if the limited write issue is moot now because of this?

    (Especially if I buy a large, say 32GB USB stick, to give lots of room for wear leveling)

    Could this restriction be removed from the installer eventually to allow easy installation to a USB stick?


  • Netgate Administrator

    It's my understanding that most flash drives do have ware leveling of some type but that it's not of the same standard as that found in an SSD. Unlike an SSD they are not designed to be run as a boot drive.
    As I said before there are people running a full install from flash and many have been doing so for years.
    If you choose to do that make sure you have enough RAM to prevent any swap usage, that will massively increase writes, move /tmp and /var to ram drives. The Nano images mount the slices with the -noatime switch which also reduces the writes considerably. Consider editing the fstab to do that.

    One thing I must point out here is that earlier in this thread I believe I gave some bad advice. In fact the Nano images do not use the fstab instead using the scripts to mount the slices. I'm not sure how the above posters got past that.  :-\ It could explain why the initial setup started again for Drunkbumper.
    https://forum.pfsense.org/index.php?topic=66268.msg361539#msg361539

    Steve



  • Cool thanks. Sounds easier/better to use an SSD overall…

    ... I have since read more about USB Flash wear levelling... Though it is there, it's usually dynamic wear levelling, not static (as usually used on SSDs), read more here:

    http://en.wikipedia.org/wiki/Wear_leveling#Comparison

    http://web.archive.org/web/20071013150729/http://www.corsair.com/_faq/FAQ_flash_drive_wear_leveling.pdf



  • Been running full install on 4GB  USB flash drives for about 18 months now.  On the second $5 stick now.  The first one die in about 6 months.  But I was doing a lot of 2.1 beta upgrades and also running without the tmp and var ram disk during that time.  Also the first stick was plugged directly into the notebook so would pick up quite a bit of heat.  This second stick is connected with a USB extension cable so it stays much cooler and also not been doing may upgrades and running with the tmp and var ram disk enabled.

    Not running any packages.

    Performance of USB flash drive can vary significantly between models.  USB 3.0 and/or ReadyBoost compatible will probably be the best chance for good performance.  Even if USB port is only 2.0, performance may be better with a 3.0 device.

    Where I notice performance differences with USB flash drive is 1) during install process, 2) upgrade process, 3) web GUI responsiveness.



  • @NOYB:

    Been running full install on 4GB  USB flash drives for about 18 months now.  On the second $5 stick now.  The first one die in about 6 months.  But I was doing a lot of 2.1 beta upgrades and also running without the tmp and var ram disk during that time.  Also the first stick was plugged directly into the notebook so would pick up quite a bit of heat.  This second stick is connected with a USB extension cable so it stays much cooler and also not been doing may upgrades and running with the tmp and var ram disk enabled.

    Not running any packages.

    Performance of USB flash drive can vary significantly between models.  USB 3.0 and/or ReadyBoost compatible will probably be the best chance for good performance.  Even if USB port is only 2.0, performance may be better with a 3.0 device.

    Where I notice performance differences with USB flash drive is 1) during install process, 2) upgrade process, 3) web GUI responsiveness.

    Is there a simple way of doing this? I'm going to make a USB installable key for pfsense shortly on a test machine, do I simply plug it in, plug in my destination USB key,  boot off the installer and then install to the destination key?

    I'm content with it frying the USB key, this is purely for testing.
    As a long term solution, I'll consider other means (although some kind of mirrord USB key, like FreeNAS offers would be quite nice)

    Note: I don't want the basic nano install.

    Cheers


  • Netgate Administrator

    @potentialcustomer:

    do I simply plug it in, plug in my destination USB key,  boot off the installer and then install to the destination key?

    Yes.

    You could gmirror across two USB drives. Or even ZFS across them in 2.4Beta if you wanted.

    Running from USB is not recommended though. Fine for a test as you said though.

    Nano is not particularly limited in 2.3.X. It will probably do all you need to test out pfSense.

    Steve