Booting pfSense from USB flash on a fit-PC - near success (detailed)

CliftonR

Hi, all. I've been able to make substantial headway on 1) creating a pfSense bootable install image for a USB flash drive, and 2) installing on fit-PC from a USB flash drive. Here's some info which should be helpful to others as far as what to try, what not to try. FWIW, I've been using and administering FreeBSD since 3.4, though I'm by no means a kernel hacker, so I understand at least some key parts of the whole boot process.

Unfortunately, on the fit-PC I'm stuck at the very last stage of kernel boot, after the mountroot step. I'd really rather not have to mail-order a USB CD/DVD drive, or order a notebook IDE adapter and pull apart the fit-PC to get this to work, especially since I think I'm very close.

How to configure a fit-PC (v1) to boot from a USB flash drive

Hit Del while booting the fit-PC to go into the BIOS. On the left-hand menu for BIOS drive mappings, change the mapping for C: to "Nand Flash". Leave the boot priority set to boot from "C:" Save settings and exit to boot from the USB drive. That's all.

NOTE: It is logical that you should be able to configure D: as "Nand Flash", leaving C: as the internal hard drive, and then set the boot priority to boot from D: ahead of C: This doesn't work, so don't waste any time on it.

Using this approach, I was able to convert the FreeBSD 7.1-RELEASE bootonly ISO image into an image (see below), load it onto a flash drive, and boot up into sysinstall with no problems.

Booting an embedded pfSense image on the fit-PC

Not so successful.

This was actually the second thing to try, but logically it should have been the first one, so I'll list it here. I used dd to copy the unzipped pfSense-1.2.2-Embedded.img file to a standard USB flash drive, same one I used for the FreeBSD 7.1 test:
sudo dd if=pfSense-1.2.2-Embedded.img of=/dev/da0 bs=16k

It executes the initial boot block stage successfully, then gives errors:
error 170 lba 0
error 170 lba 0
No /boot/loader
and comes out to the standard stage 1 boot prompt:
FreeBSD/i386 boot
Default: 0:ad(0,a)/boot/kernel/kernel

At this stage the keyboard is working, but any attempt to point it to the next stage loader or kernel fails. Enter gives
boot: error 170 lba 0
No /boot/kernel/kernel
I tried some permutations of 0:da(0,a)/boot/loader, et al. but all failed the same way.

How to build a bootable FreeBSD USB Flash image from a FreeBSD or pfSense ISO

I found several scripts for this; after some testing, the best one seems to be Dario Freni's posted to freebsd-hackers in 2006, but I made some enhancements to it to make it more usable with the pfSense ISO and other FreeBSD ISOs. Beerware license; you owe Dario a beer if you find it useful. (I wouldn't turn down one myself if you like the enhancements.)

#!/bin/sh
# Original version by Dario Freni 9/2006
# Options and enhancements by Clifton Royston 3/2009.
# License: Beerware

# You can set some variables here. Edit them to fit your needs.

# Set serial variable to 0 if you don't want serial console at all,
# 1 if you want comconsole and 2 if you want comconsole and vidconsole
serial=0

# Set nofstab=1 here or with "-n" to not create any initial fstab on the USB drive;
# this makes the next two settings largely irrelevant.
nofstab=0

# Set rootperm=rw for root fs to mount r/w from the USB drive
#  (Should be unnecessary.)
rootperm=ro

# Set USBLABEL here or with -L label to label the image file system,
#  to help the loader find the root file system when booting;
#  otherwise the USB must come up as da0 to finish loading successfully.
USBLABEL=
lbparams=

# Set dopause=1 here or with -p to pause and allow review or editing of
#  the flash image before finalizing the image.
dopause=0

pause() {
 echo "Press enter to continue"
 read foo
}

set -u

if [ $# -ge 3 ]; then
  flag=$1
  if [ ${flag} = "-p" ]; then
    dopause=1
    shift
    flag=$1
  fi
  if [ ${flag} = "-n" ]; then
    nofstab=1
    shift
    flag=$1
  fi
  if [ ${flag} = "-L" ]; then
    shift;
    USBLABEL=$1; shift
    lbparams="-L ${USBLABEL}"
  fi
fi
if [ $# -lt 2 ]; then
  echo "Usage: $0 [-p] [-n] [-L vollabel] source-iso-path output-img-path"
  echo "        -p      pause for review before finalizing image"
  echo "        -n      don't update the /etc/fstab within the image"
  echo "        -L      set file system label on image, to help loader find it"
  exit 1
fi

isoimage=$1; shift
imgoutfile=$1; shift

export tmpdir=$(mktemp -d -t fbsdmount)

# Temp file and directory to be used later
export tmpfile=$(mktemp -t bsdmount)

export isodev=$(mdconfig -a -t vnode -f ${isoimage})

echo "#### Building bootable UFS image ####"

ISOSIZE=$(du -k ${isoimage} | awk '{print $1}')
SECTS=$((($ISOSIZE + ($ISOSIZE/5))*2))

# Root partition size

echo "Initializing image..."
dd if=/dev/zero of=${imgoutfile} count=${SECTS}
ls -l ${imgoutfile}
export imgdev=$(mdconfig -a -t vnode -f ${imgoutfile})

bsdlabel -w -B ${imgdev}
newfs -O1 ${lbparams} /dev/${imgdev}a

mkdir -p ${tmpdir}/iso ${tmpdir}/img

mount -r -t cd9660 /dev/${isodev} ${tmpdir}/iso
mount /dev/${imgdev}a ${tmpdir}/img

echo "Copying files to the image via cpio"
( cd ${tmpdir}/iso && find . -print -depth | cpio -dump ${tmpdir}/img )
# Dump doesn't work from an ISO file system, too bad.
# echo "Copying files to the image via dump/restore..."
## dump -0f - /dev/${isodev} | (cd ${tmpdir}/img && restore -r -f - )

#bzcat ${tmpdir}/iso/dist/root.dist.bz2 | mtree -PUr -p ${tmpdir}/img 2>&1 > /d

if [ ${nofstab} -ne 1 ]; then
  echo "Saving original /etc/fstab as /etc/fstab.orig"
  mv ${tmpdir}/img/etc/fstab  ${tmpdir}/img/etc/fstab.orig
  echo "Replacing /etc/fstab, so loader can find root filesystem on flash!"
  if [ "${USBLABEL}" != "" ]; then
    echo "/dev/ufs/${USBLABEL} / ufs ${rootperm} 0 0" > ${tmpdir}/img/etc/fstab
    ## echo "devfs /dev devfs rw 0 0" >> ${tmpdir}/img/etc/fstab
  else
    echo "/dev/da0a / ufs ${rootperm} 0 0" > ${tmpdir}/img/etc/fstab
    ## echo "devfs /dev devfs rw 0 0" >> ${tmpdir}/img/etc/fstab
  fi
else
  echo "Skipping write of image /etc/fstab"

fi

if [ ${serial} -eq 2 ]; then
        mv ${tmpdir}/img/boot.config ${tmpdir}/img/boot.config.orig
        mv ${tmpdir}/img/boot/loader.conf ${tmpdir}/img/boot/loader.conf.orig
        echo "-D" > ${tmpdir}/img/boot.config
        echo 'console="comconsole, vidconsole"' >> ${tmpdir}/img/boot/loader.co
elif [ ${serial} -eq 1 ]; then
        mv ${tmpdir}/img/boot.config ${tmpdir}/img/boot.config.orig
        mv ${tmpdir}/img/boot/loader.conf ${tmpdir}/img/boot/loader.conf.orig
        echo "-h" > ${tmpdir}/img/boot.config
        echo 'console="comconsole"' >> ${tmpdir}/img/boot/loader.conf
fi

if [ ${dopause} -eq 1 ]; then
  echo "Pausing to allow manual review and modification of image file:"
  echo "Image is located in ${tmpdir}/img"
  echo "If you need to fix up ${tmpdir}/img/etc/fstab, now is the time."
  pause
fi

cleanup() {
    umount ${tmpdir}/iso
    mdconfig -d -u ${isodev}
    umount ${tmpdir}/img
    mdconfig -d -u ${imgdev}
    rm -rf ${tmpdir} ${tmpfile}
}

cleanup

ls -lh ${imgoutfile}

echo "To write the image to flash, use dd, for example:"
echo "   dd if=${imgoutfile} of=/dev/da0 bs=4M"

This works really nicely; it built an image from the FreeBSD 7.1 bootonly ISO with:

sudo fbsd-install-iso2img -p -n ~/7.1-RELEASE-i386-bootonly.iso ~/7.1-RELEASE-i386-bootonly.img

(Note, no fstab created or required.) This boots on the fit-PC as well as on my desktop.

It also builds an image nicely from the pfSense 1.2.2 LiveCD, using either the volume label capability:

sudo fbsd-install-iso2img -p -L pfSense ~/pfSense-1.2.2-LiveCD-Installer.iso ~/pfSense-1.2.2-LiveFlash-Installer.img

or hardcoding /dev/da0a as root:

sudo bin/fbsd-install-iso2img -p  ~/pfSense-1.2.2-LiveCD-Installer.iso ~/pfSense-1.2.2-LiveFlash-Installer.img

Note: if you suppress the fstab with "-n", you'll get the usual "unable to mount root" scenario. With either of these, they find it and mount root successfully.

Unfortunately, they fail in two ways just after that. See next.

Problems booting from a pfSense USB flash image

If you build an image as above, it will boot through the first two stages of the loader fine; you get to the standard loader menu with the pfSense version of beastie. The keyboard is functional through this stage. Choosing "no ACPI" or "Safe mode" die horribly (page fault & restart) which is not untypical for modern chipsets. Choosing either the default or single-user boot result in the kernel booting normally; all hardware seems to be detected, devices appear, the actual hard drive (ad0) and USB drive (da0) appear near the end of the boot, and if the image was built using a label, the "pfSense" label on the file system is detected. However,

by the time it gets to the mountroot prompt (where it was getting if I screwed up the image /etc/fstab or removed it) it has lost the keyboard, so there's no way to respond to a prompt.
and, if it has the fstab configured so it can find root, it mounts the root, and then immediately reports the following series of errors:

Lookup of /dev for devfs, error 20
exec /sbin/init: error 20
exec /sbin/oinit: error 20
…

It runs through a series of possible backup locations for init, then reboots. As I have no keyboard, I can't press a key to suspend the reboot, so I've had to copy this by hand and can't include all the device messages.

Does anybody have any idea why it would be able to mount the root file system and then be unable to create /dev? I've never seen that happen before. (The /dev mountpoint does exist in the image; I checked that that's not the problem.)

Things I plan to try:

Build a USB image from the FreeBSD 7.1 liveCD, disc1.iso and see if that works on the fit-PC.
Re-check that I can boot the pfSense USB stick on my main desktop computer. (I did this at one point, and it got far enough into the installer to complain that I had only one LAN interface, so that was working at that point.)

Any other ideas?

CliftonR

P.S. It's probably obvious to many readers, but I should have noted that all the steps for creating a bootable USB drive from an ISO need to be run on a FreeBSD machine. (Definitely not on Linux or Windows.) I was running them on 6.4-RELEASE, specifically.

CliftonR

This evening, I downloaded one of the 1.2.3 snapshots, as the developers have been recommending to others, ran the ISO to image script I posted above, copied the image to flash, and had immediate success.

pfSense is now up and running on the fit-PC; no need to pull the box apart, no need for an actual CD-ROM anywhere in the process. At present, I'm running off the flash drive and having fun playing with the set-up; later I'll wipe out the Linux install and move the image onto the hard drive.

Creating a pfSense 1.2.3 Bootable USB Flash Drive Installer Image

The exact command line used to create the flash image on a FreeBSD system, using the shell script listed above, was:

sudo bin/fbsd-install-iso2img -p -L pfSense /data/isos/pfSense-1.2.3-20090307-1245.iso fSense-1.2.3-LiveFlash-Installer.img

Copy it to your flash drive with dd, according to the instructions the script gives you.

Then configure your fit-PC to boot off USB as described above, hook up video and keyboard (temporarily) and it should boot right into the 1.2.3 installer with no problems.

I'm incredibly impressed with the web interface and the capabilities. I've set it up for a bridging firewall on my statically allocated network, which took mere minutes, and I'm transposing all the ipf rules I had written on my old firewall into the pfSense web interface, and finding it easy going. It looks like I should be able to recreate all the old functionality I used to have running, with about 1/100th the effort.

GoldServe

Can someone with a freebsd machine make an image file so I can DD to a microdrive? I'm having an extremely difficult time using the ISO trying to install to the microdrive.

Thanks!

matrix200 · Current network "hardware" : Running 2.2RC in Virtualbox 4.2.16.

I join the petition :)
Also having issues installing full version on a microdrive for alix.
Would be great if one of the lucky ones who have the setup working could share their image.

timurx

Follow our guidelines on SecMachine.com to create a pfsense insteller on usb flash drive.
There are two guides:
The generic one: https://secmachine.com/wp/how-tos/construct-usb-installer-freebsd/
And the particular pfsense guideline: https://secmachine.com/wp/how-tos/pfsense-installation/construct-usb-installer/

the latter is part in the chain of our guide to install hardened pfSense

Booting your fit-pc off this installer on usb key you will be able to install pfSense on the final usb flash drive.

You do not need a freebsd box for the exercise, since the pfSense LiveCD will make any pc a freebsd box for that short time you need it (you are supposed to have an other pc box besides the fit one ;)). Though using a normal freebsd installation will be more comfortable as you can copy-paste the commands from the guide and avoid vi and csh.

Your comments are welcome.

djzort

this worked perfectly for me, i would really like to see this added to the wiki.

also an 'official' usb install image would be great.

i personally struggle to find a cd-r / cd-rw but i have otherwise useless <1gig usb's lying around which are great for the purpose of installing OS's

clyss

I join also the petition …
I wasn't able yet to create the img
Would be great if one of the lucky ones who have the setup working could share their image...
I can find space to put it if needed...
THANKS:)

dreamslacker

@GoldServe:

Can someone with a freebsd machine make an image file so I can DD to a microdrive? I'm having an extremely difficult time using the ISO trying to install to the microdrive.

Thanks!

If you're running Windows, try downloading and using Unetbootin.

clyss

Hello,
Dan just put on rapidshare a version made with this script…
you can download it on :
http://rapidshare.com/files/295724278/pfSense-1.2.3-RC1-LiveFlash-Installer.img.bz2
THANKS DAN :)

clyss

To reply to a private msg, the previous image WORK very well, including if you write it from WINDOWS (including Vista X64)
Just use PhysDiskWrite and not DD…
Thanks again DAN :)

mevans336

@dreamslacker:

@GoldServe:

Can someone with a freebsd machine make an image file so I can DD to a microdrive? I'm having an extremely difficult time using the ISO trying to install to the microdrive.

Thanks!

If you're running Windows, try downloading and using Unetbootin.

UNetbootin doesn't write the pfSense image properly, although it writes a generic FreeBSD 7 ISO just fine.

Cbyrne

Hey,

Im not trying to revive an old thread but I thought I would share on my success.

After spending about 12-14 hours trying to get my old toshiba 3505 laptop to boot to the pfsense image I finally had some luck. The laptop doesn't have a cd-rom drive or support usb cdrom drives or boot from usb. but it does support usb floppy drives. So if any of you find your self in this situation here is what I did. Keep in mind this was all done in Windows 7.

Use the PLoP bootmanager found here
http://www.plop.at/en/bootmanager.html#runflp
create the floopy image.

This allows your computer to boot to a mini bootmanager that has usb drivers and supports usb booting if your computer doesn't.

Then use flashnul to write the image that Dan made and clyss linked http://rapidshare.com/files/295724278/pfSense-1.2.3-RC1-LiveFlash-Installer.img.bz2
The flashnul program worked while UNetbootin and PhysDiskWrite did not. flashnul found here http://shounen.ru/soft/flashnul/ or the English translated version here http://translate.google.com/translate?u=http%3A%2F%2Fshounen.ru%2Fsoft%2Fflashnul%2F&hl=en&ie=UTF8&sl=ru&tl=en

Start the command prompt as an administrator (type cmd in the start menu search then when the icon appears right click and run as administrator)
Go to the directory where you extracted flashnul
flashnul -p this allows you to see your drives
example

D:\flashnuldir>flashnul -p

Avaible physical drives:
0       size = 1000204886016 (931 Gb)
1       size = 293462867968 (273 Gb)
2       size = -1 ( -not avaible- )
3       size = -1 ( -not avaible- )
4       size = 2031091712 (1937 Mb)

Avaible logical disks:
A:\
C:\
D:\
E:\
F:\
G:\
H:\
I:\
J:\

Press ENTER to exit.

Next enter the the parameters of the drive letter and where the image is at
flashnul [drive letter] -L [path to .img]
Ex. flashnul I: -L d:\flashnuldir\pfsense.img

my Example

D:\flashnuldir>flashnul I: -L d:\flashnuldir\pfSense-1.2.3-RC1-LiveFlash-Installer.img
GetFreeDiskSpaceEx() failed: Access is denied.

Disk I: (UNC name: \\.\I:)
------------------------------------------------------------[Drive geometry]--
Cylinders/heads/sectors  = 246/255/63
Bytes per sector         = 512
CHS size                 = 2023418880 (1929 Mb)
---------------------------------------------------------------[Device size]--
Device size              = 2031091712 (1937 Mb)
delta to near power of 2 = 116391936 (111 Mb), 5%
-----------------------------------------------[Adapter & Device properties]--
Bus type                 = (7) USB
Removable device         = Yes
Command Queue            = Unsupported
Device vendor            = Kingston
Device name              = DataTraveler 2.0
Revision                 = PMAP
Device serial            = ▼
--------------------------------------------------------------[Hotplug info]--
Device hotplug           = Yes
Media hotplug            = No

        Selected operation:      load file content
        Selected drive:          I:, 2031091712b (1937 Mb)

        THIS OPERATION IS DESTRUCTIVE!!!
        Type 'yes' to confirm operation. All other text will stop it.

        Really destroy data on drive I:?  :

make sure that's your flash drive you want! and enter "yes" then there will be some progress stats and then it will say if there were any errors if not than it should be good to go!

Hopefully this is useful to anyone in the future. I'm just hoping that my netgear gigabit PCMCIA nic is supported now.

Thanks again for the useful information above my post and Dan for the image.