Transfer image from one CF to another
-
Please mail me a gun so I can shoot myself. Also, please promote use of the gun in your documentation but don't explain it's proper use. Also, when someone asks about proper use of the gun, please inform them they are on their own.
See how frickin' stupid that sounds? Notice any parallels to your statement(s) above? I'm not trying to be mean, I'm trying to point out some easily-resolvable issues with mentioning RawDisk in the Wiki. It only has 3 or 4 command-line options, so it can't be too easy to screw up. :o Read, write, disk enumeration… What's not to understand about that?
If there's something that may (even theoretically) need to be done following such a cloning process, such as change a disk label or something, then clue me in. I'm assuming this CAN work, since it's on the Wiki. I'm looking for those who can help explain what needs to happen to make it work.
So, please don't reply to me if you:
- Think I'm stupid
- Don't have a solution or any thoughts that might begin to point me in the direction of a solution
- Want to respond saying how busy you are
- Can't definitively say "this will never work"
Cause guess what? None of those help anyone. If you don't have a solution, don't waste your time replying to me. Please ignore me. I don't want to spoil any more dev time than this thread has already done.
-
Free software comes with no implied warranty what so ever. The wiki has tons of experimental stuff, so has HEAD code, if you like to use it, by all means do so. But do not complain about stuff not working, and do not expect others already sharing their work for free, to actually comply to the demands of users !!
A lot of devs feels like, hey I give something away, so that others may benefit from my research/work, and what I get in return is more "work".
Why should we even support pfSense at all ? We do aprechiate bug reports, but lately everything seems to change into demands !
If you really need help then post a bounty and put your money where your mouth is.
I'm sure that your need to put the same setup on 9 boxes means this is somehow connected to a comercial project of some sort, so take the money you saved by getting a free firewall and put some of them up as a bounty and see if you can get help. -
You guys just don't get what I'm trying to say, do you? I ask a few questions, and it devolves into "everyone's picking on me" whining.
I'm not demanding anything, and I never have. The devs are wasting everyone's time by replying "That's not supported" or similar. Whether a dev will support it or not is actually COMPLETELY IRRELEVANT to the issue. I don't expect support from the devs. I post here because this is typically referred to as a "user forum". I'm posting to see if another "user" has done something similar to what I'm trying to do. If none has, then so be it…maybe I'll be the first to post the method here.
Seriously, you devs need to take a chill pill and quit trying to be Superman. You're burning yourselves out. I may post a bounty just so some of you can raise some beer money. ;)
My advice to the devs, if any of them gave a flyin' rip, would be to NOT try to be the pfSense tier 1 support department. Spend your precious time on true problems or issues with the software, not on average joe's like me just trying to get things to work. In short, if it ain't broke, shut up and walk away. Believe me, I mean this with all possible respect.
-
on windows xp open the pfsense image with winrar
make youre chanes now close winrar
and burn youre chaned image to cd or cf as normal -
Why don't you just use dd to read the image and write it back. Theres a version for windows xp if you need that. I've used it before to backup an image and write it back when I was testing out some things.
Just did a quick google search and this looks like the page I used.
http://www.chrysocome.net/dd
-
I use "dd" for Windows to dup[licate] my pfSense CF cards (of same brand/model/size); works great.
(although, it's just as easy to do a pfSense backup, edit the .xml with new IPs, restore into newly flashed CF).dd –list
My flash card on this PC is always "6", and "6" is the same disk number I use in "physdiskwrite.exe".
Copy from CF to hd...
dd bs=16k if=\?\Device\Harddisk6\Partition0 of=foo.img.gz
Copy from hd to CF...
dd bs=16k if=foo.img.gz of=\?\Device\Harddisk6\Partition0
-
I have created some scripts that I use to create a custom image to flash to my compact flash card. It starts with the base RC3 release, adds the RC3a, RC3b, RC3c, RC3d patches, replaces the stock config with mine, and adds in miniupnpd.
You have to use it on a freebsd system and I personally use freesbie in vmware. I run the one file, it downloads the files from my webserver and goes through all the steps to create the image. When its done I upload the file and flash it on my windows box with physdiskwrite.
Here are the files. Look in each one for a description of what it does. sh-custom-image.sh calls them all and does the dirty work. If you use miniupnpd you will need to add that cfg-add-miniupnpd to your config file <installedpackages>section to make it work.
Slightly modification of the below files should get you a working way to setup your images.
http://wgnrs.dynalias.com:81/pfsense/sh-custom-image.sh
http://wgnrs.dynalias.com:81/pfsense/sh-add-patch.sh
http://wgnrs.dynalias.com:81/pfsense/sh-replace-config.sh
http://wgnrs.dynalias.com:81/pfsense/sh-add-miniupnpd.shhttp://wgnrs.dynalias.com:81/pfsense/miniupnpd.tar.gz
http://wgnrs.dynalias.com:81/pfsense/cfg-add-miniupnpd.xml</installedpackages> -
I have created some scripts that I use to create a custom image to flash to my compact flash card. It starts with the base RC3 release, adds the RC3a, RC3b, RC3c, RC3d patches, replaces the stock config with mine, and adds in miniupnpd.
You have to use it on a freebsd system and I personally use freesbie in vmware. I run the one file, it downloads the files from my webserver and goes through all the steps to create the image. When its done I upload the file and flash it on my windows box with physdiskwrite.
Here are the files. Look in each one for a description of what it does. sh-custom-image.sh calls them all and does the dirty work. If you use miniupnpd you will need to add that cfg-add-miniupnpd to your config file <installedpackages>section to make it work.
Slightly modification of the below files should get you a working way to setup your images.
http://wgnrs.dynalias.com:81/pfsense/sh-custom-image.sh
http://wgnrs.dynalias.com:81/pfsense/sh-add-patch.sh
http://wgnrs.dynalias.com:81/pfsense/sh-replace-config.sh
http://wgnrs.dynalias.com:81/pfsense/sh-add-miniupnpd.shhttp://wgnrs.dynalias.com:81/pfsense/miniupnpd.tar.gz
http://wgnrs.dynalias.com:81/pfsense/cfg-add-miniupnpd.xml</installedpackages>Neat-o ;D
-
I have created some scripts that I use to create a custom image to flash to my compact flash card. It starts with the base RC3 release, adds the RC3a, RC3b, RC3c, RC3d patches, replaces the stock config with mine, and adds in miniupnpd.
You have to use it on a freebsd system and I personally use freesbie in vmware. I run the one file, it downloads the files from my webserver and goes through all the steps to create the image. When its done I upload the file and flash it on my windows box with physdiskwrite.
Here are the files. Look in each one for a description of what it does. sh-custom-image.sh calls them all and does the dirty work. If you use miniupnpd you will need to add that cfg-add-miniupnpd to your config file <installedpackages>section to make it work.
Slightly modification of the below files should get you a working way to setup your images.
http://wgnrs.dynalias.com:81/pfsense/sh-custom-image.sh
http://wgnrs.dynalias.com:81/pfsense/sh-add-patch.sh
http://wgnrs.dynalias.com:81/pfsense/sh-replace-config.sh
http://wgnrs.dynalias.com:81/pfsense/sh-add-miniupnpd.shhttp://wgnrs.dynalias.com:81/pfsense/miniupnpd.tar.gz
http://wgnrs.dynalias.com:81/pfsense/cfg-add-miniupnpd.xml</installedpackages>Forgot to mention…
The miniupnpd.tar.gz files are the latest files with the 20060924 binary. I tried out the 20060930 binary and it seems to have regressed with errors filling the system log. You can read about here
http://forum.pfsense.org/index.php/topic,551.msg13483.html#msg13483
Miniupnpd uses the miniupnp.sh script to start the service. This file contains your ip address and nic interface hardcoded. Not a big deal as the sync_package_miniupnpd() recreates it. However it doesn't seem to run this when syncing packages at startup. I looked in the code and it looks like the miniupnpd.xml file should have a custom_php_resync_config_command section which calles sync_package_miniupnpd(). I think the thought was that once its configured theres really no need to keep recreating the same startup file.
All I did was go to the miniupnpd settings page and click change (this calles sync_package_miniupnpd) and the service restarted and it worked from then on out. The ideal way, which I do now is to modify the miniupnpd.sh file which is in miniupnpd.tar.gz link above. Find the below line. Change fxp2 to your wan interface name and the 10.10.1.1 to your lan ip address.
killall miniupnpd; /usr/local/sbin/miniupnpd -p 2869 -i fxp2 -a 10.10.1.1
Now when you build and flash your image you can stick it in and your good to go.
-
Forgot to mention…
The miniupnpd.tar.gz files are the latest files with the 20060924 binary. I tried out the 20060930 binary and it seems to have regressed with errors filling the system log. You can read about here
http://forum.pfsense.org/index.php/topic,551.msg13483.html#msg13483
Miniupnpd uses the miniupnp.sh script to start the service. This file contains your ip address and nic interface hardcoded. Not a big deal as the sync_package_miniupnpd() recreates it. However it doesn't seem to run this when syncing packages at startup. I looked in the code and it looks like the miniupnpd.xml file should have a custom_php_resync_config_command section which calles sync_package_miniupnpd(). I think the thought was that once its configured theres really no need to keep recreating the same startup file.
All I did was go to the miniupnpd settings page and click change (this calles sync_package_miniupnpd) and the service restarted and it worked from then on out. The ideal way, which I do now is to modify the miniupnpd.sh file which is in miniupnpd.tar.gz link above. Find the below line. Change fxp2 to your wan interface name and the 10.10.1.1 to your lan ip address.
killall miniupnpd; /usr/local/sbin/miniupnpd -p 2869 -i fxp2 -a 10.10.1.1
Now when you build and flash your image you can stick it in and your good to go.
With the recent comitted changes it is no longer necessary to configure the miniupnpd.sh file. It will be automatically created upon first boot. It's as simple as using my scripts with the included files and your good to go.
-
Your method didn't work for me, I tried to embed my config but it didn't seem to take. I edited in the appropriate XML stuff first and no worky…
Maybe it's because it couldn't find an RC3f patch to apply? All that is on the mirrors is RC3e, at least as of yesterday night.
-
here is the link to rc3f
http://www.pfsense.com/~sullrich/pfSense-1.0-RC3f-Full-Embedded.tgz -
@bradenmcg:
Your method didn't work for me, I tried to embed my config but it didn't seem to take. I edited in the appropriate XML stuff first and no worky…
Maybe it's because it couldn't find an RC3f patch to apply? All that is on the mirrors is RC3e, at least as of yesterday night.
RC3f was in the sullrich folder or on my site with the scripts. Although now it doesn't matter as 1.0 is out. Check out the following files they should do what you want.
http://wgnrs.dynalias.com:81/pfsense/sh-replace-config.sh
http://wgnrs.dynalias.com:81/pfsense/sh-add-miniupnpd.sh
http://wgnrs.dynalias.com:81/pfsense/miniupnpd.tar.gz
http://wgnrs.dynalias.com:81/pfsense/cfg-add-miniupnpd.xmlOR
http://wgnrs.dynalias.com:81/pfsense/sh-custom-image.sh
As I rewrote the the above script. It is updated for 1.0-RELEASE, includes error checking, and the ability to specify a config file at the command line. If no config file is specified it will use the default config file with the miniupnpd package section added.
Download file to freebsd machine
chmod +x sh-custom-image.sh
./sh-custom-image.sh
or
./sh-custom-image.sh yourconfig.xmlTry the above and let me know if it doesn't work.
-
and more for the colection: sh-add-files.sh
this file will read the file filelist and add the files listed there to the image
start the file with sh-add-files.sh pfSense.img
use the complete filepath to the file in file filelist
like this: /bin/routed -
added some text to sh-add-miniupnpd.sh
it will now check of the config for miniupnp is in the config.xml
if its not it will download the file cfg-add-miniupnpd.xml
and add it to the correct place in config.xml#!/bin/sh # This script adds the miniupnpd files to image specified. # Make sure the miniupnpd.tar.gz file is in the script directory. # # # The miniupnpd.tar.gz file was made by creating a directory named # miniupnpd. In there should be the following files: # miniupnpd, miniupnpd.sh, miniupnpd.inc, miniupnpd.xml, status_upnp.php # Then you execute the command "tar cfz miniupnpd.tar.gz miniupnpd/" # # Usage {miniupnpd} {imagefile} # Example ./sh-add-miniupnpd.sh miniupnpd.tar.gz pfSense.img TARFILE="$1" IMGFILE="$2" WORKDIR=`pwd` UPNPDIR="$WORKDIR/miniupnpd" MD=`mdconfig -a -t vnode -f $WORKDIR/$IMGFILE` mkdir $WORKDIR/a mount /dev/${MD}a $WORKDIR/a tar zxf $TARFILE cp $UPNPDIR/miniupnpd.inc $WORKDIR/a/usr/local/pkg/miniupnpd.inc chmod 755 $WORKDIR/a/usr/local/pkg/miniupnpd.inc cp $UPNPDIR/miniupnpd.xml $WORKDIR/a/usr/local/pkg/miniupnpd.xml chmod 644 $WORKDIR/a/usr/local/pkg/miniupnpd.xml cp $UPNPDIR/miniupnpd $WORKDIR/a/usr/local/sbin/miniupnpd chmod 755 $WORKDIR/a/usr/local/sbin/miniupnpd cp $UPNPDIR/status_upnp.php $WORKDIR/a/usr/local/www/status_upnp.php chmod 755 $WORKDIR/a/usr/local/www/status_upnp.php cd $UPNPDIR echo -n "looking for miniupnp config in config.xml.." if grep -q miniupnp $WORKDIR/a/conf/config.xml then echo "..miniupnp config found." else echo "..miniupnp config not found." cp $WORKDIR/a/conf/config.xml $UPNPDIR/test.xml echo "downloading miniupnp xml config" fetch -q -o cfg_add_miniupnp.xml http://wgnrs.dynalias.com:81/pfsense/cfg-add-m iniupnpd.xml # the downloaded file has <installedpackages>and</installedpackages> #the test.xml has them to so we remove them from the downloaded file echo "removing not needed text" sed -e '/<installedpackages>/ d' -e '/<\/installedpackages>/ d' cfg_add_miniupn p.xml >upnp.xml # now we add upnp.xml to test.xml afther <installedpackages>echo "adding miniupnp config." sed '/<installedpackages>/ r upnp.xml' test.xml > output.xml mv output.xml $WORKDIR/a/conf/config.xml fi echo “all done” rm -rf miniupnpd umount $WORKDIR/a rm -rf $WORKDIR/a mdconfig -d -u ${MD}</installedpackages></installedpackages></installedpackages> -
added some text to sh-add-miniupnpd.sh
it will now check of the config for miniupnp is in the config.xml
if its not it will download the file cfg-add-miniupnpd.xml
and add it to the correct place in config.xml#!/bin/sh # This script adds the miniupnpd files to image specified. # Make sure the miniupnpd.tar.gz file is in the script directory. # # # The miniupnpd.tar.gz file was made by creating a directory named # miniupnpd. In there should be the following files: # miniupnpd, miniupnpd.sh, miniupnpd.inc, miniupnpd.xml, status_upnp.php # Then you execute the command "tar cfz miniupnpd.tar.gz miniupnpd/" # # Usage {miniupnpd} {imagefile} # Example ./sh-add-miniupnpd.sh miniupnpd.tar.gz pfSense.img TARFILE="$1" IMGFILE="$2" WORKDIR=`pwd` UPNPDIR="$WORKDIR/miniupnpd" MD=`mdconfig -a -t vnode -f $WORKDIR/$IMGFILE` mkdir $WORKDIR/a mount /dev/${MD}a $WORKDIR/a tar zxf $TARFILE cp $UPNPDIR/miniupnpd.inc $WORKDIR/a/usr/local/pkg/miniupnpd.inc chmod 755 $WORKDIR/a/usr/local/pkg/miniupnpd.inc cp $UPNPDIR/miniupnpd.xml $WORKDIR/a/usr/local/pkg/miniupnpd.xml chmod 644 $WORKDIR/a/usr/local/pkg/miniupnpd.xml cp $UPNPDIR/miniupnpd $WORKDIR/a/usr/local/sbin/miniupnpd chmod 755 $WORKDIR/a/usr/local/sbin/miniupnpd cp $UPNPDIR/status_upnp.php $WORKDIR/a/usr/local/www/status_upnp.php chmod 755 $WORKDIR/a/usr/local/www/status_upnp.php cd $UPNPDIR echo -n "looking for miniupnp config in config.xml.." if grep -q miniupnp $WORKDIR/a/conf/config.xml then echo "..miniupnp config found." else echo "..miniupnp config not found." cp $WORKDIR/a/conf/config.xml $UPNPDIR/test.xml echo "downloading miniupnp xml config" fetch -q -o cfg_add_miniupnp.xml http://wgnrs.dynalias.com:81/pfsense/cfg-add-m iniupnpd.xml # the downloaded file has <installedpackages>and</installedpackages> #the test.xml has them to so we remove them from the downloaded file echo "removing not needed text" sed -e '/<installedpackages>/ d' -e '/<\/installedpackages>/ d' cfg_add_miniupn p.xml >upnp.xml # now we add upnp.xml to test.xml afther <installedpackages>echo "adding miniupnp config." sed '/<installedpackages>/ r upnp.xml' test.xml > output.xml mv output.xml $WORKDIR/a/conf/config.xml fi echo “all done” rm -rf miniupnpd umount $WORKDIR/a rm -rf $WORKDIR/a mdconfig -d -u ${MD}</installedpackages></installedpackages></installedpackages>Have you tested this script out?? As you need to mount the other partition in the image as thats where the config is stored. Look at my sh-mount-image.sh or sh-replace-config.sh
http://wgnrs.dynalias.com:81/pfsense/sh-mount-image.sh
http://wgnrs.dynalias.com:81/pfsense/sh-replace-config.shRegardless I like the script. Billm had suggested that the script automatically add the required section last night and you beat me to writing it :). I will update mine with the changes. Thanks!
-
hmm i have only tested the part that add's the miniupnp code to config.xml on my soekris
will look at the replace config file
[edit]
oke the config is on party D not on Atook from the replace config file what was needed
#!/bin/sh # This script adds the miniupnpd files to image specified. # Make sure the miniupnpd.tar.gz file is in the script directory. # # # The miniupnpd.tar.gz file was made by creating a directory named # miniupnpd. In there should be the following files: # miniupnpd, miniupnpd.sh, miniupnpd.inc, miniupnpd.xml, status_upnp.php # Then you execute the command "tar cfz miniupnpd.tar.gz miniupnpd/" # # Usage {miniupnpd} {imagefile} # Example ./sh-add-miniupnpd.sh miniupnpd.tar.gz pfSense.img TARFILE="$1" IMGFILE="$2" WORKDIR=`pwd` UPNPDIR="$WORKDIR/miniupnpd" MD=`mdconfig -a -t vnode -f $WORKDIR/$IMGFILE` mkdir $WORKDIR/a mount /dev/${MD}a $WORKDIR/a tar zxf $TARFILE cp $UPNPDIR/miniupnpd.inc $WORKDIR/a/usr/local/pkg/miniupnpd.inc chmod 755 $WORKDIR/a/usr/local/pkg/miniupnpd.inc cp $UPNPDIR/miniupnpd.xml $WORKDIR/a/usr/local/pkg/miniupnpd.xml chmod 644 $WORKDIR/a/usr/local/pkg/miniupnpd.xml cp $UPNPDIR/miniupnpd $WORKDIR/a/usr/local/sbin/miniupnpd chmod 755 $WORKDIR/a/usr/local/sbin/miniupnpd cp $UPNPDIR/status_upnp.php $WORKDIR/a/usr/local/www/status_upnp.php chmod 755 $WORKDIR/a/usr/local/www/status_upnp.php cd $UPNPDIR MD1=`mdconfig -a -t vnode -f $WORKDIR/$IMGFILE` mkdir $WORKDIR/d mount /dev/${MD1}d $WORKDIR/d echo -n "looking for miniupnp config in config.xml.." if grep -q miniupnp $WORKDIR/d/conf/config.xml then echo "..miniupnp config found." else echo "..miniupnp config not found." cp $WORKDIR/d/conf/config.xml $UPNPDIR/test.xml echo "downloading miniupnp xml config" fetch -q -o cfg_add_miniupnp.xml http://wgnrs.dynalias.com:81/pfsense/cfg-add-m iniupnpd.xml # the downloaded file has <installedpackages>and</installedpackages> #the test.xml has them to so we remove them from the downloaded file echo "removing not needed text" sed -e '/<installedpackages>/ d' -e '/<\/installedpackages>/ d' cfg_add_miniupn p.xml >upnp.xml # now we add upnp.xml to test.xml afther <installedpackages>echo "adding miniupnp config." sed '/<installedpackages>/ r upnp.xml' test.xml > output.xml mv output.xml $WORKDIR/d/conf/config.xml chmod 640 $WORKDIR/d/conf/config.xml fi echo “all done” rm -rf miniupnpd umount $WORKDIR/a rm -rf $WORKDIR/a mdconfig -d -u ${MD} umount $WORKDIR/d rm -rf $WORKDIR/d mdconfig -d -u ${MD1}</installedpackages></installedpackages></installedpackages> -
Since we are downloading the current config section might as well get the current miniupnpd files as well.
#!/bin/sh # This script adds the miniupnpd files to image specified. # # Usage {imagefile} # Example ./sh-add-miniupnpd.sh pfSense.img IMGFILE="$1" WORKDIR=`pwd` MD=`mdconfig -a -t vnode -f $WORKDIR/$IMGFILE` mkdir $WORKDIR/d mount /dev/${MD}d $WORKDIR/d mkdir $WORKDIR/a mount /dev/${MD}a $WORKDIR/a echo "downloading miniupnp files..." fetch -q -o $WORKDIR/a/usr/local/sbin/miniupnpd "http://pfsense.com/packages/config/packages/miniupnpd/sbin/miniupnpd" fetch -q -o $WORKDIR/a/usr/local/pkg/miniupnpd.inc "http://pfsense.com/packages/config/packages/miniupnpd/miniupnpd.inc" fetch -q -o $WORKDIR/a/usr/local/pkg/miniupnpd.xml "http://pfsense.com/packages/config/packages/miniupnpd/miniupnpd.xml" fetch -q -o $WORKDIR/a/usr/local/www/status_upnp.php "http://pfsense.com/packages/config/packages/miniupnpd/status_upnp.php" chmod 755 $WORKDIR/a/usr/local/pkg/miniupnpd.inc chmod 644 $WORKDIR/a/usr/local/pkg/miniupnpd.xml chmod 755 $WORKDIR/a/usr/local/sbin/miniupnpd chmod 755 $WORKDIR/a/usr/local/www/status_upnp.php echo "looking for miniupnp config in config.xml..." if grep -q miniupnp $WORKDIR/d/conf/config.xml then echo " miniupnpd config already exists!" else cp $WORKDIR/d/conf/config.xml $WORKDIR/config.xml echo " downloading miniupnp config..." fetch -q "http://wgnrs.dynalias.com:81/pfsense/cfg-add-miniupnpd.xml" # lets remove <installedpackages>and</installedpackages> sed -e '/<installedpackages>/ d' -e '/<\/installedpackages>/ d' cfg-add-miniupnpd.xml > cfg-miniupnpd.xml echo " adding miniupnp config..." # add miniupnpd section after <installedpackages>sed '/<installedpackages>/ r cfg-miniupnpd.xml' config.xml > $WORKDIR/d/conf/config.xml chmod 640 $WORKDIR/d/conf/config.xml rm -f cfg-add-miniupnpd.xml cfg-miniupnpd.xml config.xml fi echo "miniupnpd successfully added!" umount $WORKDIR/d rm -rf $WORKDIR/d umount $WORKDIR/a rm -rf $WORKDIR/a mdconfig -d -u ${MD}</installedpackages></installedpackages></installedpackages> -
looking nice
is it me or is the file now less lines ? -
looking nice
is it me or is the file now less lines ?Yes, there is less lines now. Downloading the miniupnpd files saved space. I consolidated some of the file copying back and forth. Also removed the second mdconfig as it was unneeded. I verified the script works and will be using it to build my images once I modify my other script. Thanks.