sh script to create bootable USB-drive with LATEST OFFICIAL REL of pfSense CE

elvisimprsntr

I just download the memstick img, use BelenaEtcher, boot and install on SSD

Works automagically!

Really shouldn't be running pfSense (or any OS) off thumb drives.

Thumb drive flash memory does not have the same error correction, wear leveling, and PE cycles as a SSD.

Sergei_Shablovsky

@elvisimprsntr said in sh script to create bootable USB-drive with LATEST OFFICIAL REL of pfSense CE:

I just download the memstick img, use BelenaEtcher, boot and install on SSD

Works automagically!

I just need sh script to avoid this “download the memstik img, using Balena Etcher” procedure, script must AUTOMATICALLY making bootable “rescue” USB-drive INSIDE my server on crone job or manually.
(Please read previous updated my answer to @stephenw10)

Really shouldn't be running pfSense (or any OS) off thumb drives.

Thumb drive flash memory does not have the same error correction, wear leveling, and PE cycles as a SSD.

Agree. Of course.

elvisimprsntr

@Sergei_Shablovsky

I just run a daily cron job to backup the config.xml file to my TrueNAS server. I know pfsense now has some sort of backup mechanism, but I would rather have it local since the file contains credentials.


#!/bin/sh
VERSION=`cat /etc/version`
DATE=`date +%Y%m%d`
FILE="config_`hostname -s`_"$DATE"_"$VERSION".xml"
NAS="nas-1"
PATH="/mnt/data/Software/pfsense"

# mkdir /media/usb

/sbin/mount_msdosfs /dev/da0s1 /media/usb
if [ "$?" -eq "0" ]; then
	echo "USB found"
	/bin/cp /cf/conf/config.xml /media/usb/$FILE
	echo "Backup $FILE created"
	/usr/bin/find /media/usb/ -name "config_*.xml" -mtime +365 -exec rm {} \;
	/sbin/umount /media/usb	
else
	echo "USB not found"
fi	

/sbin/ping -c 3 $NAS > /dev/null 2>&1
	if [ $? -eq 0 ]; then
		echo "$NAS found" 
		/usr/bin/scp /cf/conf/config.xml root@$NAS:$PATH/$FILE
		/usr/bin/scp /root/pkg_check.php root@$NAS:$PATH/
		/usr/bin/scp /root/att_cidr.sh root@$NAS:$PATH/
		/usr/bin/scp /root/backup.sh root@$NAS:$PATH/
		echo "Backup $FILE copied to $NAS"
	else
		echo "$NAS not found"
	fi
	
# install cron package and add cron job
# 0 4 * * Sun /bin/sh /root/backup.sh > /dev/null

stephenw10

I guess you would need to pull the file list from: https://atxfiles.netgate.com/mirror/downloads/
Then parse the latest version and compare it with something cached.

Sergei_Shablovsky

@stephenw10 said in sh script to create bootable USB-drive with LATEST OFFICIAL REL of pfSense CE:

I guess you would need to pull the file list from: https://atxfiles.netgate.com/mirror/downloads/
Then parse the latest version and compare it with something cached.

Exactly!
Thank You so much!

Is this https://atxfiles.netgate.com/mirror/downloads/ path fixed constant ?

stephenw10

It has been that for a long time. I can't promise 100% it will remain that but I see no reason to change it.

The script could throw an error if it can't see any files.

Sergei_Shablovsky

@stephenw10 said in sh script to create bootable USB-drive with LATEST OFFICIAL REL of pfSense CE:

It has been that for a long time. I can't promise 100% it will remain that but I see no reason to change it.

Nice news, thank You!

[UPDATE]
The same url (+extra for statistics collection) used on official pfSense CE download page.

The script could throw an error if it can't see any files.

I think that special package would be useful for this: package intended for all users who have ability to install 8Gb USB-memstick inside their pfSense server/device/appliance or in free USB port.

Like on this image:

In settings of this package would be:

this URL to Netgate download directory;
filename scheme;
version;
frequency (in a days/weeks/months) of creating this “rescue boot drive”;
check box to also initiate “Auto Config Backup” on a Netgate remote servers;
checkbox to also initiate “Backup / Restore” to /backup-restore on this “rescue drive”;
checkbox to also initiate “Backup Files/Dir” (if installed) to /backup-filedir on this “rescue drive”;

Reasonable?

Sergei_Shablovsky

@elvisimprsntr said in sh script to create bootable USB-drive with LATEST OFFICIAL REL of pfSense CE:

@Sergei_Shablovsky

I just run a daily cron job to backup the config.xml file to my TrueNAS server. I know pfsense now has some sort of backup mechanism, but I would rather have it local since the file contains credentials.


#!/bin/sh
VERSION=`cat /etc/version`
DATE=`date +%Y%m%d`
FILE="config_`hostname -s`_"$DATE"_"$VERSION".xml"
NAS="nas-1"
PATH="/mnt/data/Software/pfsense"

# mkdir /media/usb

/sbin/mount_msdosfs /dev/da0s1 /media/usb
if [ "$?" -eq "0" ]; then
	echo "USB found"
	/bin/cp /cf/conf/config.xml /media/usb/$FILE
	echo "Backup $FILE created"
	/usr/bin/find /media/usb/ -name "config_*.xml" -mtime +365 -exec rm {} \;
	/sbin/umount /media/usb	
else
	echo "USB not found"
fi	

/sbin/ping -c 3 $NAS > /dev/null 2>&1
	if [ $? -eq 0 ]; then
		echo "$NAS found" 
		/usr/bin/scp /cf/conf/config.xml root@$NAS:$PATH/$FILE
		/usr/bin/scp /root/pkg_check.php root@$NAS:$PATH/
		/usr/bin/scp /root/att_cidr.sh root@$NAS:$PATH/
		/usr/bin/scp /root/backup.sh root@$NAS:$PATH/
		echo "Backup $FILE copied to $NAS"
	else
		echo "$NAS not found"
	fi
	
# install cron package and add cron job
# 0 4 * * Sun /bin/sh /root/backup.sh > /dev/null

Thank You so much for sharing script!

I use it as part of mine. ;)

Why you use pinging the remote host instead of checking if certain remote path exist (or checking the success of this remote path creating)?

elvisimprsntr

@Sergei_Shablovsky said in sh script to create bootable USB-drive with LATEST OFFICIAL REL of pfSense CE:

Why you use pinging the remote host instead of checking if certain remote path exist (or checking the success of this remote path creating)?

I have two pfSense sites on a Tailscale MESH VPN, one behind double NAT.
I use the same script to backup the remote site to my local NAS.
Sometimes it take a few pings before the NAS is reachable via Tailscale.

Sergei_Shablovsky

@elvisimprsntr said in sh script to create bootable USB-drive with LATEST OFFICIAL REL of pfSense CE:

@Sergei_Shablovsky said in sh script to create bootable USB-drive with LATEST OFFICIAL REL of pfSense CE:

Why you use pinging the remote host instead of checking if certain remote path exist (or checking the success of this remote path creating)?

I have two pfSense sites on a Tailscale MESH VPN, one behind double NAT.
I use the same script to backup the remote site to my local NAS.

I have decision that procedure of correct and flawless backup/restore pfSense configuration still are so called “headpain point” for most of all pfSense users

And even provided “rollback to last good configuration by using ZFS snapshots feature” - not so help with this: this ZFS-rollback” really good in the middle of working system, but not good if you need quick restore after hardware failure when needed to setup fresh on bare metal another server.

Several times I see how after disc crash in Netgate Appliance and replace disc on same Appliance, procedure of “complete restore from last good ACB configuration from remote Netgate servers” not flawless: sometime some packages not installed for unknown reasons, and hw rebooting between some packages still needed…

Sad bud true…

Sometimes it take a few pings before the NAS is reachable via Tailscale.

Why You not using FreeBSD famous net/rclone, backup/zapzend, backup/zfs_autobackup, backup/sanoid and syncoid?

Each of this solution give You more flexibility, because You not only need to backup one pfSense config.xml, but may be a bunch of other scripts and edited BSD system files with custom settings.