Backup PFSense to a FTP/SFTP server everynight. (How-To Guide)

vito

Well, this is an attempt to help users backup their systems. :)
This is working well on my pfsense box
This might even give an option to do automatic backups of remote systems for this post.. http://forum.pfsense.org/index.php/topic,7949.0.html (just an idea)
A package would be better with a nice GUI/wizard in the firewall interface, but that is beyond me at this time. :)
Let me know if something does not work or the directions need to be cleaned up. I looked over the directions a few times.
If a mode thinks this is a bad idea or bad directions, please delete this post ;)
As always, use at your own risk

Yes, allot of text on the page, but i added allot of notes to try and help. The steps need should not really be this long.

Edit 3/21/08
A Big Note: SFTP absolutely should be used in stead of FTP. SFTP is much more secure transferring data. This is a starter doc and i hope to get the SFTP directions soon. If anyone would like to start testing SFTP, please add to this post to help.
Edit 3/22/08: Added the steps for simple SFTP backups. in addition you should consider encrypting the backups (Directions in this doc). Private/Public keys should be used, but not added yet to this doc. REMEMBER, your passwords in the simp.conf are not HASHED. They are in clear text. As always, make sure you have a strong firewall username and password.

Requirements:
SimpleBackup 1.8.1 http://sourceforge.net/projects/migas-sbackup/
Perl and possibly the mods included in the simple backup tar
NOTE: I have Perl already installed from other packages. So far, I have not needed to install any of the mods from the simple backup download. Everything so far is working for me. Installing Perl or any mods by itself will not be covered in the How To.
Packages I have installed: Squid, Squid Guard, LightSquid
Also note: These directions are done from a windows computer. (Sorry) I am sure most of this can be done right from CLI, but i am not an expert at FreeBSD.
SFTP can be used, but is not part of this current document.

Setup:
Note: please read the full documentation of SimpleBackup in the TAR. This How To covers a very simple setup.

1) SimpleBackup Setup on Server
a. Download and un tar the simplebackup program to your local computer.
b. Using a program like WinSCP open a session to your firewall
c. Create a directory called simplebackup in usr/local/etc/
d. Browse to your local files of simplebackup and copy the contents of the UNIX folder /usr/local/etc/simplebackup
e. Open an SSH session to your firewall and execute this command /usr/bin/perl /usr/local/etc/simplebackup/simplebackup.pl –econf simp.conf This will create a config file called simp.conf
f. If you are going to use SFTP do the following otherwise skip to step H: While in your session, issue this command NOTE AGAIN I already have perl installed for other paclages. Down the road i will add steps to install perl from scratch (hopefully) pkg_add -r http://62.4.17.14/pub/FreeBSD/ports/i386/packages-6.2-release/All/p5-Expect-1.20.tbz {THANKS PERRY! :) } This will install the need perl package to allow SFTP from simplebackup.
h. Next, refresh your WinSCP session and right click on simp.conf. Select Edit.

2) SimpleBackup Configuration file
SimpleBackup has may features, you can do full backups, incremental, and differential. You can also backup to FTP, SFTP. We are going to configure just a simple config.xml backup for now to an FTP server.
a. Line 47: Enter the type of backup you would like to use. EX: differential
b. Line 64: Enter what day the full backup will happen on. EX: monday
c. Line 83: enter the backup format. EX: tar.qz
d. Line 119: enter the locations you would like to backup. We are only going to backup the config directory for now. EX: /cf/conf (Note: if you would like to add other directories, you must use this format /usr/local/etc < /cf/conf always add a < before the next directory listing
e. Line 200: enter the place you would like the backup to go. EX ftp://ftpuser:password@192.168.1.10:21/backup I have noticed that you do need to specify a directory. So in this user’s home directory, make a subfolder for the backup. In this example, it is called backup

If you will not be using SFTP go to step F, Otherwise read below
NOTE for SFTP: All you need to do is change this same line from ftp://ftpuser:password@192.168.1.10:21/backup to sftp://ftpuser:password@192.168.1.10:22/backup . My SFTP box is on port 22, notice how i changed the port from 21 to 22. A very important step if you will use SFTP is to make sure your system has the sftp server key in it's local "known_hosts" store/file, otherwise Simplebackup will fail. The simplest thing to get this done is go to log into your box with ssh and issue this command sftp yoursftpbackusername@yoursftpserverHOSTNAMEorIP EX: sftp pfsense@192.168.1.1 You will get some type of message about the RSA key. type in yes and hit enter, You will now get a message that the key will be added to the list of know_host. At this point enter the SFTP Backup Users password on your SFTP/SSH box. You should now have what is needed to do a SFTP backup. This is a quick way of setting up SFTP. Using Private/ Public keys would be much better, but i do not have those steps yet. Some of the SFTP/SSH programs i have tested on windows would be http://www.freesshd.com/, http://www.coreftp.com/server/ and http://www.freeftpd.com/

f. Line 404: Enter a location for the log file: EX /usr/local/etc/simplebackup/backup.log
h. If you want to Encrypt the file before it sends, add a password to Line 384: EX: pfsense
Make sure you read Lines 358-382 so you under stand how the file is Encrypted.
Save the file

3) Testing your Backup
a. In your SSH session, execute the following commands /usr/bin/perl /usr/local/etc/simplebackup/simplebackup.pl –conf simp.conf
b. Now, you should see a log in on your FTP server. Simplebackup will log in first and scan the backup directory for backup sets and disconnect. When the backup is ready to be transferred, SimpleBackup will log in again to the FTP server.

4) Setting up the backup to run daily.
a. Log in to your PFSense web GUI and go to Diagnostics: Edit File
b. Load /conf/config.xml
c. Find your cronjob section <cron>You will notice other cron jobs located here. Go to the last entry in the cron section (before</cron> in between and put this entry in. Note: you can copy and paste another entry and just modify the section you need for the simple backup sections.
Example: This will run every night at 11:30pm
<task_name>SimpleBackup</task_name>
<minute>/30</minute>
<hour>23</hour>
<mday></mday>
<month></month>
<wday></wday>
<who>root</who>
<command></command>/usr/bin/perl/usr/local/etc/simplebackup/simplebackup.pl –conf /usr/local/etc/simplebackup/simp.conf

You might have a reboot to get the cron line added to crontab. (if there is a better way, please let me know. At the time i was not sure how to do this with out a reboot)

5) Restoring/Opening your Files
a. On your FTP server, find your files in your FTP folders. (Note: i am backing up to Filezilla Server on a windows box)
b. IF YOU DID NOT ENCRYPT YOUR FILE GOTO STEP CTo Decrypt file on windows to restore, make sure you have the WINDOWS simplecrypt.exe located in your original simplebackup zip.
syntax: simplecrypt.exe –decrypt "yourbackupfile.tar.gz.sc" NOTE: the file you want to restore will end in sc You also need the QUOTES for the file name!! :)
You will then see the following:
Where to decrypt (path into the output file)
[ yourbackupfile.tar.gz ]
path to new file > c:\dc NOTE: This is really a file name!!! This will also be the file you will extract with your ZIP program after you are done decrypting the file.
Accept the defaults if you like for the next few steps by hitting enter.
Number of crypting times (the higher the safer and slower)
[ 2 ]
number >

Encrypting level (bit level is safer but very slow)
[ byte ]
byte/bit >

Erase the file to decrypt after ending (with sucess) ?
[ no ]
yes/no >

Please type in the encryption password
password > password NOTE: This is your password that you put on line 384 in step 2,h

Details :
Decrypting file [ yourbackupfile.gz.sc]
Decrypting file will not be erased
Into file [ c:\dc ]
Crypting times [ 2 ]
Crypting level [ byte ]

Notice :
The . char represents a read + decrypt operation and the * char
represents a write operation.

Decrypting
.*

File [ c:\dc ] was created. NOTE AGAIN: This is the file you will extract or open with your ZIP program

Operation ended in success.

c. Using your favorite archive program, open/unzip the file and find what you need. ( I am using 7-zip)

hoba

Nice, but be aware that ftp transfer is not very secure for sensible data like your firewallconfig. You should prefer sftp. Maybe you can update your guide how to do it with sftp :)

vito

Hi hoba
100% absolutely agree, SFTP should be used.
When I started to test with this, I wanted to start basic to make sure the concept worked.
At first, I was not going to post the steps because it was not SFTP, but I figured somebody might want to jump in and try SFTP to help.

I think I will change the post to state that this is a starter doc and beware SFTP should be used. (something like that)

SFTP is next on my list and I will certainly up date the directions for this.

~~Simplebackup also has an option to encrypt the data before sending, but was not sure if that would work yet on pfsense.~~
Edit: 3/21/2008: Seems to work fine and i will update the doc

BTW, did you see anything that I should not be doing in these steps or something that might make it easier?

Thanks again for your input!!

Edit: added a note to the post about SFTP :)

hoba

You should make it a package ;D

vito

lol… i know.. :)

vito

Ok,
Trying to get SFTP going.
I need to get these perl mods installed…
Expect-1.20 (or 1.21 since i can not find 1.20 on the server)
and
IO-Tty-1.07

Both packages would like to have perl 5.8.8.1 but i would think 5.8.8 would work also... (a guess)

I saw them as packages on the FreeBSD Ftp server, so i tried to install them

pkg_add ftp://ftp4.freebsd.org/pub/FreeBSD/ports/packages/perl5/p5-Expect-1.21.tbz

It looks like IO-Tty-1.07 is installed with Expect because it is needed.

here is what i get (this is after the second time i tried, so i think they are installed now, but Simplebackup still errors trying to find Except)

pkg_add ftp://ftp4.freebsd.org/pub/FreeBSD/ports/packages/perl5/p5-Expect-1.21.tbz
Fetching ftp://ftp4.freebsd.org/pub/FreeBSD/ports/packages/perl5/p5-Expect-1.21.tbz... Done.
Fetching ftp://ftp4.freebsd.org/pub/FreeBSD/ports/packages/All/p5-IO-Tty-1.07.tbz... Done.
pkg_add: warning: package 'p5-IO-Tty-1.07' requires 'perl-5.8.8_1', but 'perl-5.8.8' is installed
pkg_add: warning: package 'p5-Expect-1.21' requires 'perl-5.8.8_1', but 'perl-5.8.8' is installed
pkg_add: package 'p5-Expect-1.21' or its older version already installed

i do see them in the list when i do pkg_info

I am doing something wrong...i just do not know yet..been google'n and searching....
Thanks!!

Perry

pkg_add -r http://62.4.17.14/pub/FreeBSD/ports/i386/packages-6.2-release/All/p5-Expect-1.20.tbz

You can ask pkg_add to go to a different location by modifying the PACKAGESITE environmental variable by typing this as one line:
setenv PACKAGESITE ftp://62.4.17.14/pub/FreeBSD/ports/i386/packages-6.2-release/Latest/

sullrich

Seems like a ton of dependencies for something so simple that could be scripted with one line of shell code + fetch!?

Not trying to put down your efforts at all, just wondering.

vito

Hi sullrich
I don’t think your question downs my efforts at all. It a good question and I hope I can explain why. ;)

Yes, I am sure this can be done with a script, but the simplebackup already has everything I thought someone could use. We use this on our web servers (and some windows servers) and have found it to be a nice program that fit our need. So that is why i wanted to try it on PFsense.

With simplebackup, you have the option to use FTP, SFTP, WEBDEV, and Email backups. Plus the option to encrypt the backups and email the reports to you. It also able to use Full, Incremental, and differential backup sets. (Yes, just backing up the config.xml you would not really need the diff and incremental) But, if you need to backup more then just the config, this certainly could help someone. I am by far not the right person to say this can not be done another way (since my programming skills are very lacking), but I did not want to reinvent the wheel when I thought I found what I needed.

The idea I had for the whole thing was to have something that was easy to configure with one file. I am guessing here, but I would think if I made a package down the road, I could just manipulate the config file so users can pick the options in the GUI.
All the options Simple backup has to offer could give many options to users. (thought I saw posts about remote/auto backups and a few people needing to backup other things then just the config.xml. I could be wrong)

Just curious, when you say dependencies, do you mean Perl mods or just the amount of steps to get this to work? Besides Perl which a few packages look like they use and install anyway, I only needed to add the p5-expect/ IO-tty package. (So far that I have tested)

It may look like a lot of steps, but it really is not. (first try maybe)

We are going to start using PFSense more in the field (moving from our current firewalls) and wanted to make sure we get good backups of systems.
Even if it is to a local FTP/SFTP server and the PFSense backup is included in an office current backup rotation. (encrypted of course) ;)

Again, thanks for your question. This might not be the perfect setup but right now it is the only one I thought might work. ;)

Cry Havok

Another option (one I use) is to install the rsync package (or just the binary) and then use rsync over SSH (in the form of rsnapshot). It does make it a pull rather than a push. I may document the steps at some point in case it helps others.

Juve

I'm currently using a simple bash script calling wget with some POST parameters (auth + backup page parameters).
working well since november 2005. ;)

buraglio

@Juve:

I'm currently using a simple bash script calling wget with some POST parameters (auth + backup page parameters).
working well since november 2005. ;)

I finally have time to get this integrated into my RANCID backup environment and was looking to do JUST that. Rather than recreating the wheel, could you post your code to do what you're doing? It'll save me time and likely a headache.

nb