Using wget for scheduled backup on 2.0.1



  • Hey guys,

    I am trying to set up scheduled backups from my pfSense installs to a local server on the same LAN.  I have been reading through options and figure the simplest for me is running wget from my backup server.  I have tried a few different command lines and am having some issues.  I have followed the instructions and used the command line from here http://doc.pfsense.org/index.php/Remote_Config_Backup, using the info for pulling the config from 2.0 but it fails.  I set up logging and get info like http://doc.pfsense.org/index.php/Remote_Config_Backup in the console window when running the first line.  The first command I run is like this:

    wget -o log.txt –keep-session-cookies --save-cookies cookies.txt --post-data 'login=Login&usernamefld=admin&passwordfld=pfsense' –no-check-certificate https://192.168.2.250/diag_backup.php

    I am by far an expert on wget or the BSD/Unix filesystems/commands so any help is appreciated.

    Thanks.

    Aaron



  • Okay, after some forehead smacking, and sorting out some syntax for use in a Windows environment, I have this working…mostly.  My commands now look like this:

    Login and save cookies command:  wget –keep-session-cookies --save-cookies cookies.txt --no-check-certificate --post-data="usernamefld=username&passwordfld=password&login=Login" https://192.168.2.250/diag_backup.php  note the double quotes around the post-data info, stops Windows from messing with the ampersand

    Download config:  wget –keep-session-cookies --load-cookies cookies.txt  --no-check-certificate --post-data "Submit=download&donotbackuprrd=yes" https://192.168.2.250/diag_backup.php -O config-router1-date +%Y%m%d%H%M%S.xml  same double quotes around the post-data info

    It gets past the login info and downloads the config file fine, however, when saving the file it gets confused by the date portion of the command.  It saves the file as config-router1-'date, then tries to resolve the date code as a hostname and I get this output:  Resolving +%y%m%d%h%m%s`.xml… failed: No data record of requested type.  I've tried a bunch of different bracketing around the date code but can't seem to get it to work properly.  I'm guessing it's a Windows thing again, but I'm not having luck finding the right syntax.



  • You've probably already done it, but i'm going to say this anyway… :) Make sure to make a specifik backup user in the firewall, and make sure that this user only has access to the diag_backup.php

    Now, regarding wget for windows... My backup script looks like this:
    wget --post-data="Submit=download&donotbackuprrd=1"  https://IP/diag_backup.php --no-check-certificate -O config-router-"fw10.something.local_%date%".xml --keep-session-cookies --load-cookies cookies.txt

    this gives me a backup with the filename "config-router-fw10.something.local_31-08-2012.xml"



  • @acherman:

    Okay, after some forehead smacking, and sorting out some syntax for use in a Windows environment, I have this working…mostly.  My commands now look like this:

    Login and save cookies command:  wget –keep-session-cookies --save-cookies cookies.txt --no-check-certificate --post-data="usernamefld=username&passwordfld=password&login=Login" https://192.168.2.250/diag_backup.php   note the double quotes around the post-data info, stops Windows from messing with the ampersand

    Download config:  wget –keep-session-cookies --load-cookies cookies.txt  --no-check-certificate --post-data "Submit=download&donotbackuprrd=yes" https://192.168.2.250/diag_backup.php -O config-router1-date +%Y%m%d%H%M%S.xml   same double quotes around the post-data info

    It gets past the login info and downloads the config file fine, however, when saving the file it gets confused by the date portion of the command.  It saves the file as config-router1-'date, then tries to resolve the date code as a hostname and I get this output:  Resolving +%y%m%d%h%m%s`.xml… failed: No data record of requested type.  I've tried a bunch of different bracketing around the date code but can't seem to get it to work properly.  I'm guessing it's a Windows thing again, but I'm not having luck finding the right syntax.

    If you are doing this as part of a script, you can use

    thedate=date +%Y%m%d-%H%M%S

    then in the call

    wget –keep-session-cookies --load-cookies cookies.txt  --no-check-certificate --post-data "Submit=download&donotbackuprrd=yes" https://192.168.2.250/diag_backup.php -O config-router1-$thedate.xml

    wget is not windows, what are you using to run the script/commands? The date command in windows does completely different things.



  • wget for windows: http://gnuwin32.sourceforge.net/packages/wget.htm

    the date command in windows gives you the, ehrm, date. :)



  • no it does not, run it in a command prompt window, it wants you to set the, ehrm, date.
    You can do a "date /T" to tell you the date, but there is no formatting that I can tell. same with "time /T".



  • look at my post above, now back to this…




  • hmmm … see I miss read that ... my bad. I am on windows 7 and I am not getting the same output. I get

    echo %date%-%time%
    Fri 08/31/2012-17:24:07.37

    The "/" in there would throw off a script unless you removed them somehow.



  • found this to help with the variables in the time format.

    @echo off
    :: similar to above methods but this works
    :: get current date/time into vars
    :: vars= day month year hour mins secs mili
    for /f "tokens=1* delims= " %%a in ('date/t') do set dayname=%%a
    for /f "tokens=1* delims= " %%a in ('date/t') do set mmddyyyy=%%a
    for /f "tokens=1* delims=/" %%a in ('echo %mmddyyyy%') do set day=%%a
    for /f "tokens=2* delims=/" %%a in ('echo %mmddyyyy%') do set month=%%a
    for /f "tokens=3* delims=/" %%a in ('echo %mmddyyyy%') do set year=%%a
    for /f "tokens=1* delims=:" %%a in ('echo %time%') do set hour=%%a
    for /f "tokens=2* delims=:" %%a in ('echo %time%') do set mins=%%a
    for /f "tokens=3* delims=:" %%a in ('echo %time%') do set sec=%%a
    for /f "tokens=1* delims=." %%a in ('echo %sec%') do set secs=%%a
    for /f "tokens=2* delims=." %%a in ('echo %sec%') do set mili=%%a
    
    echo day =%day%
    echo month=%month%
    echo year =%year%
    echo hour =%hour%
    echo mins =%mins%
    echo secs =%secs%
    echo mili =%mili%
    

    ^- not tested



  • On my windows I reset the script to:

    @echo off
    :: similar to above methods but this works
    :: get current date/time into vars
    :: vars= day month year hour mins secs mili
    for /f "tokens=1* delims= " %%a in ('date/t') do set dayname=%%a
    for /f "tokens=2* delims= " %%a in ('date/t') do set mmddyyyy=%%a
    for /f "tokens=2* delims=/" %%a in ('echo %mmddyyyy%') do set day=%%a
    for /f "tokens=1* delims=/" %%a in ('echo %mmddyyyy%') do set month=%%a
    for /f "tokens=3* delims=/" %%a in ('echo %mmddyyyy%') do set year=%%a
    for /f "tokens=1* delims=:" %%a in ('echo %time%') do set hour=%%a
    for /f "tokens=2* delims=:" %%a in ('echo %time%') do set mins=%%a
    for /f "tokens=3* delims=:" %%a in ('echo %time%') do set sec=%%a
    for /f "tokens=1* delims=." %%a in ('echo %sec%') do set secs=%%a
    for /f "tokens=2* delims=." %%a in ('echo %sec%') do set mili=%%a
    
    echo day =%day%
    echo month=%month%
    echo year =%year%
    echo hour =%hour%
    echo mins =%mins%
    echo secs =%secs%
    echo mili =%mili%
    

Locked