Found a small error "Remote_Config_Backup" in Wiki.
-
Hello Jim,
I think I found a small error in the "Remote_Config_Backup" in Wiki. Or at least differences depending on what platform it is executed on.
https://doc.pfsense.org/index.php/Remote_Config_BackupI have found that the script for taking backup and more specific the "donotbackuprrd=no" stopped working between 2017-02-01 -> 2017-03-01 timeframe
(which is the monthly backup schedule) where it stopped backup the XML with full RRD.I started to check the script and made a change from "donotbackuprrd=no" to "donotbackuprrd=0" and then it started working again.
Please update the WIKI page with this. Possible a note that on some platforms "=0" is needed instead of "=no".All my backups taken with donotbackuprrd=no does NOT contain RRD data but when I changed to "donotbackuprrd=0" it started working directly.
Script running on Synology DS713+ with DSM 6.1-15047 Update 1root@DiskStation3TB:/volume1/web_backend/tools# /usr/bin/wget --version GNU Wget 1.15 built on linux-gnu. +digest +https +ipv6 -iri +large-file -nls +ntlm +opie +ssl/opensslMy working script now contains:
#!/bin/ash BACKUPDIR="/volume1/BACKUPNEW/pfsensebak/backup/daily" USERNAME="<removed>" PASSWORD="<removed>" PORT="80" SITES="x.x.x.x" ZIP="/usr/bin/zip" FIND="/usr/bin/find" RMFILE="/bin/rm" WGET="/usr/bin/wget" BACKUPDAYS="30" RMFILE="/bin/rm" cd /volume1/web_backend/tools for site in $SITES do $WGET -qO- --keep-session-cookies --save-cookies /tmp/cookies.txt --no-check-certificate --timeout=10 http://$site:$PORT/diag_backup.php | grep "name='__csrf_magic'" | sed 's/.*value="\(.*\)".*/\1/' > /tmp/csrf.txt $WGET -qO- --keep-session-cookies --load-cookies /tmp/cookies.txt --save-cookies /tmp/cookies.txt --no-check-certificate --post-data "login=Login&usernamefld=$USERNAME&passwordfld=$PASSWORD&__csrf_magic=$(cat /tmp/csrf.txt)" --timeout=10 http://$site:$PORT/diag_backup.php | grep "name='__csrf_magic'" | sed 's/.*value="\(.*\)".*/\1/' > /tmp/csrf2.txt if [ -e /tmp/csrf2.txt ]; then DATETMP="`date +%Y%m%d-%H%M%S`" FILENAME1="$BACKUPDIR/config-$site-$DATETMP.xml" FILENAME2="$BACKUPDIR/config-$site-$DATETMP-withrrd.xml" $WGET --keep-session-cookies --load-cookies /tmp/cookies.txt --no-check-certificate --post-data "download=download&donotbackuprrd=1&__csrf_magic=$(head -n 1 /tmp/csrf2.txt)" --timeout=10 http://$site:$PORT/diag_backup.php -O $FILENAME1 > /dev/null 2>&1 $WGET --keep-session-cookies --load-cookies /tmp/cookies.txt --no-check-certificate --post-data "download=download&donotbackuprrd=0&__csrf_magic=$(head -n 1 /tmp/csrf2.txt)" --timeout=120 http://$site:$PORT/diag_backup.php -O $FILENAME2 > /dev/null 2>&1 rm -f /tmp/cookies.txt rm -f /tmp/csrf.txt rm -f /tmp/csrf2.txt $ZIP -q -9 -j $FILENAME1.zip $FILENAME1 $ZIP -q -9 -j $FILENAME2.zip $FILENAME2 $RMFILE $FILENAME1 $RMFILE $FILENAME2 else echo "Failed to retrieve backup from $site" fi done $FIND $BACKUPDIR -type f -name "*.xml.gz" -mtime +$BACKUPDAYS -exec rm {} \; $FIND $BACKUPDIR -type f -name "*.xml.zip" -mtime +$BACKUPDAYS -exec rm {} \;</removed></removed>I also have a modified version as well (it requires rar executable as well):
#!/bin/ash BACKUPDIR="/volume1/BACKUPNEW/pfsensebak/hansbuhlin/daily" USERNAME="<removed>" PASSWORD="<removed>" PORT="443" SITES="x.x.x.x" RAR="/volume1/web_backend/tools/rar" FIND="/usr/bin/find" RMFILE="/bin/rm" WGET="/usr/bin/wget" BACKUPDAYS="30" RMFILE="/bin/rm" cd /volume1/web_backend/tools /volume1/web_backend/tools/rar for site in $SITES do $WGET -qO- --keep-session-cookies --save-cookies /tmp/cookies.txt --no-check-certificate --timeout=10 https://$site:$PORT/diag_backup.php | grep "name='__csrf_magic'" | sed 's/.*value="\(.*\)".*/\1/' > /tmp/csrf.txt $WGET -qO- --keep-session-cookies --load-cookies /tmp/cookies.txt --save-cookies /tmp/cookies.txt --no-check-certificate --post-data "login=Login&usernamefld=$USERNAME&passwordfld=$PASSWORD&__csrf_magic=$(cat /tmp/csrf.txt)" --timeout=10 https://$site:$PORT/diag_backup.php | grep "name='__csrf_magic'" | sed 's/.*value="\(.*\)".*/\1/' > /tmp/csrf2.txt if [ -e /tmp/csrf2.txt ]; then DATETMP="`date +%Y%m%d-%H%M%S`" FILENAME1="$BACKUPDIR/config-$site-$DATETMP-rarpasswordencrypted.xml" FILENAME2="$BACKUPDIR/config-$site-$DATETMP-withrrd-rarpasswordencrypted.xml" $WGET --keep-session-cookies --load-cookies /tmp/cookies.txt --no-check-certificate --post-data "download=download&donotbackuprrd=1&__csrf_magic=$(head -n 1 /tmp/csrf2.txt)" --timeout=10 https://$site:$PORT/diag_backup.php -O $FILENAME1 > /dev/null 2>&1 $WGET --keep-session-cookies --load-cookies /tmp/cookies.txt --no-check-certificate --post-data "download=download&donotbackuprrd=0&__csrf_magic=$(head -n 1 /tmp/csrf2.txt)" --timeout=120 https://$site:$PORT/diag_backup.php -O $FILENAME2 > /dev/null 2>&1 rm -f /tmp/cookies.txt rm -f /tmp/csrf.txt rm -f /tmp/csrf2.txt $RAR a -ep -m5 -hp <replace with="" own="" password="">$FILENAME1.rar $FILENAME1 $RAR a -ep -m5 -hp <replace with="" own="" password="">$FILENAME2.rar $FILENAME2 $RMFILE $FILENAME1 $RMFILE $FILENAME2 else echo "Failed to retrieve backup from $site" fi done $FIND $BACKUPDIR -type f -name "*.xml.rar" -mtime +$BACKUPDAYS -exec rm {} \;</replace></replace></removed></removed>//Dan Lundqvist
-
On what version? 2.3.x? 2.4 snapshots?
Nothing changed on that page during the relevant time period that would alter the behavior of the script that I see.
It's always checked if donotbackuprrd is set vs unset, not the value. Using 0 would cause it to evaluate as if it were unset. Just omit the parameter entirely, rather than giving it any value.
if ($_POST['backuparea'] !== "rrddata" && !$_POST['donotbackuprrd']) {That line has not changed since May of 2012.
$_POST['donotbackuprrd'] = "no"; var_dump(!$_POST['donotbackuprrd']);Outputs "bool(false)"
$_POST['donotbackuprrd'] = 0; var_dump(!$_POST['donotbackuprrd']);Outputs "bool(true)"
The page is fine, it does not offer any guidance for backing up RRD. That could be added, but nothing changed in the behavior stated on the page.
-
Hi Jim,
It is really weird. The problem was seen on 2.3.3_1 at the latest and as soon as I changed the script from "no" to "0" it started working and downloading RRD as it should.
"yes" and "no" defined it always exclude RRD.I just tried again on 2.3.4 but same issue… with yes/no it downloads both XML config without RRD and if i change script back again to 1/0 it starts working, then one contains the RRD data.
I just made a tcpdump of working and non-working session if you want? I can send it to you offlist as it contains real config data.//Dan Lundqvist
-
The code has not changed in years. It behaves as I described 100% of the time. yes or no does not matter, the value itself is never checked, only the variable is tested, and the way it is tested.
If the variable is defined to a non-zero value, it will not backup RRD. If it is undefined or set to 0, it will backup RRD.
-
I can not explain it other than it did stop working at a certain date. If that was due to upgrade of the NAS or something else I am not sure
but something happened.BUT, if this is a local issue only at my installation, then we leave it at that. But bare in mind if you hear anyone else with same issue.
//Danne
-
Thanks so much for bringing this up and posting it. Setting this up today for the first time (on 2.3.4-RELEASE-p1), I had the same hard time trying to figure out why deleting the variable or setting the value to yes/no wouldn't work. After some Googling I found this post, set it to 0 and now I'm getting my Config+RRD data in full.
-
Thanks so much for bringing this up and posting it. Setting this up today for the first time (on 2.3.4-RELEASE-p1), I had the same hard time trying to figure out why deleting the variable or setting the value to yes/no wouldn't work. After some Googling I found this post, set it to 0 and now I'm getting my Config+RRD data in full.
This means that I am not alone with this issue and also that the solution I found works for more than one.
Jim, did you see this?Best regards
Dan Lundqvist
Stockholm, Sweden
