Help with pfsense backup script

wgstarks · Oct 4, 2018, 11:32 PM

I found a simple script to run backups from a remote computer which works but is generating some errors.

#!/bin/bash

BACKUP_HOST=10.0.1.1
BACKUP_USER=<username>
BACKUP_PASSWORD=<password>

# Create config file directory if it doesn't exist
[ -d files/ ] || mkdir files

# Get CSRF token
CSRF=$(wget -qO- --keep-session-cookies --save-cookies cookies.txt --no-check-certificate https://${BACKUP_HOST}/ | grep "name='__csrf_magic'" | sed 's/.*value="\(.*\)".*/\1/')

# Log into pfSense
CSRF=$(wget -qO- --keep-session-cookies --load-cookies cookies.txt --save-cookies cookies.txt --no-check-certificate --post-data "login=Login&usernamefld=${BACKUP_USER}&passwordfld=${BACKUP_PASSWORD}&__csrf_magic=${CSRF}" https://${BACKUP_HOST}/ | grep "name='__csrf_magic'" | sed 's/.*value="\(.*\)".*/\1/')

# Save configuration file
wget --keep-session-cookies --load-cookies cookies.txt --no-check-certificate --post-data "download=Download%20configuration%20as%20XML&donotbackuprrd=yes&__csrf_magic=${CSRF}" https://${BACKUP_HOST}/diag_backup.php -O ./files/config_${BACKUP_HOST}_$(date +%Y-%m-%d-%H-%M-%S).xml 2>/dev/null

# Clean up
rm cookies.txt
unset BACKUP_HOST BACKUP_USER BACKUP_PASSWORD

# Remove files older than 100 days
find /mnt/user/odin_backup/OdinBackUp/files/ -type f -name '*.xml' -mtime +100 -exec rm {} \;

Every time it runs I see this error in the pfsense system log-

Edit: Just realized there are two errors-

php-fpm		/index.php: backup@10.0.1.20 (Local Database) attempted to access /index.php but does not have access to that page. Redirecting to diag_backup.php.

And then-

php-fpm		PHPSESSION 1 open sessions left at shutdown script!Array ( [0] => #### phpsession_begin #### simplestacktrace(..) - /etc/inc/phpsessionmanager.inc:38 phpsession_begin(..) - /etc/inc/auth.inc:1955 session_auth(..) - /etc/inc/authgui.inc:33 require_once(..) - /usr/local/www/guiconfig.inc:51 require_once(..) - /usr/local/www/index.php:44 [1] => #### phpsession_end #### simplestacktrace(..) - /etc/inc/phpsessionmanager.inc:54 phpsession_end(..) - /etc/inc/auth.inc:2091 session_auth(..) - /etc/inc/authgui.inc:33 require_once(..) - /usr/local/www/guiconfig.inc:51 require_once(..) - /usr/local/www/index.php:44 [2] => #### phpsession_begin #### simplestacktrace(..) - /etc/inc/phpsessionmanager.inc:38 phpsession_begin(..) - /etc/inc/authgui.inc:38 require_once(..) - /usr/local/www/guiconfig.inc:51 require_once(..) - /usr/local/www/index.php:44 [3] => #### phpsession_begin #### simplestacktrace(..) - /etc/inc/phpsessionmanager.inc:38 phpsession_begin(..) - /etc/inc/priv.inc:

I'm not really a coder (at all ), but maybe the script just needs a simple change?

stephenw10 · Oct 5, 2018, 2:55 PM

The first one looks like you didn't add any permissions to the 'backup' user so it can't access that page.

I'm not sure that second error is related at all. Are you sure it appears everytime after running the script?

Steve

wgstarks · Oct 5, 2018, 3:01 PM

@stephenw10
The backup user has permission for webcfg- backup and restore. Should I add more?

The second error is definitely related to the script. If I run the script manually I see the error immediately in the pfsense system log, every time the script is run.

stephenw10 · Oct 5, 2018, 3:05 PM

@wgstarks said in Help with pfsense backup script:

attempted to access /index.php

It tries to access the idex page first do give it permission to remove that error. It's not actually causing a problem though, it just gets redirected to the only page it had access to.

Steve

Gertjan · Oct 5, 2018, 3:20 PM

You saw https://www.netgate.com/docs/pfsense/backup/remote-config-backup.html ?

I just tested the 3 wget lines at the top (I only changed the password ...)
Worked perfectly well, I recived a file called " config-router-20181005171831.xml" which is a copy of my config.

Btw : ran the wget commands from my NAS, a Synology drive.

stephenw10 · Oct 5, 2018, 3:25 PM

Ah, well spotted. I'd assumed it was that script. I should know to assume nothing by now!

Steve

wgstarks · Oct 5, 2018, 3:40 PM

@gertjan
Yes. The script works, just trying to clear up the errors generated in pfsense when I use it.

stephenw10 · Oct 5, 2018, 5:59 PM

You'll notice the script linked though is slightly different to the one you're using and doesn't generate errors.

I would switch to that, it's relatively well tested.

Steve

wgstarks · Oct 5, 2018, 6:40 PM

@stephenw10
Yeah. I’m a little slow sometimes.
Didn’t see the difference right off. I’ll give the changes a shot later today and probably change the users permissions too, to get rid of that error as well.

@Gertjan
Thanks for posting the link.

stephenw10 · Oct 5, 2018, 7:46 PM

You shouldn't need to change the permissions with the script change. It opens the backup page directly.

Steve

wgstarks · Oct 5, 2018, 7:49 PM

@stephenw10
Thanks again.

wgstarks · Oct 6, 2018, 12:53 AM

Here's what I finally worked out-

BACKUP_HOST=<gateway_IP>
BACKUP_USER=<user_name>
BACKUP_PASSWORD=<user_password>

# Create config file directory if it doesn't exist
[ -d files/ ] || mkdir files

# Fetch the login form and save the cookies and CSRF token:
wget -qO- --keep-session-cookies --save-cookies cookies.txt \
  --no-check-certificate https://${BACKUP_HOST}/diag_backup.php \
  | grep "name='__csrf_magic'" | sed 's/.*value="\(.*\)".*/\1/' > csrf.txt

# Submit the login form along with the first CSRF token and save the second CSRF token (can’t reuse the same file) – now the script is logged in and can take action:
wget -qO- --keep-session-cookies --load-cookies cookies.txt \
  --save-cookies cookies.txt --no-check-certificate \
  --post-data "login=Login&usernamefld=${BACKUP_USER}&passwordfld=${BACKUP_PASSWORD}&__csrf_magic=$(cat csrf.txt)" \
  https://${BACKUP_HOST}/diag_backup.php  | grep "name='__csrf_magic'" \
  | sed 's/.*value="\(.*\)".*/\1/' > csrf2.txt

# Submit the download form along with the second CSRF token to save a copy of config.xml:
wget --keep-session-cookies --load-cookies cookies.txt --no-check-certificate \
  --post-data "download=download&donotbackuprrd=yes&__csrf_magic=$(head -n 1 csrf2.txt)" \
  https://${BACKUP_HOST}/diag_backup.php -O ./files/config_${BACKUP_HOST}_$(date +%Y-%m-%d-%H-%M-%S).xml 2>/dev/null

# Clean up
rm cookies.txt csrf.txt csrf2.txt
unset BACKUP_HOST BACKUP_USER BACKUP_PASSWORD

# Remove files older than 100 days
find /mnt/user/odin_backup/OdinBackUp/files/ -type f -name '*.xml' -mtime +100 -exec rm {} \;

I did have to change permissions for the backup user though. Even when I used the code in the link that @Gertjan provided and just substituted the correct IP, user and password I would still get the error shown in my first post. Once I added "all pages" to the backup user's permissions the errors went away. I think that the default code in the link didn't generate an error because it uses the default admin/pfsense user which has full privileges IIRC. Just a guess.

@Gertjan and @stephenw10 Thanks again for your help. Very much appreciate it.