Help with pfsense backup script



  • I found a simple script to run backups from a remote computer which works but is generating some errors.

    #!/bin/bash
    
    BACKUP_HOST=10.0.1.1
    BACKUP_USER=<username>
    BACKUP_PASSWORD=<password>
    
    # Create config file directory if it doesn't exist
    [ -d files/ ] || mkdir files
    
    # Get CSRF token
    CSRF=$(wget -qO- --keep-session-cookies --save-cookies cookies.txt --no-check-certificate https://${BACKUP_HOST}/ | grep "name='__csrf_magic'" | sed 's/.*value="\(.*\)".*/\1/')
    
    # Log into pfSense
    CSRF=$(wget -qO- --keep-session-cookies --load-cookies cookies.txt --save-cookies cookies.txt --no-check-certificate --post-data "login=Login&usernamefld=${BACKUP_USER}&passwordfld=${BACKUP_PASSWORD}&__csrf_magic=${CSRF}" https://${BACKUP_HOST}/ | grep "name='__csrf_magic'" | sed 's/.*value="\(.*\)".*/\1/')
    
    # Save configuration file
    wget --keep-session-cookies --load-cookies cookies.txt --no-check-certificate --post-data "download=Download%20configuration%20as%20XML&donotbackuprrd=yes&__csrf_magic=${CSRF}" https://${BACKUP_HOST}/diag_backup.php -O ./files/config_${BACKUP_HOST}_$(date +%Y-%m-%d-%H-%M-%S).xml 2>/dev/null
    
    # Clean up
    rm cookies.txt
    unset BACKUP_HOST BACKUP_USER BACKUP_PASSWORD
    
    # Remove files older than 100 days
    find /mnt/user/odin_backup/OdinBackUp/files/ -type f -name '*.xml' -mtime +100 -exec rm {} \;
    

    Every time it runs I see this error in the pfsense system log-

    Edit: Just realized there are two errors-

    php-fpm		/index.php: backup@10.0.1.20 (Local Database) attempted to access /index.php but does not have access to that page. Redirecting to diag_backup.php.
    

    And then-

    php-fpm		PHPSESSION 1 open sessions left at shutdown script!Array ( [0] => #### phpsession_begin #### simplestacktrace(..) - /etc/inc/phpsessionmanager.inc:38 phpsession_begin(..) - /etc/inc/auth.inc:1955 session_auth(..) - /etc/inc/authgui.inc:33 require_once(..) - /usr/local/www/guiconfig.inc:51 require_once(..) - /usr/local/www/index.php:44 [1] => #### phpsession_end #### simplestacktrace(..) - /etc/inc/phpsessionmanager.inc:54 phpsession_end(..) - /etc/inc/auth.inc:2091 session_auth(..) - /etc/inc/authgui.inc:33 require_once(..) - /usr/local/www/guiconfig.inc:51 require_once(..) - /usr/local/www/index.php:44 [2] => #### phpsession_begin #### simplestacktrace(..) - /etc/inc/phpsessionmanager.inc:38 phpsession_begin(..) - /etc/inc/authgui.inc:38 require_once(..) - /usr/local/www/guiconfig.inc:51 require_once(..) - /usr/local/www/index.php:44 [3] => #### phpsession_begin #### simplestacktrace(..) - /etc/inc/phpsessionmanager.inc:38 phpsession_begin(..) - /etc/inc/priv.inc:
    

    I'm not really a coder (at all 😏 ), but maybe the script just needs a simple change?


  • Netgate Administrator

    The first one looks like you didn't add any permissions to the 'backup' user so it can't access that page.

    I'm not sure that second error is related at all. Are you sure it appears everytime after running the script?

    Steve



  • @stephenw10
    The backup user has permission for webcfg- backup and restore. Should I add more?

    The second error is definitely related to the script. If I run the script manually I see the error immediately in the pfsense system log, every time the script is run.


  • Netgate Administrator

    @wgstarks said in Help with pfsense backup script:

    attempted to access /index.php

    It tries to access the idex page first do give it permission to remove that error. It's not actually causing a problem though, it just gets redirected to the only page it had access to.

    Steve



  • You saw https://www.netgate.com/docs/pfsense/backup/remote-config-backup.html ?

    I just tested the 3 wget lines at the top (I only changed the password ...)
    Worked perfectly well, I recived a file called " config-router-20181005171831.xml" which is a copy of my config.

    Btw : ran the wget commands from my NAS, a Synology drive.


  • Netgate Administrator

    Ah, well spotted. I'd assumed it was that script. I should know to assume nothing by now! 🙄

    Steve



  • @gertjan
    Yes. The script works, just trying to clear up the errors generated in pfsense when I use it.


  • Netgate Administrator

    You'll notice the script linked though is slightly different to the one you're using and doesn't generate errors.

    I would switch to that, it's relatively well tested.

    Steve



  • @stephenw10
    Yeah. I’m a little slow sometimes.😁
    Didn’t see the difference right off. I’ll give the changes a shot later today and probably change the users permissions too, to get rid of that error as well.

    @Gertjan
    Thanks for posting the link.


  • Netgate Administrator

    You shouldn't need to change the permissions with the script change. It opens the backup page directly.

    Steve



  • @stephenw10
    Thanks again.



  • Here's what I finally worked out-

    BACKUP_HOST=<gateway_IP>
    BACKUP_USER=<user_name>
    BACKUP_PASSWORD=<user_password>
    
    # Create config file directory if it doesn't exist
    [ -d files/ ] || mkdir files
    
    # Fetch the login form and save the cookies and CSRF token:
    wget -qO- --keep-session-cookies --save-cookies cookies.txt \
      --no-check-certificate https://${BACKUP_HOST}/diag_backup.php \
      | grep "name='__csrf_magic'" | sed 's/.*value="\(.*\)".*/\1/' > csrf.txt
    
    # Submit the login form along with the first CSRF token and save the second CSRF token (can’t reuse the same file) – now the script is logged in and can take action:
    wget -qO- --keep-session-cookies --load-cookies cookies.txt \
      --save-cookies cookies.txt --no-check-certificate \
      --post-data "login=Login&usernamefld=${BACKUP_USER}&passwordfld=${BACKUP_PASSWORD}&__csrf_magic=$(cat csrf.txt)" \
      https://${BACKUP_HOST}/diag_backup.php  | grep "name='__csrf_magic'" \
      | sed 's/.*value="\(.*\)".*/\1/' > csrf2.txt
    
    # Submit the download form along with the second CSRF token to save a copy of config.xml:
    wget --keep-session-cookies --load-cookies cookies.txt --no-check-certificate \
      --post-data "download=download&donotbackuprrd=yes&__csrf_magic=$(head -n 1 csrf2.txt)" \
      https://${BACKUP_HOST}/diag_backup.php -O ./files/config_${BACKUP_HOST}_$(date +%Y-%m-%d-%H-%M-%S).xml 2>/dev/null
    
    # Clean up
    rm cookies.txt csrf.txt csrf2.txt
    unset BACKUP_HOST BACKUP_USER BACKUP_PASSWORD
    
    # Remove files older than 100 days
    find /mnt/user/odin_backup/OdinBackUp/files/ -type f -name '*.xml' -mtime +100 -exec rm {} \;
    

    I did have to change permissions for the backup user though. Even when I used the code in the link that @Gertjan provided and just substituted the correct IP, user and password I would still get the error shown in my first post. Once I added "all pages" to the backup user's permissions the errors went away. I think that the default code in the link didn't generate an error because it uses the default admin/pfsense user which has full privileges IIRC. Just a guess.

    @Gertjan and @stephenw10 Thanks again for your help. Very much appreciate it.