Fetch configuration with curl or wget



  • Today my pfsense crashed after a power surge…
    The xml-file it could recover from the SSD was rejected, so I used a backup I recently made using the webif.

    I'm running a Zabbix-server on a remote location that I can instruct to do automated tasks....
    I could fetch the XML-file with curl and save it each day or twice a day...
    Zabbix can even alarm me if there's no recent XML-file in a certain directory....

    What link should I use to fetch the file with http?
    Something like this with an instruction behind it.....

    https://user:pass@pfsense.mydomain.com/diag_backup.php
    

    I'm interested in other ways of course, but if it's possible I would really want to know which link I should use…..



  • You can do a php script to send it to your zabix server.

    /usr/local/www/zabix.php

    #zabix server ip
    $zabix_ip='192.168.1.122';
    
    if ($_SERVER["REMOTE_ADDR"]==$zabix_ip)
      print base64_encode(file_get_contents('/conf/config.xml'));
    
    ?>
    
    

    This sample code returns a base64 encoded config.xml file.

    To read it, just do a base64 decode on string you fetch.


  • Rebel Alliance Developer Netgate

    You can also use the wget method shown in the wiki:

    http://doc.pfsense.org/index.php/Remote_Config_Backup



  • Thank you very much….

    I created this hourly cronjob on a remote server.
    It will delete the XML if it's the same as the previous one.
    This way you can immediately see when configs have changed.

    I did this before and it's very helpful.

    ln -s /usr/local/sbin/pfsense_getxml /etc/cron.hourly/

    /usr/local/sbin/pfsense_getxml

    
    #!/bin/sh
    
    DATESTAMP=`date +%Y-%m-%d.%H:%M`
    FNAME=pfsense.${DATESTAMP}.xml
    FOLDER=/var/www/vhosts/mydomain.com/pfsense
    
    USER=admin
    PASS=pfsense
    
    IP=80.232.169.117
    PORT=80
    
    if cd ${FOLDER} ; then
    
      FGROUP=`stat -c%G .`
      FUSER=`stat -c%U .`
    
      LASTXML=`ls -1t pfsense*xml 2>/dev/null | head -n1`
    
      if curl -u${USER}:${PASS} http://${IP}:${PORT}/zabbix.php 2>/dev/null | base64 -d 2>/dev/null >${FNAME} ; then
        chown ${FUSER}:${FGROUP} ${FNAME}
    
        if [ ! -z "${LASTXML}" ] ; then
          if [ ! "${LASTXML}" = "${FNAME}" ] ; then
            diff ${LASTXML} ${FNAME} >/dev/null && rm -f ${FNAME}
          fi
        fi
      else
        rm -f ${FNAME}
      fi
    else
      exit 1
    fi
    
    

  • Rebel Alliance Developer Netgate

    If you go through that much trouble you may as well have it check the config into an SCM like git or svn. Then you can view the diffs, and you wouldn't have a bunch of redundant identical copies floating around.



  • There are no redundant identical copies….


  • Rebel Alliance Developer Netgate

    Ah, yeah I see the && rm now. Still seems a bit over-eager.

    On another note, I wouldn't want a non-password-protected page feeding up the config.xml file though, even protected by IP, but that's me.



  • @jimp:

    On another note, I wouldn't want a non-password-protected page feeding up the config.xml file though, even protected by IP, but that's me.

    I agree with you, it was just a fast example on how to do this.

    It's hard to decide between ip auth or leaving firewall password on a clear text script in zabix server not managed by firewall guys

    So a better example could be:

    
    #zabix server ip
    $zabix_ip='192.168.1.122';
    $password="some_password_to_secure_script";
    if ($_SERVER["REMOTE_ADDR"]==$zabix_ip && $_REQUEST['pass']== $password)
      print base64_encode(file_get_contents('/conf/config.xml'));
    
    ?>
    
    

  • Rebel Alliance Developer Netgate

    Setup ssh keys and copy the config that way, no need to have passwords in plaintext anywhere. Whether you want to copy it to, or from, the firewalls is the only question there. Make the keys (without a passphrase), add them where you want, and cron a command to scp the config.



  • I understand the criticism and acknowledge the security risks…
    If someone has root access to my remote server it would be a real problem...
    Access to my pfsense config is not that trivial....

    I did change the protocol from http to https and I'm also saving a diff for a quick and dirty changelog....

    #!/bin/sh
    
    DATESTAMP=`date +%Y-%m-%d.%H:%M`
    FNAME=pfsense.${DATESTAMP}.xml
    FOLDER=/var/www/vhosts/mr-wolf.nl/pfsense
    
    USER=admin
    PASS=pfsense
    
    PROTO=https             # http or https
    IP=pfsense.yourdomain.com    # DNS or IP of webif (remote side)
    PORT=443              # port of webif (remote side)
    
    if cd ${FOLDER} ; then
    
      FGROUP=`stat -c%G .`
      FUSER=`stat -c%U .`
      LASTXML=`ls -1t pfsense*xml 2>/dev/null | head -n1`
    
      if curl -u${USER}:${PASS} ${PROTO}://${IP}:${PORT}/zabbix.php 2>/dev/null | base64 -d 2>/dev/null >${FNAME} ; then
        chown ${FUSER}:${FGROUP} ${FNAME}
    
        if [ ! -z "${LASTXML}" ] ; then
          if [ ! "${LASTXML}" = "${FNAME}" ] ; then
            if diff ${LASTXML} ${FNAME} >${FNAME}.diff ; then
              rm -f ${FNAME}*
            else
              chown ${FUSER}:${FGROUP} ${FNAME}.diff
            fi
          fi
        fi
      else
        rm -f ${FNAME}
        exit 1
      fi
    else
      exit 1
    fi
    
    


  • I somehow missed that wiki-entry or maybe it wasn't there when I started this thread.
    Recently I replaced my pfsense machine and needed to recreate the little file on my pfsense.
    But this isn't necessary if I would use the way it was described in the wiki.
    Here's the code I'm using now.
    It doesn't need a change for your pfsense

    #!/bin/sh
    
    DATESTAMP=`date +%Y-%m-%d.%H:%M`
    FNAME=pfsense.${DATESTAMP}.xml
    FOLDER=/var/www/vhosts/yourdomain.com/pfsense
    
    USER=admin
    PASS=pfsense
    
    PROTO=https             # http or https
    IP=80.23.120.38         # DNS or IP of webif (remote side)
    PORT=6443               # port of webif (remote side)
    WGETOPT=
    
    # turn off certificate checking
    [ "${PROTO}" = "https" ] && WGETOPT="${WGETOPT} --no-check-certificate"
    
    if cd ${FOLDER} ; then
    
      FGROUP=`stat -c%G .`
      FUSER=`stat -c%U .`
      LASTXML=`ls -1t pfsense*xml 2>/dev/null | head -n1`
    
      wget -qO/dev/null --keep-session-cookies --save-cookies /tmp/pfsense_cookies.txt  --post-data "login=Login&usernamefld=${USER}&passwordfld=${PASS}"  --no-check-certificate ${PROTO}://${IP}:${PORT}/diag_backup.php
      wget -qO${FNAME} --keep-session-cookies --load-cookies /tmp/pfsense_cookies.txt  --post-data 'Submit=download&donotbackuprrd=yes' ${WGETOPT} ${PROTO}://${IP}:${PORT}/diag_backup.php
    
      if [ -s ${FNAME} ] ; then
        chown ${FUSER}:${FGROUP} ${FNAME}
    
        if [ ! -z "${LASTXML}" ] ; then
          if [ ! "${LASTXML}" = "${FNAME}" ] ; then
            if diff ${LASTXML} ${FNAME} >${FNAME}.diff ; then
              rm -f ${FNAME}*
            else
              chown ${FUSER}:${FGROUP} ${FNAME}.diff
            fi
          fi
        fi
      else
        rm -f ${FNAME}
        exit 1
      fi
    else
      exit 1
    fi
    
    


  • Frater,  hide you public IP address and pasword from your post.



  • @marcelloc:

    Frater,  hide you public IP address and pasword from your post.

    Those were fake…
    But thanks for your concern...

    I can't edit my post, but I saw a little error in the first wget where I hardcoded the --no-check-certificate
    That option is inside the variable "${WGETOPT}"

      wget -qO/dev/null --keep-session-cookies --save-cookies /tmp/pfsense_cookies.txt  --post-data "login=Login&usernamefld=${USER}&passwordfld=${PASS}" ${WGETOPT} ${PROTO}://${IP}:${PORT}/diag_backup.php
      wget -qO${FNAME} --keep-session-cookies --load-cookies /tmp/pfsense_cookies.txt  --post-data 'Submit=download&donotbackuprrd=yes' ${WGETOPT} ${PROTO}://${IP}:${PORT}/diag_backup.php
    
    

    I don't know if anyone will be using it, but if it even helps only one man it was worth posting it.


Locked