Fetch configuration with curl or wget

frater · Jan 5, 2012, 1:45 PM

Today my pfsense crashed after a power surge…
The xml-file it could recover from the SSD was rejected, so I used a backup I recently made using the webif.

I'm running a Zabbix-server on a remote location that I can instruct to do automated tasks....
I could fetch the XML-file with curl and save it each day or twice a day...
Zabbix can even alarm me if there's no recent XML-file in a certain directory....

What link should I use to fetch the file with http?
Something like this with an instruction behind it.....

https://user:pass@pfsense.mydomain.com/diag_backup.php

I'm interested in other ways of course, but if it's possible I would really want to know which link I should use…..

marcelloc · Jan 5, 2012, 2:15 PM

You can do a php script to send it to your zabix server.

/usr/local/www/zabix.php

#zabix server ip
$zabix_ip='192.168.1.122';

if ($_SERVER["REMOTE_ADDR"]==$zabix_ip)
  print base64_encode(file_get_contents('/conf/config.xml'));

?>

This sample code returns a base64 encoded config.xml file.

To read it, just do a base64 decode on string you fetch.

jimp · Jan 5, 2012, 3:03 PM

You can also use the wget method shown in the wiki:

http://doc.pfsense.org/index.php/Remote_Config_Backup

frater · Jan 5, 2012, 3:47 PM

Thank you very much….

I created this hourly cronjob on a remote server.
It will delete the XML if it's the same as the previous one.
This way you can immediately see when configs have changed.

I did this before and it's very helpful.

ln -s /usr/local/sbin/pfsense_getxml /etc/cron.hourly/

/usr/local/sbin/pfsense_getxml


#!/bin/sh

DATESTAMP=`date +%Y-%m-%d.%H:%M`
FNAME=pfsense.${DATESTAMP}.xml
FOLDER=/var/www/vhosts/mydomain.com/pfsense

USER=admin
PASS=pfsense

IP=80.232.169.117
PORT=80

if cd ${FOLDER} ; then

  FGROUP=`stat -c%G .`
  FUSER=`stat -c%U .`

  LASTXML=`ls -1t pfsense*xml 2>/dev/null | head -n1`

  if curl -u${USER}:${PASS} http://${IP}:${PORT}/zabbix.php 2>/dev/null | base64 -d 2>/dev/null >${FNAME} ; then
    chown ${FUSER}:${FGROUP} ${FNAME}

    if [ ! -z "${LASTXML}" ] ; then
      if [ ! "${LASTXML}" = "${FNAME}" ] ; then
        diff ${LASTXML} ${FNAME} >/dev/null && rm -f ${FNAME}
      fi
    fi
  else
    rm -f ${FNAME}
  fi
else
  exit 1
fi

jimp · Jan 5, 2012, 4:26 PM

If you go through that much trouble you may as well have it check the config into an SCM like git or svn. Then you can view the diffs, and you wouldn't have a bunch of redundant identical copies floating around.

frater · Jan 5, 2012, 4:35 PM

There are no redundant identical copies….

jimp · Jan 5, 2012, 4:38 PM

Ah, yeah I see the && rm now. Still seems a bit over-eager.

On another note, I wouldn't want a non-password-protected page feeding up the config.xml file though, even protected by IP, but that's me.

marcelloc · Jan 5, 2012, 5:38 PM

@jimp:

On another note, I wouldn't want a non-password-protected page feeding up the config.xml file though, even protected by IP, but that's me.

I agree with you, it was just a fast example on how to do this.

It's hard to decide between ip auth or leaving firewall password on a clear text script in zabix server not managed by firewall guys

So a better example could be:


#zabix server ip
$zabix_ip='192.168.1.122';
$password="some_password_to_secure_script";
if ($_SERVER["REMOTE_ADDR"]==$zabix_ip && $_REQUEST['pass']== $password)
  print base64_encode(file_get_contents('/conf/config.xml'));

?>

jimp · Jan 5, 2012, 5:42 PM

Setup ssh keys and copy the config that way, no need to have passwords in plaintext anywhere. Whether you want to copy it to, or from, the firewalls is the only question there. Make the keys (without a passphrase), add them where you want, and cron a command to scp the config.

frater · Jan 16, 2012, 11:19 AM

I understand the criticism and acknowledge the security risks…
If someone has root access to my remote server it would be a real problem...
Access to my pfsense config is not that trivial....

I did change the protocol from http to https and I'm also saving a diff for a quick and dirty changelog....

#!/bin/sh

DATESTAMP=`date +%Y-%m-%d.%H:%M`
FNAME=pfsense.${DATESTAMP}.xml
FOLDER=/var/www/vhosts/mr-wolf.nl/pfsense

USER=admin
PASS=pfsense

PROTO=https             # http or https
IP=pfsense.yourdomain.com    # DNS or IP of webif (remote side)
PORT=443              # port of webif (remote side)

if cd ${FOLDER} ; then

  FGROUP=`stat -c%G .`
  FUSER=`stat -c%U .`
  LASTXML=`ls -1t pfsense*xml 2>/dev/null | head -n1`

  if curl -u${USER}:${PASS} ${PROTO}://${IP}:${PORT}/zabbix.php 2>/dev/null | base64 -d 2>/dev/null >${FNAME} ; then
    chown ${FUSER}:${FGROUP} ${FNAME}

    if [ ! -z "${LASTXML}" ] ; then
      if [ ! "${LASTXML}" = "${FNAME}" ] ; then
        if diff ${LASTXML} ${FNAME} >${FNAME}.diff ; then
          rm -f ${FNAME}*
        else
          chown ${FUSER}:${FGROUP} ${FNAME}.diff
        fi
      fi
    fi
  else
    rm -f ${FNAME}
    exit 1
  fi
else
  exit 1
fi

frater · Aug 4, 2012, 11:51 AM

I somehow missed that wiki-entry or maybe it wasn't there when I started this thread.
Recently I replaced my pfsense machine and needed to recreate the little file on my pfsense.
But this isn't necessary if I would use the way it was described in the wiki.
Here's the code I'm using now.
It doesn't need a change for your pfsense

#!/bin/sh

DATESTAMP=`date +%Y-%m-%d.%H:%M`
FNAME=pfsense.${DATESTAMP}.xml
FOLDER=/var/www/vhosts/yourdomain.com/pfsense

USER=admin
PASS=pfsense

PROTO=https             # http or https
IP=80.23.120.38         # DNS or IP of webif (remote side)
PORT=6443               # port of webif (remote side)
WGETOPT=

# turn off certificate checking
[ "${PROTO}" = "https" ] && WGETOPT="${WGETOPT} --no-check-certificate"

if cd ${FOLDER} ; then

  FGROUP=`stat -c%G .`
  FUSER=`stat -c%U .`
  LASTXML=`ls -1t pfsense*xml 2>/dev/null | head -n1`

  wget -qO/dev/null --keep-session-cookies --save-cookies /tmp/pfsense_cookies.txt  --post-data "login=Login&usernamefld=${USER}&passwordfld=${PASS}"  --no-check-certificate ${PROTO}://${IP}:${PORT}/diag_backup.php
  wget -qO${FNAME} --keep-session-cookies --load-cookies /tmp/pfsense_cookies.txt  --post-data 'Submit=download&donotbackuprrd=yes' ${WGETOPT} ${PROTO}://${IP}:${PORT}/diag_backup.php

  if [ -s ${FNAME} ] ; then
    chown ${FUSER}:${FGROUP} ${FNAME}

    if [ ! -z "${LASTXML}" ] ; then
      if [ ! "${LASTXML}" = "${FNAME}" ] ; then
        if diff ${LASTXML} ${FNAME} >${FNAME}.diff ; then
          rm -f ${FNAME}*
        else
          chown ${FUSER}:${FGROUP} ${FNAME}.diff
        fi
      fi
    fi
  else
    rm -f ${FNAME}
    exit 1
  fi
else
  exit 1
fi

marcelloc · Aug 5, 2012, 1:56 AM

Frater, hide you public IP address and pasword from your post.

frater · Aug 8, 2012, 1:58 PM

@marcelloc:

Frater, hide you public IP address and pasword from your post.

Those were fake…
But thanks for your concern...

I can't edit my post, but I saw a little error in the first wget where I hardcoded the --no-check-certificate
That option is inside the variable "${WGETOPT}"

  wget -qO/dev/null --keep-session-cookies --save-cookies /tmp/pfsense_cookies.txt  --post-data "login=Login&usernamefld=${USER}&passwordfld=${PASS}" ${WGETOPT} ${PROTO}://${IP}:${PORT}/diag_backup.php
  wget -qO${FNAME} --keep-session-cookies --load-cookies /tmp/pfsense_cookies.txt  --post-data 'Submit=download&donotbackuprrd=yes' ${WGETOPT} ${PROTO}://${IP}:${PORT}/diag_backup.php

I don't know if anyone will be using it, but if it even helps only one man it was worth posting it.