Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Fetch configuration with curl or wget

    Scheduled Pinned Locked Moved Problems Installing or Upgrading pfSense Software
    13 Posts 3 Posters 8.0k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • F
      frater
      last edited by

      Thank you very much….

      I created this hourly cronjob on a remote server.
      It will delete the XML if it's the same as the previous one.
      This way you can immediately see when configs have changed.

      I did this before and it's very helpful.

      ln -s /usr/local/sbin/pfsense_getxml /etc/cron.hourly/

      /usr/local/sbin/pfsense_getxml

      
      #!/bin/sh
      
      DATESTAMP=`date +%Y-%m-%d.%H:%M`
      FNAME=pfsense.${DATESTAMP}.xml
      FOLDER=/var/www/vhosts/mydomain.com/pfsense
      
      USER=admin
      PASS=pfsense
      
      IP=80.232.169.117
      PORT=80
      
      if cd ${FOLDER} ; then
      
        FGROUP=`stat -c%G .`
        FUSER=`stat -c%U .`
      
        LASTXML=`ls -1t pfsense*xml 2>/dev/null | head -n1`
      
        if curl -u${USER}:${PASS} http://${IP}:${PORT}/zabbix.php 2>/dev/null | base64 -d 2>/dev/null >${FNAME} ; then
          chown ${FUSER}:${FGROUP} ${FNAME}
      
          if [ ! -z "${LASTXML}" ] ; then
            if [ ! "${LASTXML}" = "${FNAME}" ] ; then
              diff ${LASTXML} ${FNAME} >/dev/null && rm -f ${FNAME}
            fi
          fi
        else
          rm -f ${FNAME}
        fi
      else
        exit 1
      fi
      
      
      1 Reply Last reply Reply Quote 0
      • jimpJ
        jimp Rebel Alliance Developer Netgate
        last edited by

        If you go through that much trouble you may as well have it check the config into an SCM like git or svn. Then you can view the diffs, and you wouldn't have a bunch of redundant identical copies floating around.

        Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

        Need help fast? Netgate Global Support!

        Do not Chat/PM for help!

        1 Reply Last reply Reply Quote 0
        • F
          frater
          last edited by

          There are no redundant identical copies….

          1 Reply Last reply Reply Quote 0
          • jimpJ
            jimp Rebel Alliance Developer Netgate
            last edited by

            Ah, yeah I see the && rm now. Still seems a bit over-eager.

            On another note, I wouldn't want a non-password-protected page feeding up the config.xml file though, even protected by IP, but that's me.

            Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

            Need help fast? Netgate Global Support!

            Do not Chat/PM for help!

            1 Reply Last reply Reply Quote 0
            • marcellocM
              marcelloc
              last edited by

              @jimp:

              On another note, I wouldn't want a non-password-protected page feeding up the config.xml file though, even protected by IP, but that's me.

              I agree with you, it was just a fast example on how to do this.

              It's hard to decide between ip auth or leaving firewall password on a clear text script in zabix server not managed by firewall guys

              So a better example could be:

              
              #zabix server ip
              $zabix_ip='192.168.1.122';
              $password="some_password_to_secure_script";
              if ($_SERVER["REMOTE_ADDR"]==$zabix_ip && $_REQUEST['pass']== $password)
                print base64_encode(file_get_contents('/conf/config.xml'));
              
              ?>
              
              

              Treinamentos de Elite: http://sys-squad.com

              Help a community developer! ;D

              1 Reply Last reply Reply Quote 0
              • jimpJ
                jimp Rebel Alliance Developer Netgate
                last edited by

                Setup ssh keys and copy the config that way, no need to have passwords in plaintext anywhere. Whether you want to copy it to, or from, the firewalls is the only question there. Make the keys (without a passphrase), add them where you want, and cron a command to scp the config.

                Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                Need help fast? Netgate Global Support!

                Do not Chat/PM for help!

                1 Reply Last reply Reply Quote 0
                • F
                  frater
                  last edited by

                  I understand the criticism and acknowledge the security risks…
                  If someone has root access to my remote server it would be a real problem...
                  Access to my pfsense config is not that trivial....

                  I did change the protocol from http to https and I'm also saving a diff for a quick and dirty changelog....

                  #!/bin/sh
                  
                  DATESTAMP=`date +%Y-%m-%d.%H:%M`
                  FNAME=pfsense.${DATESTAMP}.xml
                  FOLDER=/var/www/vhosts/mr-wolf.nl/pfsense
                  
                  USER=admin
                  PASS=pfsense
                  
                  PROTO=https             # http or https
                  IP=pfsense.yourdomain.com    # DNS or IP of webif (remote side)
                  PORT=443              # port of webif (remote side)
                  
                  if cd ${FOLDER} ; then
                  
                    FGROUP=`stat -c%G .`
                    FUSER=`stat -c%U .`
                    LASTXML=`ls -1t pfsense*xml 2>/dev/null | head -n1`
                  
                    if curl -u${USER}:${PASS} ${PROTO}://${IP}:${PORT}/zabbix.php 2>/dev/null | base64 -d 2>/dev/null >${FNAME} ; then
                      chown ${FUSER}:${FGROUP} ${FNAME}
                  
                      if [ ! -z "${LASTXML}" ] ; then
                        if [ ! "${LASTXML}" = "${FNAME}" ] ; then
                          if diff ${LASTXML} ${FNAME} >${FNAME}.diff ; then
                            rm -f ${FNAME}*
                          else
                            chown ${FUSER}:${FGROUP} ${FNAME}.diff
                          fi
                        fi
                      fi
                    else
                      rm -f ${FNAME}
                      exit 1
                    fi
                  else
                    exit 1
                  fi
                  
                  
                  1 Reply Last reply Reply Quote 0
                  • F
                    frater
                    last edited by

                    I somehow missed that wiki-entry or maybe it wasn't there when I started this thread.
                    Recently I replaced my pfsense machine and needed to recreate the little file on my pfsense.
                    But this isn't necessary if I would use the way it was described in the wiki.
                    Here's the code I'm using now.
                    It doesn't need a change for your pfsense

                    #!/bin/sh
                    
                    DATESTAMP=`date +%Y-%m-%d.%H:%M`
                    FNAME=pfsense.${DATESTAMP}.xml
                    FOLDER=/var/www/vhosts/yourdomain.com/pfsense
                    
                    USER=admin
                    PASS=pfsense
                    
                    PROTO=https             # http or https
                    IP=80.23.120.38         # DNS or IP of webif (remote side)
                    PORT=6443               # port of webif (remote side)
                    WGETOPT=
                    
                    # turn off certificate checking
                    [ "${PROTO}" = "https" ] && WGETOPT="${WGETOPT} --no-check-certificate"
                    
                    if cd ${FOLDER} ; then
                    
                      FGROUP=`stat -c%G .`
                      FUSER=`stat -c%U .`
                      LASTXML=`ls -1t pfsense*xml 2>/dev/null | head -n1`
                    
                      wget -qO/dev/null --keep-session-cookies --save-cookies /tmp/pfsense_cookies.txt  --post-data "login=Login&usernamefld=${USER}&passwordfld=${PASS}"  --no-check-certificate ${PROTO}://${IP}:${PORT}/diag_backup.php
                      wget -qO${FNAME} --keep-session-cookies --load-cookies /tmp/pfsense_cookies.txt  --post-data 'Submit=download&donotbackuprrd=yes' ${WGETOPT} ${PROTO}://${IP}:${PORT}/diag_backup.php
                    
                      if [ -s ${FNAME} ] ; then
                        chown ${FUSER}:${FGROUP} ${FNAME}
                    
                        if [ ! -z "${LASTXML}" ] ; then
                          if [ ! "${LASTXML}" = "${FNAME}" ] ; then
                            if diff ${LASTXML} ${FNAME} >${FNAME}.diff ; then
                              rm -f ${FNAME}*
                            else
                              chown ${FUSER}:${FGROUP} ${FNAME}.diff
                            fi
                          fi
                        fi
                      else
                        rm -f ${FNAME}
                        exit 1
                      fi
                    else
                      exit 1
                    fi
                    
                    
                    1 Reply Last reply Reply Quote 0
                    • marcellocM
                      marcelloc
                      last edited by

                      Frater,  hide you public IP address and pasword from your post.

                      Treinamentos de Elite: http://sys-squad.com

                      Help a community developer! ;D

                      1 Reply Last reply Reply Quote 0
                      • F
                        frater
                        last edited by

                        @marcelloc:

                        Frater,  hide you public IP address and pasword from your post.

                        Those were fake…
                        But thanks for your concern...

                        I can't edit my post, but I saw a little error in the first wget where I hardcoded the --no-check-certificate
                        That option is inside the variable "${WGETOPT}"

                          wget -qO/dev/null --keep-session-cookies --save-cookies /tmp/pfsense_cookies.txt  --post-data "login=Login&usernamefld=${USER}&passwordfld=${PASS}" ${WGETOPT} ${PROTO}://${IP}:${PORT}/diag_backup.php
                          wget -qO${FNAME} --keep-session-cookies --load-cookies /tmp/pfsense_cookies.txt  --post-data 'Submit=download&donotbackuprrd=yes' ${WGETOPT} ${PROTO}://${IP}:${PORT}/diag_backup.php
                        
                        

                        I don't know if anyone will be using it, but if it even helps only one man it was worth posting it.

                        1 Reply Last reply Reply Quote 0
                        • First post
                          Last post
                        Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.