Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Fetch configuration with curl or wget

    Scheduled Pinned Locked Moved Problems Installing or Upgrading pfSense Software
    13 Posts 3 Posters 8.0k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • jimpJ
      jimp Rebel Alliance Developer Netgate
      last edited by

      You can also use the wget method shown in the wiki:

      http://doc.pfsense.org/index.php/Remote_Config_Backup

      Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

      Need help fast? Netgate Global Support!

      Do not Chat/PM for help!

      1 Reply Last reply Reply Quote 0
      • F
        frater
        last edited by

        Thank you very much….

        I created this hourly cronjob on a remote server.
        It will delete the XML if it's the same as the previous one.
        This way you can immediately see when configs have changed.

        I did this before and it's very helpful.

        ln -s /usr/local/sbin/pfsense_getxml /etc/cron.hourly/

        /usr/local/sbin/pfsense_getxml

        
        #!/bin/sh
        
        DATESTAMP=`date +%Y-%m-%d.%H:%M`
        FNAME=pfsense.${DATESTAMP}.xml
        FOLDER=/var/www/vhosts/mydomain.com/pfsense
        
        USER=admin
        PASS=pfsense
        
        IP=80.232.169.117
        PORT=80
        
        if cd ${FOLDER} ; then
        
          FGROUP=`stat -c%G .`
          FUSER=`stat -c%U .`
        
          LASTXML=`ls -1t pfsense*xml 2>/dev/null | head -n1`
        
          if curl -u${USER}:${PASS} http://${IP}:${PORT}/zabbix.php 2>/dev/null | base64 -d 2>/dev/null >${FNAME} ; then
            chown ${FUSER}:${FGROUP} ${FNAME}
        
            if [ ! -z "${LASTXML}" ] ; then
              if [ ! "${LASTXML}" = "${FNAME}" ] ; then
                diff ${LASTXML} ${FNAME} >/dev/null && rm -f ${FNAME}
              fi
            fi
          else
            rm -f ${FNAME}
          fi
        else
          exit 1
        fi
        
        
        1 Reply Last reply Reply Quote 0
        • jimpJ
          jimp Rebel Alliance Developer Netgate
          last edited by

          If you go through that much trouble you may as well have it check the config into an SCM like git or svn. Then you can view the diffs, and you wouldn't have a bunch of redundant identical copies floating around.

          Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

          Need help fast? Netgate Global Support!

          Do not Chat/PM for help!

          1 Reply Last reply Reply Quote 0
          • F
            frater
            last edited by

            There are no redundant identical copies….

            1 Reply Last reply Reply Quote 0
            • jimpJ
              jimp Rebel Alliance Developer Netgate
              last edited by

              Ah, yeah I see the && rm now. Still seems a bit over-eager.

              On another note, I wouldn't want a non-password-protected page feeding up the config.xml file though, even protected by IP, but that's me.

              Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

              Need help fast? Netgate Global Support!

              Do not Chat/PM for help!

              1 Reply Last reply Reply Quote 0
              • marcellocM
                marcelloc
                last edited by

                @jimp:

                On another note, I wouldn't want a non-password-protected page feeding up the config.xml file though, even protected by IP, but that's me.

                I agree with you, it was just a fast example on how to do this.

                It's hard to decide between ip auth or leaving firewall password on a clear text script in zabix server not managed by firewall guys

                So a better example could be:

                
                #zabix server ip
                $zabix_ip='192.168.1.122';
                $password="some_password_to_secure_script";
                if ($_SERVER["REMOTE_ADDR"]==$zabix_ip && $_REQUEST['pass']== $password)
                  print base64_encode(file_get_contents('/conf/config.xml'));
                
                ?>
                
                

                Treinamentos de Elite: http://sys-squad.com

                Help a community developer! ;D

                1 Reply Last reply Reply Quote 0
                • jimpJ
                  jimp Rebel Alliance Developer Netgate
                  last edited by

                  Setup ssh keys and copy the config that way, no need to have passwords in plaintext anywhere. Whether you want to copy it to, or from, the firewalls is the only question there. Make the keys (without a passphrase), add them where you want, and cron a command to scp the config.

                  Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                  Need help fast? Netgate Global Support!

                  Do not Chat/PM for help!

                  1 Reply Last reply Reply Quote 0
                  • F
                    frater
                    last edited by

                    I understand the criticism and acknowledge the security risks…
                    If someone has root access to my remote server it would be a real problem...
                    Access to my pfsense config is not that trivial....

                    I did change the protocol from http to https and I'm also saving a diff for a quick and dirty changelog....

                    #!/bin/sh
                    
                    DATESTAMP=`date +%Y-%m-%d.%H:%M`
                    FNAME=pfsense.${DATESTAMP}.xml
                    FOLDER=/var/www/vhosts/mr-wolf.nl/pfsense
                    
                    USER=admin
                    PASS=pfsense
                    
                    PROTO=https             # http or https
                    IP=pfsense.yourdomain.com    # DNS or IP of webif (remote side)
                    PORT=443              # port of webif (remote side)
                    
                    if cd ${FOLDER} ; then
                    
                      FGROUP=`stat -c%G .`
                      FUSER=`stat -c%U .`
                      LASTXML=`ls -1t pfsense*xml 2>/dev/null | head -n1`
                    
                      if curl -u${USER}:${PASS} ${PROTO}://${IP}:${PORT}/zabbix.php 2>/dev/null | base64 -d 2>/dev/null >${FNAME} ; then
                        chown ${FUSER}:${FGROUP} ${FNAME}
                    
                        if [ ! -z "${LASTXML}" ] ; then
                          if [ ! "${LASTXML}" = "${FNAME}" ] ; then
                            if diff ${LASTXML} ${FNAME} >${FNAME}.diff ; then
                              rm -f ${FNAME}*
                            else
                              chown ${FUSER}:${FGROUP} ${FNAME}.diff
                            fi
                          fi
                        fi
                      else
                        rm -f ${FNAME}
                        exit 1
                      fi
                    else
                      exit 1
                    fi
                    
                    
                    1 Reply Last reply Reply Quote 0
                    • F
                      frater
                      last edited by

                      I somehow missed that wiki-entry or maybe it wasn't there when I started this thread.
                      Recently I replaced my pfsense machine and needed to recreate the little file on my pfsense.
                      But this isn't necessary if I would use the way it was described in the wiki.
                      Here's the code I'm using now.
                      It doesn't need a change for your pfsense

                      #!/bin/sh
                      
                      DATESTAMP=`date +%Y-%m-%d.%H:%M`
                      FNAME=pfsense.${DATESTAMP}.xml
                      FOLDER=/var/www/vhosts/yourdomain.com/pfsense
                      
                      USER=admin
                      PASS=pfsense
                      
                      PROTO=https             # http or https
                      IP=80.23.120.38         # DNS or IP of webif (remote side)
                      PORT=6443               # port of webif (remote side)
                      WGETOPT=
                      
                      # turn off certificate checking
                      [ "${PROTO}" = "https" ] && WGETOPT="${WGETOPT} --no-check-certificate"
                      
                      if cd ${FOLDER} ; then
                      
                        FGROUP=`stat -c%G .`
                        FUSER=`stat -c%U .`
                        LASTXML=`ls -1t pfsense*xml 2>/dev/null | head -n1`
                      
                        wget -qO/dev/null --keep-session-cookies --save-cookies /tmp/pfsense_cookies.txt  --post-data "login=Login&usernamefld=${USER}&passwordfld=${PASS}"  --no-check-certificate ${PROTO}://${IP}:${PORT}/diag_backup.php
                        wget -qO${FNAME} --keep-session-cookies --load-cookies /tmp/pfsense_cookies.txt  --post-data 'Submit=download&donotbackuprrd=yes' ${WGETOPT} ${PROTO}://${IP}:${PORT}/diag_backup.php
                      
                        if [ -s ${FNAME} ] ; then
                          chown ${FUSER}:${FGROUP} ${FNAME}
                      
                          if [ ! -z "${LASTXML}" ] ; then
                            if [ ! "${LASTXML}" = "${FNAME}" ] ; then
                              if diff ${LASTXML} ${FNAME} >${FNAME}.diff ; then
                                rm -f ${FNAME}*
                              else
                                chown ${FUSER}:${FGROUP} ${FNAME}.diff
                              fi
                            fi
                          fi
                        else
                          rm -f ${FNAME}
                          exit 1
                        fi
                      else
                        exit 1
                      fi
                      
                      
                      1 Reply Last reply Reply Quote 0
                      • marcellocM
                        marcelloc
                        last edited by

                        Frater,  hide you public IP address and pasword from your post.

                        Treinamentos de Elite: http://sys-squad.com

                        Help a community developer! ;D

                        1 Reply Last reply Reply Quote 0
                        • F
                          frater
                          last edited by

                          @marcelloc:

                          Frater,  hide you public IP address and pasword from your post.

                          Those were fake…
                          But thanks for your concern...

                          I can't edit my post, but I saw a little error in the first wget where I hardcoded the --no-check-certificate
                          That option is inside the variable "${WGETOPT}"

                            wget -qO/dev/null --keep-session-cookies --save-cookies /tmp/pfsense_cookies.txt  --post-data "login=Login&usernamefld=${USER}&passwordfld=${PASS}" ${WGETOPT} ${PROTO}://${IP}:${PORT}/diag_backup.php
                            wget -qO${FNAME} --keep-session-cookies --load-cookies /tmp/pfsense_cookies.txt  --post-data 'Submit=download&donotbackuprrd=yes' ${WGETOPT} ${PROTO}://${IP}:${PORT}/diag_backup.php
                          
                          

                          I don't know if anyone will be using it, but if it even helps only one man it was worth posting it.

                          1 Reply Last reply Reply Quote 0
                          • First post
                            Last post
                          Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.