Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    cURL backup not working anymore

    General pfSense Questions
    2
    5
    403
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • C
      CPM 0
      last edited by CPM 0

      I used to backup my pfSense config to a Synology NAS. The script that is was using is this one: pfSense Autobackup.

      That script stopped working. It didn't download the config anymore and was complaining about: ngix 301 Moved Permanently.

      Then i was trying to use the official cURL commands from the Netgate documentation:

      
      *curl -L -k --cookie-jar cookies.txt \
           https://192.168.2.2/ \
           | grep "name='__csrf_magic'" \
           | sed 's/.*value="\(.*\)".*/\1/' > csrf.txt
      curl -L -k --cookie cookies.txt --cookie-jar cookies.txt \
           --data-urlencode "login=Login" \
           --data-urlencode "usernamefld=BackupUser" \
           --data-urlencode "passwordfld=BackupPassword" \
           --data-urlencode "__csrf_magic=$(cat csrf.txt)" \
           https://192.168.2.2/ > /dev/null
      curl -L -k --cookie cookies.txt --cookie-jar cookies.txt \
           https://192.168.2.2/diag_backup.php  \
           | grep "name='__csrf_magic'"   \
           | sed 's/.*value="\(.*\)".*/\1/' > csrf.txt
      curl -L -k --cookie cookies.txt --cookie-jar cookies.txt \
           --data-urlencode "download=download" \
           --data-urlencode "donotbackuprrd=yes" \
           --data-urlencode "__csrf_magic=$(head -n 1 csrf.txt)" \
           https://192.168.2.2/diag_backup.php > config-router-`date +%Y%m%d%H%M%S`.xml*
      

      It creates csrf.txt file on the Synology.
      It creates cookies.txt file on the Synology.
      It creates the config file: config-router-20240206201526.xml

      But when i open the file it doesn't contain the config. When opening that file with a webbrowser gives the following:

      This page contains the following errors:
      error on line 9 at column 7: StartTag: invalid element name
      Below is a rendering of the page up to the first error.
      pfSense - Login //

      Looking into the .xml files (first lines)(SID removed)(dont know if a full post here is safe?)

      <!DOCTYPE html>
      <html lang="en">
      	<head>
      		<meta name="viewport" content="width=device-width, initial-scale=1">
      	    <link rel="stylesheet" href="/vendor/bootstrap/css/bootstrap.min.css" type="text/css">
      	    <link rel="stylesheet" href="/css/login.css?v=1701893362" type="text/css">
      		<title>pfSense - Login</title>
      		<script type="text/javascript">
      			//<![CDATA{
      			var events = events || [];
      			//]]>
      		</script>
      	<script type="text/javascript">if (top != self) {top.location.href = self.location.href;}</script><script type="text/javascript">var csrfMagicToken = "sid:SID REMOVED BY ME";var csrfMagicName = "__csrf_magic";</script><script src="/csrf/csrf-magic.js" type="text/javascript"></script></head>
      
      

      Somebody can help me out? If a full post of the .xml is needed please tell me what to delete to post is safe here.
      (sorry for my bad English)

      1 Reply Last reply Reply Quote 0
      • stephenw10S
        stephenw10 Netgate Administrator
        last edited by

        Hmm, so it's just hitting the login page still.

        What version did it last work in?

        You might try just using SCP or one of the other methods shown here:
        https://docs.netgate.com/pfsense/en/latest/backup/remote-backup.html

        Steve

        C 1 Reply Last reply Reply Quote 0
        • C
          CPM 0 @stephenw10
          last edited by CPM 0

          @stephenw10

          I don't exactly know exactly what version still worked. I migrated from the free version to the lastest CE edition. I noticed it recently that the config.xml was empty. Couldn't track it back because configs older then 30days where deleted. I think it was around oktober that I implemented this and around December is stopped working.
          Both ways look like a similar issue, webpage actions aren't applied correct.

          This is an issue easily to reproduce, so maybe someone can test this for me?

          1 Reply Last reply Reply Quote 0
          • stephenw10S
            stephenw10 Netgate Administrator
            last edited by

            It works as expected for me against 23.09.1:

            steve@steve-NUC9i9QNX:~$ curl -L -k --cookie-jar cookies.txt \
                 https://172.21.16.1/ \
                 | grep "name='__csrf_magic'" \
                 | sed 's/.*value="\(.*\)".*/\1/' > csrf.txt
              % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                             Dload  Upload   Total   Spent    Left  Speed
            100 10979    0 10979    0     0   178k      0 --:--:-- --:--:-- --:--:--  178k
            
            steve@steve-NUC9i9QNX:~$ curl -L -k --cookie cookies.txt --cookie-jar cookies.txt \
                 --data-urlencode "login=Login" \
                 --data-urlencode "usernamefld=admin" \
                 --data-urlencode "passwordfld=xxxxxxxx" \
                 --data-urlencode "__csrf_magic=$(cat csrf.txt)" \
                 https://172.21.16.1/ > /dev/null
              % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                             Dload  Upload   Total   Spent    Left  Speed
            100   181    0     0  100   181      0    615 --:--:-- --:--:-- --:--:--   617
            100  155k    0  155k    0     0  13942      0 --:--:--  0:00:11 --:--:-- 38801
            
            steve@steve-NUC9i9QNX:~$ curl -L -k --cookie cookies.txt --cookie-jar cookies.txt \
                 https://172.21.16.1/diag_backup.php  \
                 | grep "name='__csrf_magic'"   \
                 | sed 's/.*value="\(.*\)".*/\1/' > csrf.txt
              % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                             Dload  Upload   Total   Spent    Left  Speed
            100 32889    0 32889    0     0  72703      0 --:--:-- --:--:-- --:--:-- 72602
            
            steve@steve-NUC9i9QNX:~$ curl -L -k --cookie cookies.txt --cookie-jar cookies.txt \
                 --data-urlencode "download=download" \
                 --data-urlencode "donotbackuprrd=yes" \
                 --data-urlencode "__csrf_magic=$(head -n 1 csrf.txt)" \
                 https://172.21.16.1/diag_backup.php > config-router-`date +%Y%m%d%H%M%S`.xml
              % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                             Dload  Upload   Total   Spent    Left  Speed
            100  335k  100  335k  100   109  2164k    702 --:--:-- --:--:-- --:--:-- 2180k
            
            steve@steve-NUC9i9QNX:~$ ls -ls config-router*
            336 -rw-rw-r-- 1 steve steve 343809 Feb  7 14:07 config-router-20240207140749.xml
            

            The resulting file is the backup config from that firewall.

            Has your backup user expired maybe? Or somehow no longer has permissions to access that page? Does it work if you just use the admin user?

            Steve

            C 1 Reply Last reply Reply Quote 1
            • C
              CPM 0 @stephenw10
              last edited by CPM 0

              @stephenw10

              Thanks for testing and the support. I found the problem in the "unofficial" script. Somehow only using the ip-adress wasn't working anymore. Adding https to it fixed it.
              Sometimes the solution is simple but the error was misleading.
              This case is closed.
              Thanks again.

              1 Reply Last reply Reply Quote 1
              • First post
                Last post
              Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.