cURL backup not working anymore
-
I used to backup my pfSense config to a Synology NAS. The script that is was using is this one: pfSense Autobackup.
That script stopped working. It didn't download the config anymore and was complaining about: ngix 301 Moved Permanently.
Then i was trying to use the official cURL commands from the Netgate documentation:
*curl -L -k --cookie-jar cookies.txt \ https://192.168.2.2/ \ | grep "name='__csrf_magic'" \ | sed 's/.*value="\(.*\)".*/\1/' > csrf.txt curl -L -k --cookie cookies.txt --cookie-jar cookies.txt \ --data-urlencode "login=Login" \ --data-urlencode "usernamefld=BackupUser" \ --data-urlencode "passwordfld=BackupPassword" \ --data-urlencode "__csrf_magic=$(cat csrf.txt)" \ https://192.168.2.2/ > /dev/null curl -L -k --cookie cookies.txt --cookie-jar cookies.txt \ https://192.168.2.2/diag_backup.php \ | grep "name='__csrf_magic'" \ | sed 's/.*value="\(.*\)".*/\1/' > csrf.txt curl -L -k --cookie cookies.txt --cookie-jar cookies.txt \ --data-urlencode "download=download" \ --data-urlencode "donotbackuprrd=yes" \ --data-urlencode "__csrf_magic=$(head -n 1 csrf.txt)" \ https://192.168.2.2/diag_backup.php > config-router-`date +%Y%m%d%H%M%S`.xml*It creates csrf.txt file on the Synology.
It creates cookies.txt file on the Synology.
It creates the config file: config-router-20240206201526.xmlBut when i open the file it doesn't contain the config. When opening that file with a webbrowser gives the following:
This page contains the following errors:
error on line 9 at column 7: StartTag: invalid element name
Below is a rendering of the page up to the first error.
pfSense - Login //Looking into the .xml files (first lines)(SID removed)(dont know if a full post here is safe?)
<!DOCTYPE html> <html lang="en"> <head> <meta name="viewport" content="width=device-width, initial-scale=1"> <link rel="stylesheet" href="/vendor/bootstrap/css/bootstrap.min.css" type="text/css"> <link rel="stylesheet" href="/css/login.css?v=1701893362" type="text/css"> <title>pfSense - Login</title> <script type="text/javascript"> //<![CDATA{ var events = events || []; //]]> </script> <script type="text/javascript">if (top != self) {top.location.href = self.location.href;}</script><script type="text/javascript">var csrfMagicToken = "sid:SID REMOVED BY ME";var csrfMagicName = "__csrf_magic";</script><script src="/csrf/csrf-magic.js" type="text/javascript"></script></head>Somebody can help me out? If a full post of the .xml is needed please tell me what to delete to post is safe here.
(sorry for my bad English) -
Hmm, so it's just hitting the login page still.
What version did it last work in?
You might try just using SCP or one of the other methods shown here:
https://docs.netgate.com/pfsense/en/latest/backup/remote-backup.htmlSteve
-
I don't exactly know exactly what version still worked. I migrated from the free version to the lastest CE edition. I noticed it recently that the config.xml was empty. Couldn't track it back because configs older then 30days where deleted. I think it was around oktober that I implemented this and around December is stopped working.
Both ways look like a similar issue, webpage actions aren't applied correct.This is an issue easily to reproduce, so maybe someone can test this for me?
-
It works as expected for me against 23.09.1:
steve@steve-NUC9i9QNX:~$ curl -L -k --cookie-jar cookies.txt \ https://172.21.16.1/ \ | grep "name='__csrf_magic'" \ | sed 's/.*value="\(.*\)".*/\1/' > csrf.txt % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 10979 0 10979 0 0 178k 0 --:--:-- --:--:-- --:--:-- 178k steve@steve-NUC9i9QNX:~$ curl -L -k --cookie cookies.txt --cookie-jar cookies.txt \ --data-urlencode "login=Login" \ --data-urlencode "usernamefld=admin" \ --data-urlencode "passwordfld=xxxxxxxx" \ --data-urlencode "__csrf_magic=$(cat csrf.txt)" \ https://172.21.16.1/ > /dev/null % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 181 0 0 100 181 0 615 --:--:-- --:--:-- --:--:-- 617 100 155k 0 155k 0 0 13942 0 --:--:-- 0:00:11 --:--:-- 38801 steve@steve-NUC9i9QNX:~$ curl -L -k --cookie cookies.txt --cookie-jar cookies.txt \ https://172.21.16.1/diag_backup.php \ | grep "name='__csrf_magic'" \ | sed 's/.*value="\(.*\)".*/\1/' > csrf.txt % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 32889 0 32889 0 0 72703 0 --:--:-- --:--:-- --:--:-- 72602 steve@steve-NUC9i9QNX:~$ curl -L -k --cookie cookies.txt --cookie-jar cookies.txt \ --data-urlencode "download=download" \ --data-urlencode "donotbackuprrd=yes" \ --data-urlencode "__csrf_magic=$(head -n 1 csrf.txt)" \ https://172.21.16.1/diag_backup.php > config-router-`date +%Y%m%d%H%M%S`.xml % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 335k 100 335k 100 109 2164k 702 --:--:-- --:--:-- --:--:-- 2180k steve@steve-NUC9i9QNX:~$ ls -ls config-router* 336 -rw-rw-r-- 1 steve steve 343809 Feb 7 14:07 config-router-20240207140749.xmlThe resulting file is the backup config from that firewall.
Has your backup user expired maybe? Or somehow no longer has permissions to access that page? Does it work if you just use the admin user?
Steve
-
Thanks for testing and the support. I found the problem in the "unofficial" script. Somehow only using the ip-adress wasn't working anymore. Adding https to it fixed it.
Sometimes the solution is simple but the error was misleading.
This case is closed.
Thanks again.
