Who Maintains this Package/How Do I File a Bug Report
-
I had some problems with the backup package becuase I tried to backup /root. The problem is the backup package stores it's data in /root/backup, and the tar statement used doesn't exclude the backup directory from the tar, so it creates a recursive archive operation that crashes pfSense.
From what I can see it just needs a simple addition to the tar statement to exclude {$backup_path} on line 57 of:.
$backup_cmd = "/usr/bin/tar --create --verbose --gzip --file {$backup_path} --exclude {$backup_path} --directory / ";It's a simple change, but I know nothing about git and I don't have a system to test any change I might make.
The link to the package is here:
https://github.com/pfsense/FreeBSD-ports/commits/devel/sysutils/pfSense-pkg-BackupAny assistance in getting this patched would be much appreciated.
If it helps, here's a copy of the file with the change applied:
<?php /* * backup.php * * part of pfSense (https://www.pfsense.org) * Copyright (c) 2015-2020 Rubicon Communications, LLC (Netgate) * Copyright (c) 2008 Mark J Crane * All rights reserved. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * * http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. */ require_once("guiconfig.inc"); require_once("/usr/local/pkg/backup.inc"); global $config, $backup_dir, $backup_filename, $backup_path; if (!is_array($config['installedpackages']['backup'])) { $config['installedpackages']['backup'] = array(); } if (!is_array($config['installedpackages']['backup']['config'])) { $config['installedpackages']['backup']['config'] = array(); } $a_backup = &$config['installedpackages']['backup']['config']; $backup_dir = "/root/backup"; $backup_filename = "pfsense.bak.tgz"; $backup_path = "{$backup_dir}/{$backup_filename}"; if ($_GET['act'] == "del") { if ($_GET['type'] == 'backup') { if ($a_backup[$_GET['id']]) { unset($a_backup[$_GET['id']]); write_config(); header("Location: backup.php"); exit; } } } if ($_GET['a'] == "download") { if ($_GET['t'] == "backup") { $i = 0; if (count($a_backup) > 0) { /* Do NOT remove the trailing space after / from $backup_cmd below!!! */ $backup_cmd = "/usr/bin/tar --create --verbose --gzip --file {$backup_path} --exclude {$backup_path} --directory / "; foreach ($a_backup as $ent) { if ($ent['enabled'] == "true") { $backup_cmd .= escapeshellarg($ent['path']) . ' '; } $i++; } system($backup_cmd); } session_cache_limiter('public'); $fd = fopen("{$backup_path}", "rb"); header("Content-Type: application/force-download"); header("Content-Type: binary/octet-stream"); header("Content-Type: application/download"); header("Content-Description: File Transfer"); header('Content-Disposition: attachment; filename="' . $backup_filename . '"'); header("Cache-Control: no-cache, must-revalidate"); header("Expires: Sat, 26 Jul 1997 05:00:00 GMT"); header("Content-Length: " . filesize($backup_path)); fpassthru($fd); exit; } } if ($_GET['a'] == "other") { if ($_GET['t'] == "restore") { // Extract the tgz file if (file_exists($backup_path)) { system("/usr/bin/tar -xpzC / -f {$backup_path}"); header("Location: backup.php?savemsg=Backup+has+been+restored."); } else { header("Location: backup.php?savemsg=Restore+failed.+Backup+file+not+found."); } exit; } } if (($_POST['submit'] == "Upload") && is_uploaded_file($_FILES['ulfile']['tmp_name'])) { move_uploaded_file($_FILES['ulfile']['tmp_name'], "{$backup_path}"); $savemsg = "Uploaded file to {$backup_dir}" . htmlentities($_FILES['ulfile']['name']); system("/usr/bin/tar -xpzC / -f {$backup_path}"); } $pgtitle = array(gettext("Diagnostics"), gettext("Backup Files and Directories"), gettext("Settings")); include("head.inc"); if ($_GET["savemsg"]) { print_info_box($_GET["savemsg"]); } $tab_array = array(); $tab_array[] = array(gettext("Settings"), true, "/packages/backup/backup.php"); $tab_array[] = array(gettext("Add"), false, "/packages/backup/backup_edit.php"); display_top_tabs($tab_array); ?> <div class="panel panel-default"> <div class="panel-heading"><h2 class="panel-title">Backups</h2></div> <div class="panel-body"> <div class="table-responsive"> <table class="table table-hover"> <tr> <td>Use this to tool to backup files and directories. The following directories are recommended for backup: <table> <tr><td><strong>pfSense Config:</strong></td><td>/cf/conf</td></tr> <tr><td><strong>RRD Graph Data Files:</strong></td><td>/var/db/rrd</td></tr> </table> </td> </tr> </table> </div> </div> <div class="panel-heading"><h2 class="panel-title">Upload Archive</h2></div> <div class="panel-body"> <div class="table-responsive"> <form action="backup.php" method="post" enctype="multipart/form-data" name="frmUpload" onsubmit=""> <table class="table table-hover"> <tr> <td colspan="2"> Restore a backup by selecting the backup archive and clicking <strong>Upload</strong>. </td> </tr> <tr> <td>File to upload:</td> <td> <input name="ulfile" type="file" class="btn btn-info" id="ulfile" /> <br /> <button name="submit" type="submit" class="btn btn-primary" id="upload" value="Upload"> <i class="fa fa-upload icon-embed-btn"></i> Upload </button> </td> </tr> </table> </form> </div> </div> <div class="panel-heading"><h2 class="panel-title">Backup and Restore</h2></div> <div class="panel-body"> <div class="table-responsive"> <form action="backup.php" method="post" enctype="multipart/form-data" name="frmUpload" onsubmit=""> <table class="table table-hover"> <tr> <td> The 'Backup' button compresses the directories that are listed below to /root/backup/pfsense.bak.tgz; after that it presents the file for download.<br /> If the backup file does not exist in /root/backup/pfsense.bak.tgz then the 'Restore' button will be hidden. </td> </tr> <tr> <td> <button type='button' class="btn btn-primary" value='Backup' onclick="document.location.href='backup.php?a=download&t=backup';"> <i class="fa fa-download icon-embed-btn"></i> Backup </button> <?php if (file_exists($backup_path)) { ?> <button type="button" class="btn btn-warning" value="Restore" onclick="document.location.href='backup.php?a=other&t=restore';"> <i class="fa fa-undo icon-embed-btn"></i> Restore </button> <?php } ?> </td> </tr> </table> </form> </div> </div> <div class="panel-heading"><h2 class="panel-title">Backup Locations</h2></div> <div class="panel-body"> <div class="table-responsive"> <form action="backup_edit.php" method="post" name="iform" id="iform"> <table class="table table-striped table-hover table-condensed"> <thead> <tr> <td width="20%">Name</td> <td width="25%">Path</td> <td width="5%">Enabled</td> <td width="40%">Description</td> <td width="10%">Actions</td> </tr> </thead> <tbody> <?php $i = 0; if (count($a_backup) > 0): foreach ($a_backup as $ent): ?> <tr> <td><?=$ent['name']?> </td> <td><?=$ent['path']?> </td> <td><? echo ($ent['enabled'] == "true") ? "Enabled" : "Disabled";?> </td> <td><?=htmlspecialchars($ent['description'])?> </td> <td> <a href="backup_edit.php?id=<?=$i?>"><i class="fa fa-pencil" alt="edit"></i></a> <a href="backup_edit.php?type=backup&act=del&id=<?=$i?>"><i class="fa fa-trash" alt="delete"></i></a> </td> </tr> <? $i++; endforeach; endif; ?> <tr> <td colspan="5"></td> <td> <a class="btn btn-small btn-success" href="backup_edit.php"><i class="fa fa-plus" alt="add"></i> Add</a> </td> </tr> </tbody> </form> </div> </div> </div> <?php include("foot.inc"); ?> -
Open a bug report here: https://redmine.pfsense.org/projects/pfsense-packages
Include steps to replicate it. Link back to here if you like.
It's pretty easy to make a pull request for a one line change like that. You can do it all from inside the github web interface.
Steve
-
@stephenw10 said in Who Maintains this Package/How Do I File a Bug Report:
Open a bug report here: https://redmine.pfsense.org/projects/pfsense-packages
Include steps to replicate it. Link back to here if you like.
It's pretty easy to make a pull request for a one line change like that. You can do it all from inside the github web interface.
Steve
OK, I took my best shot at creating the bug report:
https://redmine.pfsense.org/issues/11098
and I attached my best attempt at a fix (only change is line 57)Now what?
Sorry I know nothing about github, and I don't have a non-production system for testing, so my changes hasn't even been syntax checked.
-
That bug report looks fine. jimp has corrected the project and category.
Now we, or someone else, verify the issue and test the fix. Status changes to 'confirmed'.
Then create an actual patch and file a pull request in github (or our internal gitlab server).
That gets reviewed and if no changes are required it gets pulled in and a new package is built.
That becomes available on the package repo.
This is not a hardware issue so you should be able to test it pretty easily in a VM.
Steve
-
I can't replicate this. How exactly did you configure it?
I'm testing it with:
And it completes OK. The backup file is omitted.
Steve
-
Mmm, tried a bunch of things here and can't reproduce. The backup file is omitted from the backup every time.
What exactly was the error you saw here?
Why do you think it's trying to recursively backup itself?
Steve
-
@stephenw10 said in Who Maintains this Package/How Do I File a Bug Report:
Mmm, tried a bunch of things here and can't reproduce. The backup file is omitted from the backup every time.
What exactly was the error you saw here?
Why do you think it's trying to recursively backup itself?
Steve
Your configuration is correct. I experienced this problem a while back (2.3 or 2.4), and gave up on using the package. When I ran it with /root, it generated a infinite size tar that only stopped when all space was gone.
I didn't see any sign of changes so I just assumed the problem still existed. As I mentioned earlier, I have no way to test other than my production firewall.
Maybe I had/have had a recursive link that caused the problem. I'm so very sorry for wasting your time, it appears that you should close the case.
-
Ah, OK. Then, yeah, test the current package version in 2.4.5p1 if you can.
Excluding the backup file specifically doesn't seem like a bad idea though. The pull request is already in so it will likely be included. You may have hit some edge case that this will prevent.
Steve
-
@stephenw10 said in Who Maintains this Package/How Do I File a Bug Report:
Ah, OK. Then, yeah, test the current package version in 2.4.5p1 if you can.
Excluding the backup file specifically doesn't seem like a bad idea though. The pull request is already in so it will likely be included. You may have hit some edge case that this will prevent.
Steve
Thanks Steve, I'll give it a try when I get a window that I can afford to have the firewall crash. I was wondering if it would be safe to kill the tar job if I run into a problem?
I don't have much experience with FreeBSD other than what little bit I have to do at the command line with pfSense/FreeNAS. As a developer do you know if tar has built in protection to prevent a recursive backup?
-
If it gets stuck creating an ever larger file then killing the tar job should be safe enough. The alternative is the not kill it and it will fill the drive with all the problems that brings!
I always test in a VM first. Restoring a snapshot is very easy.
I cannot tell you what prevents it including that file currently, just that it didn't include it in any test that we did. It does fail if you try to backup
/(the entire filesystem) but it fails because you hit the 512MB limit on the php process running it. My own coding skills are....limited!But as I say adding that exclude switch seems like a good idea anyway. I can only help if there is some edge case that would otherwise cause recursion.
Steve
-
@stephenw10 said in Who Maintains this Package/How Do I File a Bug Report:
If it gets stuck creating an aver larger file then killing the tar job should be safe enough. The alternative is the not kill it and it will fill the drive with all the problems that brings!
I always test in a VM first. Restoring a snapshot is very easy.
I cannot tell you what prevents it including that file currently, just that it didn't include it in any test that we did. It does fail if you try to backup
/(the entire filesystem) but it fails because you hit the 512MB limit on the php process running it. My own coding skills are....limited!But as I say adding that exclude switch seems like a good idea anyway. I can only help if there is some edge case that would otherwise cause recursion.
Steve
Looking at the code, unless tar has built-in recursion protection (which is possible), I don't understand why it isn't causing problems.VM is clearly the way to go, I'm just not in a position to set one up now.
-
VBox on a desktop works well for a test like this. I used it for years until I got Proxmox setup.
Steve
