Crash report or programming bug ...
-
Hi,
I was trying to backup the whole pfsense installation as the xml backup works fine but every-time I have had to use it (3 times in about as many years) it never reinstalls all the packages properly (snort) and I always need to remove locked processes and reinstall manually and the manually reinstall always has issues which eventually leads me to reinstall pfsense. Which I will have to do this time also. The only good thing about this is my config is fairly easy compared to some.
I mean PFSense is awesome, but it constantly needs to be reinstalled after upgrading and/or significant configuration changes. AND, maybe that's all my fault---(still not convinced of this, I am able to backup/restore operating systems using various backup/recovery tools but pfsense? haven't figured it out yet)---- I admit it..however, IF there were a decent way to backup the whole installation, people like me would have a MUCH easier time of it.
I mean the backup systems currently in place for PFSense seem so draconian. Save the xml, save directories individually, restore the xml, pfsense attempts to but fails to reinstall the packages, delete locks, manual reinstall, which messes up, then reinstall the whole shebang and start over. I have a pfsense installed on a 128GB ssd. I bought that ssd for $28.00 Canadian. It would be 10-15% cheaper in the US. I say there is literally no one person who can't afford to spend $25 on an ssd for pfsense. I have 5 spares for when they eventually die.
Would it be so hard to have pfsense create a backup partition as large as the user wants and regularly do full/incremental/partial backups? As per user configuration. I mean who wants to bring down their router so they can boot from a USB boot stick to use Macrium Reflect to make a backup to a network share when the drive space I am using for pfsense is mostly wasting away with nothing on it?
I did a df -h on the drive and I am using 2.4GB and have 94GB free. I could literally store 20-30 full backups and STILL have enough free space...lol. IF I WERE a programmer I would have already done this as an add-on package, but I am not a programmer so all I can do is whine and complain until the devs get tired of it and make one for all us poor incompetent users:-)
OR does anyone else have any input about an "unknown" package I can install to do it? I mean at this point PFSense is being reinstalled every year or so anyways, if I have to reinstall 50 times to ensure I can do a good backup of the whole installation and then do regular backups it will be worth it in the end.
Thanks a bunch all.
-
@1OF1000Quadrillion said in Crash report or programming bug ...:
I always need to remove locked processes
You can name the issue : a package that 'misbehaves' ? I know what you mean, but never saw 'blocked' pkg instances myself.
You are using a VPN (client) that isn't set up yet yet when you install from scratch, or pfSense is forced to use it ?One of the main reasons I use pfSense that, if needed, I can can re install from scratch.
True, I'm not re installing pfSense every year - the last setup dates from .... 2016 ? (I migrated to a recent desktop device this year, stuffing it up with the quand NIC and the SSD from the previous device).
I do use some FreeBSD packages like munin that I have to install and set up manually.
I do use two 'big' pfSense packages : FreeRadius and pfBlockerNg-devel.
I don't feel the need to 'clone' my actual working setup for the in-case-of-reason. I have my actual config.xml - that's why it exists.For me, purely disk imaging exists for me because I'm more then fed up with re installing another exploded Windows device. Not for my router ...
If there was an issue, the install going bad, I would restart, set up a working WAN, then I add manually - click on them - on all the 'big' packages, without setting them up.
When all packages are there, I re import the config.xml as a whole (I guess this will not force a reload of all packages ? - I should have to try this). Their settings will get set, they will get activated as needed, etc.I would temporary stop using packages that block install / update / whatever process.
I would check the forum ones in a while to see if there were know issues before doing major steps..@1OF1000Quadrillion said in Crash report or programming bug ...:
IF there were a decent way to backup the whole installation
This lists some possibilities :
http://www.wonkity.com/~wblock/docs/html/backup.html
I guess none of them are really click-and-pray.
-
As another data point, for years I had the same piece of hardware for pfSense. It was a 1U bare-bones server chassis from Newegg that had two onboard Realtek NICs (was not thrilled with those, but at the time they worked) and I added an Intel card for a DMZ. It contained 4 GB of RAM and an Intel CPU.
I ran Snort and the apcupsd packages on that system. I installed pfSense 2.0 Release Candidate on it and then upgraded pfSense on that hardware all the way up to 2.4.3 or so. I recently switched over to an SG-5100 appliance. During all that time I never had to reinstall pfSense and never had either package fail on me during the upgrade.
If you have had issues with every single upgrade of pfSense, then you really need to examine your process. Something is amiss on your end. There have been issues reported with one package that I remember where it would hang up
pkg, but that is only for the latest 2.4.5 release branch.You state that you use some other FreeBSD packages such as munin. That could be your problem. If you are not getting those from the pfSense packages repository, then those packages may be dragging in different versions of shared libraries when you install them, and that can cause all kinds of grief on an upgrade.
There are thousands and thousands of pfSense installations worldwide. If all of them were reporting the same issue as you, then yeah that would point to a fundamental pfSense problem. However, there are few if any similar posts here. That would indicate the software is essentially solid.
-
HHHHmmmmm, now that I read your replies and have read a bit more on rules/flow-bits I think my issue is mostly impatience and ignorance. gertjan said , " I reinstall pfsense and setup a wan, then reinstall all the pkgs I was using without configuring any of them, and after that I restore the config.xml" which re-configures the packages to the way he had it before the reinstall.
See, I was using the restore config.xml file as a way to resolve issues that crop up for whatever reason. I had an issue I couldn't figure out and so thought, "I'll just go back to an older, known working, config". Usually the issue at hand was something to do with snort and probably because I do not understand yet the effects of disabling some rules while leaving others enabled and how that may affect the flow-bits.
I did some more reading on that subject last night; the effect of flow-bits when rules are disabled. I have been manually enabling and disabling rules I believe I need or don't need willy, nilly based on the rule name. For example, I enabled web-server rules on one interface because I have a web srv on that interface.
I have been getting S5 stream errors mostly when the issues start (for the past 3 times I've needed to reinstall it has all begun with S5 stream errors) and THAT (correct me if I am wrong please) may be caused by me enabling/disabling rules without understanding the actual effects?
So, for now, I set the balanced IPS policy on the WAN and one of the LAN interfaces - the interface all my clients are on. I have another LAN interface that only has srvs on it, I have set the IPS policy to Security on that interface. All my servers seem to be working as expected and the latest system log I had emailed to me did NOT contain any mention of S5 stream errors.
Other than snort I have only added available packages from the pfsense package manager. They are arpwatch, backup, mail-report,mtr-nox11,nmap,notes, ntopng, pfblockerng and snort. All official packages. I might add unofficial packages to my Linux box because I would know how to deal with the potential fallout if problems arise, I would not add unofficial packages to snort without first cloning the whole drive.
-
There is a bug in pkg that you may be hitting in 2.4.5p1 where the pkg process never closes preventing subsequent packages installing after a restore. Only some packages hit it, notably Squid and FRR may. You can get past it by either killing and package process that has frozen or making a change in the package settings. It should then continue to install other packages.
https://redmine.pfsense.org/issues/10610
It's fixed in 2.5.I restore stuff all the time and only occasionally hit that though.
If you want complete filesystem backups consider installing ZFS and using snapshots.
https://www.freebsd.org/doc/handbook/zfs-zfs.html#zfs-zfs-snapshot
Not a GUI option, yet.Steve
