Does this look dodgy?
-
I woke up this morning to find out I had no internet. I thought I'd switch on my monitor connected to my pfSense box and this was on screen. It was frozen at that so had to reset. Apparently the Drive is full too which I think is causing connection issues but this looked dodgy to me (keylogger?)
I'm doing this from a mobile so not sure if the image is accessible (might have to edit)I'm still trying to work out what type of files have filled the space but thought I'd drop this message in.
Anything I need to be worried about?
Thanks
-
@dal1980 said in Does this look dodgy?:
Apparently the Drive is full too
Probably the best way to blow up a system.
First check : console/SSH in, go to /var/log/ and sub directories.Btw : your pfSense version ?
Packages installed ? -
@gertjan thanks for replying
Apparently I have no packaged installed
"There are no packages currently installed."PfSense ver: 2.6.0-RELEASE
built on Mon Jan 31 19:57:53 UTC 2022
FreeBSD 12.3-STABLEI'll go check directories
-
Looks like lots of large log files
Do I just delete these or is there a gui option to clear these out?
I'll keep an eye on it in future
-
@dal1980 said in Does this look dodgy?:
Looks like lots of large log files
Those are not very big.. So unless you have a really small disk those shouldn't be the problem.
-
-
@dal1980 well your looking for files a lot bigger, or a lot more of them than that if you have 100G drive like your system shows.
Those are only a few MBs in total.
-
Sorry for all the images but I've got no internet and I have to remain connected to my pfSense network.
Unless that command doesn't do what I think it does there doesn't seem to be any large directories
-
@dal1980 said in Does this look dodgy?:
Apparently I have no packaged installed
But :
Doesn't look like 'pfSense' to me, I never saw that file.
It's a small file, that's not the issue.Btw : I presume you didn't asked for the right tools ^^
Let me help you : here you go : Google : pfsense check disk spaceA couple of seconds later you will find many solutions, like:
du -Pshx /*and you'll get a list with folders and their sizes.
Pick the biggest, probably a xx G bytes size folder, and repeat :
(Let's says it was /var/ that is huge) :du -Pshx /var/*etc.
You will wind up in the folder with many or huge or both files.
Now start backup these files (better get SFTP ready now, easy if you have SSH already set up).
And then it 'rm' time. -
Thanks. I googled a few ways to find file sizes but I don't understand what I'm looking for exactly since there isn't a directory that is that big. Here's two other ways (one of which you supplied thank you)
I also had a look in that file you mentioned.
I noticed the path /usr/share/bsdconfig/dialog.subr so went poking around there too... Seems there's a bunch of scripts in there wrote by Devin Taske 2006-2015. Could this be left over from a previous version
️ ? -
I'm now downloading everything over SFTP to see if I can find any large files that way (or find out where the many files live)
Update: looks like /Dev/ada0p3 is 3.79GB also looks like ada0p2 is still downloading at 10GB. What are these files? I just cancelled the /Dev folder downloads because I'm not sure what they are.
Update2: I also found them ada files in /var/dhcpd/dev/ folder too which I just cancelled.
Everything downloaded and it seems to be only them ada files that seem to be huge. I looked through everything else and found nothing of any real size.
-
/dev/ files are your devices like 'ada' is your hard drive. Don't 'touch' these.
Remember : everything for Unix and FreeBSD is files.Do also this : How to Run a pfSense Software File System Check
-
Thanks very much. That filechecker solved whatever was going on with the filesystem. Ran it 4 times for good measure, rebooted and the drive space went from 102GB/104GB used to 2GB/104GB used.
-
Humm. Good.
It's still on my 'need to understand that ones and for all' list : the disk system is marked a full, but no big or many files that occupy the place.
It's probably an 'inode' thing, like files are de allocated, but the free clusters (inodes ?) are not given back to the free drive space pool. Over time this creates what you just experienced.
That why the Netgate video exists. It's even not a pfSense thing, it's a FreeBSD thing.
The file system is pretty resilient, but people still tend to 'remove the power cord' without shutting down the system first (do that with your PC x times, and you will loose your PC (disk) !).
That's also why on newer systems pfSense prefers now the ZFS as a file system. -
@dal1980 said in Does this look dodgy?:
Ran it 4 times for good measure
Good. In TAC we recommend 5-10. Thankfully you can queue them up easily with the up arrow and enter.
-
@gertjan said in Does this look dodgy?:
Over time this creates what you just experienced.
{professional aside}
This is a similar thing to what happens when you only delete photos from a SD/CF card and never re-format. In order to spare the writes across the system it just marks the formerly used blocks as... well, still used but doesn't move the write point because of fragmentation. Seen this many times with department photogs in my last job because my supervisor told them never to format, but then the card stopped working because they ran out of the next set of available blocks.Similar things happen with APFS on the latest releases of macOS but eventually the system catches up and shuffles the data it needs to and 'frees' up the missing space for writing.
-
Thanks guys.
I normally shutdown properly but had an issue with power loss at home which was pretty regular. I have a UPS but it's just my Unraid that's plugged into that but I'll look at getting my pfSense hooked up to it too maybe. Power cuts were down to my Lava Lamp as it happens which I've now stopped using anyway.
Interesting stuff though.
