Leak causes full filesystem - how to identify?

keyser

My SG-2100 pfSense 21.05 started to exibit a strange problem after upgrading to 21.05 and installing Zabbix Agent 5

The 7Gb SSD contains the “/“ Filesystem, and “/“ is about 25% filled @ boot (2.0Gb Data)
This is the same level as before the upgrade.

After about 2 weeks the filesystem hits 90% full without me being able to identify which files/folders are the culprit.

A df -h shows “/“ as 90% full, but a du -h still only shows 2.0Gb data in the filesystem (same as @ boot). So no files or folder are reported to use the remaining 4.2Gb up to the 90% full marker.

If I reboot pfSense, the filesystem is back to 25% full and the process starts over.

How do I identify which package/daemon/service is causing the filesystem to report running full?
Why does the space get released by rebooting?

bingo600

@keyser

Could it be swap growing ...
Guess ... Might not count in du

/Bingo

provels

@keyser

du -sh /var/*

give you anything of size?

keyser

@bingo600 No, Swap is not enabled, and there is plenty of free memory as well.

keyser

@provels No, that’s the strange part. No directories reports remarkable “du”. Files and folders combined across all of “/“ (including /var) reports 2.0Gb in “du” while the filesystem is reported 90% full (6.4Gb used)

keyser

@provels Is there any command line tools that can assist in identifiying open and growing files - or perhaps a tool that can report which files are seeing writes being done?

keyser

@keyser @bingo600 After some additional digging it seems it’s not related to Zabbix but rather unbound resolver in combination with pfblockerNG-devel 3.0.16

I started suspecting unbound because “top -SH” in I/O mode (press m) showed that unbound constantly was doing disk I/O

I’m investigating further for now, but stopping pfblockerNG (which stops and reconfigures unbound) releases the allocated diskspace which then returns to the 25% it should be.

Maybe it’s something related to the new python integration i pfblockerNG and Unbound.

The Issue must have arisen when I upgraded to 21.05 from 21.02

I’ll close this thread and create a new one under the pfBlockerNG forum.