Free up space, disk storage >80%

[Gertjan]

@RobbieTT said in Free Disk space >80%:

I'm surprised how much space pfblockerNG took

I presume this happens out there in the wild :
People install pfBlockerng, and start to try out all these : Firewall > pfBlockerNG > Feeds because "why not ?".
Soon, they'll discover that there is something as a resource limit, and not only disk space, but also that Xeon 8 core processor that goes hot read. An ARM won't make it at all ....

All these 'feed' files initially downloaded, and their parsed and assembled counter parts will stay in the sub folders of this folder path /var/db/pfblockerng for until the end of days .... (or pfSense re install) or manual cleaning, as I don't think the /var/db/pfblockerng/. is emptied upon package removal. [ I have to try that one ... ]
Still, pfBlockerng is a small example. Processes (programs) that auto generate content are a real admin's nightmare - you have to watch them as you would do with babies. Even on big-iron-servers with loads of disk space (multiple T's) I do graph constantly free inodes and free disk blocks (disk space). Here is the same thing for my pfSense.

File rotating for log files is the most straightforward example : as soon as my apache2 servers (or mail server) get swamped by requests, log files start to grow rapidly - like very fast. By simply launching multiple requests the server will steadily fill up its disk space, and then the classic game starts : what comes first : log rotation or server crash.

[RobinH]

@stephenw10

Tried this today, but it seems like there is no files there?

(RobinH@192.168.3.1) Password for RobinH@fw-01.zafe.se:
[2.6.0-RELEASE][RobinH@fw-01.zafe.se]/home/RobinH: cd /var/db/ntopng/rrd
[2.6.0-RELEASE][RobinH@fw-01.zafe.se]/var/db/ntopng/rrd: ls
graphics
[2.6.0-RELEASE][RobinH@fw-01.zafe.se]/var/db/ntopng/rrd:

[stephenw10]

Did you manage to free any significant amount of space?

[RobinH]

@stephenw10 No, I cant find the entries when i look in Var/db/ntopng

[stephenw10]

What does du -h -d1 / show now?

[RobbieTT]

Is it only me that is triggered by the topic title?

No, just me and symbols then.

️

[RobinH]

@RobbieTT Yeah... maybe it should be more like Free up space, disk storage >80%

[stephenw10]

I'd say either Used Disk Space >80% or Free Disk Space <20%. I can change it?

[RobinH]

@stephenw10 Please do

[RobinH]

@stephenw10 Well, I have'nt found the entrie files, so there is nothing deleted.

4.0K /.snap
3.0K /dev
1.4G /usr
117M /cf
1.3M /bin
4.0K /proc
96K /root
7.9M /etc
4.0K /net
426M /boot
12M /lib
4.0K /media
11M /rescue
4.0K /mnt
132M /tmp
4.9M /sbin
698M /var
192K /libexec
12K /conf.default
244K /home
4.0K /.cache
2.9G /

[stephenw10]

What do you see from geom part list?

[RobinH]

@stephenw10 How do I get that?

[Gertjan]

@RobinH

By typing that command.

[23.05.1-RELEASE][root@pfSense.bhf.net]/root: geom part list
Geom name: nvd0
modified: false
state: OK
fwheads: 255
fwsectors: 63
last: 234441607
first: 40
entries: 128
scheme: GPT
Providers:
1. Name: nvd0p1
   Mediasize: 209715200 (200M)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 20480
   Mode: r1w1e2
   efimedia: HD(1,GPT,ebd48d0e-974a-11ed-be66-90ec7729392c,0x28,0x64000)
   rawuuid: ebd48d0e-974a-11ed-be66-90ec7729392c
   rawtype: c12a7328-f81f-11d2-ba4b-00a0c93ec93b
   label: efiboot0
   length: 209715200
   offset: 20480
   type: efi
   index: 1
   end: 409639
   start: 40
2. Name: nvd0p2
   Mediasize: 524288 (512K)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 209735680
   Mode: r0w0e0
   efimedia: HD(2,GPT,ebdf0e41-974a-11ed-be66-90ec7729392c,0x64028,0x400)
   rawuuid: ebdf0e41-974a-11ed-be66-90ec7729392c
   rawtype: 83bd6b9d-7f41-11dc-be0b-001560b84f0f
   label: gptboot0
   length: 524288
   offset: 209735680
   type: freebsd-boot
   index: 2
   end: 410663
   start: 409640
3. Name: nvd0p3
   Mediasize: 1073741824 (1.0G)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 210763776
   Mode: r1w1e0
   efimedia: HD(3,GPT,ebe48038-974a-11ed-be66-90ec7729392c,0x64800,0x200000)
   rawuuid: ebe48038-974a-11ed-be66-90ec7729392c
   rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b
   label: swap0
   length: 1073741824
   offset: 210763776
   type: freebsd-swap
   index: 3
   end: 2508799
   start: 411648
4. Name: nvd0p4
   Mediasize: 118749134848 (111G)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 1284505600
   Mode: r1w1e1
   efimedia: HD(4,GPT,ebe8094b-974a-11ed-be66-90ec7729392c,0x264800,0xdd30000)
   rawuuid: ebe8094b-974a-11ed-be66-90ec7729392c
   rawtype: 516e7cba-6ecf-11d6-8ff8-00022d09712b
   label: zfs0
   length: 118749134848
   offset: 1284505600
   type: freebsd-zfs
   index: 4
   end: 234440703
   start: 2508800
Consumers:
1. Name: nvd0
   Mediasize: 120034123776 (112G)
   Sectorsize: 512
   Mode: r3w3e6

[RobinH]

@Gertjan said in Free up space, disk storage >80%:

geom part list

Hey!

This is what I get:

Geom name: da0
modified: false
state: OK
fwheads: 255
fwsectors: 63
last: 125829119
first: 63
entries: 4
scheme: MBR
Providers:

1. Name: da0s1
   Mediasize: 21474803200 (20G)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 32768
   Mode: r2w2e5
   efimedia: HD(1,MBR,0x90909090,0x40,0x27fffbf)
   attrib: active
   rawtype: 165
   length: 21474803200
   offset: 32768
   type: freebsd
   index: 1
   end: 41943038
   start: 64
   Consumers:
2. Name: da0
   Mediasize: 64424509440 (60G)
   Sectorsize: 512
   Mode: r2w2e7

Geom name: da0s1
modified: false
state: OK
fwheads: 255
fwsectors: 63
last: 41942974
first: 0
entries: 8
scheme: BSD
Providers:

1. Name: da0s1a
   Mediasize: 20401094656 (19G)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 32768
   Mode: r1w1e2
   rawtype: 7
   length: 20401094656
   offset: 0
   type: freebsd-ufs
   index: 1
   end: 39845887
   start: 0
2. Name: da0s1b
   Mediasize: 1073708032 (1.0G)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 3221258240
   Mode: r1w1e1
   rawtype: 1
   length: 1073708032
   offset: 20401094656
   type: freebsd-swap
   index: 2
   end: 41942973
   start: 39845888
   Consumers:
3. Name: da0s1
   Mediasize: 21474803200 (20G)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 32768
   Mode: r2w2e5

[stephenw10]

Hmm, nothing unexpected there. There's got to be something in da0s1a that du isn't seeing for some reason.

[RobinH]

@stephenw10 Well... now I have managed to erase all of the entries and PfBlockerNG. So its pretty clean when I run "find /var/db/ -type d -ls | sort -n -r"

2166917        8 drwxr-xr-x    2 root                             wheel                                 512 Jan 31  2022 /var/db/ports
2166916        8 drwx------    2 root                             wheel                                 512 Jan 31  2022 /var/db/hyperv
2166915        8 drwx------    2 root                             wheel                                 512 Jan 31  2022 /var/db/freebsd-update
2166914        8 drwx------    2 operator                         operator                              512 Jan 31  2022 /var/db/entropy
2007323        8 drwxr-xr-x    2 root                             wheel                                 512 Mar 17  2022 /var/db/vnstat
1927668        8 drwxr-xr-x    2 root                             wheel                                 512 Jun  2 22:08 /var/db/aliastables
1926291        8 drwxr-xr-x    2 root                             wheel                                 512 Jul 22  2020 /var/db/fontconfig
1926179       64 drwxr-xr-x    2 nobody                           wheel                               32256 Oct 30 14:20 /var/db/rrd
1926162        8 drwxr-xr-x    2 root                             wheel                                 512 Jan 31  2022 /var/db/zfsd/cases
1926161        8 drwxr-xr-x    2 123                              123                                   512 Jan 31  2022 /var/db/ntp
1926160        8 drwxr-xr-x    2 root                             wheel                                 512 Jan 31  2022 /var/db/portsnap
1926159        8 drwxr-xr-x    3 root                             wheel                                 512 Jan 31  2022 /var/db/zfsd
1926158        8 drwxr-xr-x    2 root                             wheel                                 512 Nov  3 03:05 /var/db/pkg
1926157        8 drwx------    2 root                             wheel                                 512 Jan 31  2022 /var/db/ipf
1926154       16 drwxr-xr-x   15 root                             wheel                                8192 Nov  3 14:42 /var/db/

But what is taking all the storage....

[RobinH]

@RobinH When I run "gpart show -p da0" this is what I get:

=>       63  125829057    da0  MBR  (60G)
         63          1         - free -  (512B)
         64   41942975  da0s1  freebsd  [active]  (20G)
   41943039   83886081         - free -  (40G)

[stephenw10]

Yes that's expected until you use growfs to fill the new space.

[RobbieTT]

@stephenw10 said in Free up space, disk storage >80%:

Yes that's expected until you use growfs to fill the new space.

Or you find and delete data you don't need.

But as with other comments above, you really need a bigger partition than the 19GB one you have.

I'm still new to freeBSD (but not to other things) but you could use a command script* to find directory sizes in bytes such as:

find /insert_random/dir/or_file -type f |xargs stat -f %z|awk '{s+=$1} END {print s}'

As an example, on my in-production device:

[23.09-RC][admin@Router-7]/root: find /var/db/ -type f |xargs stat -f %z|awk '{s+=$1} END {print s}'
58502534
[23.09-RC][admin@Router-7]/root:

So there is 58502534 bytes, or 58.5 MB in my /var/db/ directory.

*I have no doubt there is a better or simpler method than the above but I am trying to remember stuff from my childhood and I'm in my 50's.

️

[RobbieTT]

Ok, done some learning and there is a -h flag available to make du a bit more readable to dumb humans:

So for example:

[23.09-RC][admin@Router-7]/root: du -s -h /usr/
910M	/usr/
[23.09-RC][admin@Router-7]/root: du -s -h /var/
718M	/var/
[23.09-RC][admin@Router-7]/root: du -s -h /./
1.8G	/./
[23.09-RC][admin@Router-7]/root: 

Every day a school day... again ...or something.

️