DHCPD errors /: filesystem full - but really it is not!

[Mr_T]

I keep getting the following errors:
kernel: pid 40848 (dhcpd), uid 1002 inumber 9963101 on /: filesystem full
df -ih tells me I don't have a problem with the root filesystem /
$ df -ih
Filesystem    Size    Used  Avail Capacity iused ifree %iused  Mounted on
/dev/ad0s1a    81G    63G    11G    85%    9.5M  1.5M  86%  /
devfs          1.0k    1.0k      0B  100%      0    0  100%  /dev
/dev/md0      3.6M    46k    3.3M    1%      27  739    4%  /var/run
devfs          1.0k    1.0k      0B  100%      0    0  100%  /var/dhcpd/dev

This PFSense runs inside a proxmox VM that all sits on a samsung SSD…  I have checked the SMART status of the drive and it is happy.. Zero write errors no wear-out. 
I am running a significant squid proxy which accounts for most of the hard disk use.. but i am not near 100% yet.. Heaps of RAM and cpu resources assigned. Any help would be appreciated.. The system crashes every 6 hours with loads of  kernel: pid 40848 (dhcpd), uid 1002 inumber 9963101 on /: filesystem full.
Today I turned off all the logging and RRD in the hope it stabilizes. Needless to say this is not the way I want to run long term.
Cheers

[Mr_T]

It fell over again last night with all the logging turned off. Any suggestions people!

[KOM]

Weird.  First thing I would do is blow my cache away and then recreate it with some reasonable values like 200-500 MB, and see if it lives more than a day or two before failing again.  Your cache hit rate, if it's anything like mine or what I see online, is going to be in the area of 4-7% – not great.  Using up many gigs of space for a tiny hit rate is a waste.  I find Squid's value these days is more as a filtering platform with SquidGuard.

[jimp]

When a disk is claiming to be full but does not appear to be full, there are two possibilities:

1. The disk is not out of capacity, but out of inodes (too many small files, for example). Use "df -hi" to confirm.
2. The disk is failing and the OS cannot contact the disk to make a write, or the disk returned a write error.

2 is far more common than 1.

There is also a third option but it does not apply to your setup:
3. The /var RAM disk (NanoBSD or full w/the option to use it on) is full, not /

[Mr_T]

Thanks Jimp and KOM.

KOM, When reducing the squid cache to 500mb and flushing squid did not free up any space I started to hunt for the culprit.
Jimp I wish I looked at the inodes before i started deleting 60 Gig of SARG files (Just to be sure). There were a zillion small files in there. running a single rm -R * on the sarg-reports sub directories took 7 hours to complete.  Now my disk space is pitifully empty:
$ df -hi
Filesystem    Size    Used  Avail Capacity iused ifree %iused  Mounted on
/dev/ad0s1a    81G    1.9G    72G    3%    42k  11M    0%  /
devfs          1.0k    1.0k      0B  100%      0    0  100%  /dev
/dev/md0      3.6M    46k    3.3M    1%      27  739    4%  /var/run
devfs          1.0k    1.0k      0B  100%      0    0  100%  /var/dhcpd/dev

Thanks for your help! looks like we solved this one.. I am putting it down to the inodes being full.
Needless to say I removed the zero in the reporting options
Cheers..