Bandwidthd out of inodes / Unable to administer router

ttblum

Hello,

I have bandwidthd running on an embedded platform (running on CF card) and I am getting the following:

Jun 1 09:34:56 kernel: pid 40961 (bandwidthd), uid 0 inumber 37811 on /: out of inodes
Jun 1 09:34:56 kernel: pid 40961 (bandwidthd), uid 0: exited on signal 11

All of a sudden, while the dashboard is displaying OK, all other pages are blank. Trying to log in via SSH results in 'Connection closed'. What do I do? The device still seems to be routing traffic OK.

Thanks,

Todd

Cry Havok

What version (number and type) of pfSense are you running and what packages have you installed?

jimp

If you're out of inodes means that the drive is full, not in terms of space, but in the number of files that can exist on the filesystem.

Usual ways for this to happen are from the squid cache, or some other package. I didn't think bandwidthd made a bunch of tiny little files but it might.

You can check inode usage as well as space by looking at

df -i

For example:

: df -i
Filesystem  1K-blocks   Used   Avail Capacity iused  ifree %iused  Mounted on
/dev/ad0s1a   7103078 811468 5723364    12%   29632 912446    3%   /
devfs               1      1       0   100%       0      0  100%   /dev
/dev/md0         3694     66    3334     2%      37    729    5%   /var/run
procfs              4      4       0   100%       1      0  100%   /proc
devfs               1      1       0   100%       0      0  100%   /var/dhcpd/dev

The "avail" and "capacity" refer to the drive's free space, and then "ifree" and "iused" columns refer to the drive's remaining inodes.

ttblum

Hello,

I am running 2.0.1-RELEASE (i386) on nanobsd (4g), and I only have the bandwidthd package installed.

I am unable to access the router via SSH in order to free up inodes. Do you think the serial port would be responsive enough to get to a shell? I am also considering power cycling the device, but I don't know if that would help or hurt.

Thanks,

Todd

jimp

If you're on a cf and receive such an error, it probably means your CF has failed, or is about to fail.

Get a backup, image a fresh card, swap it out.

cmb

Out of inodes is either something (would have to be a package) has gone wild and created a huge number of small files (usually you run out of disk space before you run out of inodes, unless there is a huge number of small files). The other alternative is what Jim mentioned, a failing HD or CF will at times cause out of inodes messages rather than the more common read and/or write errors.