Create/symlink failed, no inodes free
-
Hello,
My name is Anthony, I am new to pfSense.
I recently setup an old compaq desktop to try it out.
So far has fallen down twice the other day, and about to tonight.–
Packages installed- Avahi (0.6.25_1)
- RRD Summary (1.1)
- States Summary (0.5)
- bandwidth (2.0.1.2)
- darkstat (3.0.712)
- iperf (2.0.2_1)
- ntop (3.3.8)
- rate (0.9)
- squid (2.7.9_1)
Last week, proxy would stop responding.
When trying to SSH into box, i would get dropped straight after entering password.
VGA on console showed a bunch of errors, which I didnt record. (out of inodes).
Gave it a 3rd shot tonight with hopes of documenting and asking here.
Whilest I can still browse the web (squid shutdown), I am getting occasional 500 errors in GUI.eg.
http://192.168.2.123/system.php
500 - Internal Server ErrorSystem information
Name pfsense.nowlanworks.com
Version 1.2.3-RELEASE
built on Sun Dec 6 23:21:36 EST 2009
Platform pfSense
Uptime 01:16
State table size 5/10000 Show states
MBUF Usage 436 /1035
CPU usage 1%
Memory usage 16%
SWAP usage 0%
Disk usage 3%I remember this saying up 14,000 days with segmentation faults.
--
http://192.168.2.123/diag_logs.php
Last 100 system log entries
Segmentation fault
http://192.168.2.123/pkg_mgr.php
Warning: fopen(/tmp/pkg_info.cache): failed to open stream: Device not configured in /usr/local/www/pkg_mgr.php on line 36 Warning: fwrite(): supplied argument is not a valid stream resource in /usr/local/www/pkg_mgr.php on line 37 Warning: fclose(): supplied argument is not a valid stream resource in /usr/local/www/pkg_mgr.php on line 38
http://192.168.2.123/system.php
500 - Internal Server Error
When I SSH to box, and issue option 11 - restart GUI
Enter an option: 11
/etc/rc.initial: /etc/rc.restart_webgui: Device not configured
/: create/symlink failed, no inodes free
[Mon Aug 30 23:47:03 2010] [apc-error] apc_fcntl_create: open(/tmp/.apc.4CVfPF, O_RDWR|O_CREAT, 0666) failed: No space left on device–-
Fortunately I have kept another session open (recall last week i got disconnected).uname -a
FreeBSD pfsense.nowlanworks.com 7.2-RELEASE-p5 FreeBSD 7.2-RELEASE-p5 #0: Sun Dec 6 22:57:48 EST 2009 sullrich@FreeBSD_7.2_pfSense_1.2.3_snaps.pfsense.org:/usr/obj.pfSense/usr/pfSensesrc/src/sys/pfSense.7 i386
df -H
Filesystem Size Used Avail Capacity Mounted on
/dev/ad0s1a 14G 359M 12G 3% /
devfs 1.0k 1.0k 0B 100% /dev
/dev/md0 3.8M 35k 3.4M 1% /var/run
devfs 1.0k 1.0k 0B 100% /var/dhcpd/dev
procfs 4.1k 4.1k 0B 100% /procLooks to me 12gig free.
--
last pid: 10949; load averages: 0.12, 0.10, 0.12 up 0+01:06:00 23:50:53
100 processes: 2 running, 77 sleeping, 6 zombie, 15 waiting
CPU: 0.0% user, 3.0% nice, 15.7% system, 0.0% interrupt, 81.3% idle
Mem: 34M Active, 31M Inact, 45M Wired, 116K Cache, 60M Buf, 378M Free
Swap: 1024M Total, 1024M FreePID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND
11 root 1 171 ki31 0K 8K RUN 62:36 98.58% idle: cpu0
561 root 1 4 0 41736K 15676K accept 0:16 0.20% php
22 root 1 -24 - 0K 8K WAIT 0:10 0.00% swi6: task queue
12 root 1 -32 - 0K 8K WAIT 0:07 0.00% swi4: clock sio
44 root 1 20 - 0K 8K syncer 0:03 0.00% syncer
1260 root 1 44 0 7780K 3344K select 0:02 0.00% sshd--
I noticed that the box overwrote all my old logs, instead of appending.
Im not sure what other information I can provide.I know my way around a shell, but not troubleshooting so much.
I'd like to sort this out, before writing it off.
Anthony.
-
Bump?
Nuke it from orbit?
-
If you get an out of inodes error, that means that:
(a) your hdd/cf media is bad
(b) you are the the max number of files that the slice can hold.Of the two, (a) is more likely, given the other errors you have, but it could go either way,
Note that being out of inodes is not the same as being out of space. You could have too many small files (e.g. squid cache) and have lots of free space left, but no more available inodes to reference new files.
-
Thank you Jim.
I think I might try a vanilla install again, and see if same problem comes back.