Create/symlink failed, no inodes free



  • Hello,

    My name is Anthony, I am new to pfSense.

    I recently setup an old compaq desktop to try it out.
    So far has fallen down twice the other day, and about to tonight.


    Packages installed

    • Avahi (0.6.25_1)
    • RRD Summary (1.1)
    • States Summary (0.5)
    • bandwidth (2.0.1.2)
    • darkstat (3.0.712)
    • iperf (2.0.2_1)
    • ntop (3.3.8)
    • rate (0.9)
    • squid (2.7.9_1)

    Last week, proxy would stop responding.
    When trying to SSH into box, i would get dropped straight after entering password.
    VGA on console showed a bunch of errors, which I didnt record. (out of inodes).


    Gave it a 3rd shot tonight with hopes of documenting and asking here.
    Whilest I can still browse the web (squid shutdown), I am getting occasional 500 errors in GUI.

    eg.
    http://192.168.2.123/system.php
    500 - Internal Server Error

    System information
    Name pfsense.nowlanworks.com
    Version 1.2.3-RELEASE
    built on Sun Dec 6 23:21:36 EST 2009
    Platform pfSense
    Uptime 01:16
    State table size 5/10000 Show states
    MBUF Usage 436 /1035
    CPU usage 1%
    Memory usage 16%
    SWAP usage 0%
    Disk usage 3%

    I remember this saying up 14,000 days with segmentation faults.

    --
    http://192.168.2.123/diag_logs.php
    Last 100 system log entries
    Segmentation fault


    http://192.168.2.123/pkg_mgr.php
    Warning: fopen(/tmp/pkg_info.cache): failed to open stream: Device not configured in /usr/local/www/pkg_mgr.php on line 36 Warning: fwrite(): supplied argument is not a valid stream resource in /usr/local/www/pkg_mgr.php on line 37 Warning: fclose(): supplied argument is not a valid stream resource in /usr/local/www/pkg_mgr.php on line 38


    http://192.168.2.123/system.php
    500 - Internal Server Error


    When I SSH to box, and issue option 11 - restart GUI

    Enter an option: 11

    /etc/rc.initial: /etc/rc.restart_webgui: Device not configured

    /: create/symlink failed, no inodes free
    [Mon Aug 30 23:47:03 2010] [apc-error] apc_fcntl_create: open(/tmp/.apc.4CVfPF, O_RDWR|O_CREAT, 0666) failed: No space left on device

    –-
    Fortunately I have kept another session open (recall last week i got disconnected).

    uname -a

    FreeBSD pfsense.nowlanworks.com 7.2-RELEASE-p5 FreeBSD 7.2-RELEASE-p5 #0: Sun Dec  6 22:57:48 EST 2009    sullrich@FreeBSD_7.2_pfSense_1.2.3_snaps.pfsense.org:/usr/obj.pfSense/usr/pfSensesrc/src/sys/pfSense.7  i386

    df -H

    Filesystem    Size    Used  Avail Capacity  Mounted on
    /dev/ad0s1a    14G    359M    12G    3%    /
    devfs          1.0k    1.0k      0B  100%    /dev
    /dev/md0      3.8M    35k    3.4M    1%    /var/run
    devfs          1.0k    1.0k      0B  100%    /var/dhcpd/dev
    procfs        4.1k    4.1k      0B  100%    /proc

    Looks to me 12gig free.

    --

    last pid: 10949;  load averages:  0.12,  0.10,  0.12                                                                          up 0+01:06:00  23:50:53
    100 processes: 2 running, 77 sleeping, 6 zombie, 15 waiting
    CPU:  0.0% user,  3.0% nice, 15.7% system,  0.0% interrupt, 81.3% idle
    Mem: 34M Active, 31M Inact, 45M Wired, 116K Cache, 60M Buf, 378M Free
    Swap: 1024M Total, 1024M Free

    PID USERNAME  THR PRI NICE  SIZE    RES STATE    TIME  WCPU COMMAND
      11 root        1 171 ki31    0K    8K RUN    62:36 98.58% idle: cpu0
      561 root        1  4    0 41736K 15676K accept  0:16  0.20% php
      22 root        1 -24    -    0K    8K WAIT    0:10  0.00% swi6: task queue
      12 root        1 -32    -    0K    8K WAIT    0:07  0.00% swi4: clock sio
      44 root        1  20    -    0K    8K syncer  0:03  0.00% syncer
    1260 root        1  44    0  7780K  3344K select  0:02  0.00% sshd

    --

    I noticed that the box overwrote all my old logs, instead of appending.
    Im not sure what other information I can provide.

    I know my way around a shell, but not troubleshooting so much.

    I'd like to sort this out, before writing it off.

    Anthony.



  • Bump?

    Nuke it from orbit?


  • Rebel Alliance Developer Netgate

    If you get an out of inodes error, that means that:

    (a) your hdd/cf media is bad
    (b) you are the the max number of files that the slice can hold.

    Of the two, (a) is more likely, given the other errors you have, but it could go either way,

    Note that being out of inodes is not the same as being out of space. You could have too many small files (e.g. squid cache) and have lots of free space left, but no more available inodes to reference new files.



  • Thank you Jim.

    I think I might try a vanilla install again, and see if same problem comes back.


Log in to reply