Running out of Swap space



  • Hi,

    I'm running the latest version on a soekris

    Platform nanobsd (4g)
    NanoBSD Boot Slice pfsense0 / ad1s1 (rw)

    and the following errors occur, guess this is not normal ? don't know what process gets killed.

    web site still starts up.

    Jan 20 09:52:07 kernel: pid 12708 (php), uid 0, was killed: out of swap space
    Jan 20 09:52:07 kernel: pid 11367 (php), uid 0, was killed: out of swap space
    Jan 20 09:52:07 kernel: pid 11048 (php), uid 0, was killed: out of swap space
    Jan 20 09:52:07 kernel: pid 36260 (php), uid 0, was killed: out of swap space
    Jan 20 09:52:07 kernel: pid 10524 (php), uid 0, was killed: out of swap space
    Jan 20 09:52:07 kernel: pid 8974 (php), uid 0, was killed: out of swap space
    Jan 20 09:52:07 kernel: pid 293 (php), uid 0, was killed: out of swap space



  • @Stef:

    don't know what process gets killed.

    php: @Stef:

    Jan 20 09:52:07 kernel: pid 12708 (php), uid 0, was killed: out of swap space

    If too many processes start at once they can exhaust memory. The nanobsd version has no swap space.

    What model soekris do you have? How much RAM does it have?



  • and what features and packages are you using. e.g. OpenVPN servers and clients grab quite a bit of memory as they initialise/come up then "settle down" once they are running.



  • It's running on an older

    net4801-48: 233 Mhz CPU, 128 Mbyte SDRA

    There are no packages installed.

    Ipsec & openvpn are both configured. and they seem to both work.

    This error occurs quite early in the boot process.



  • I know that on my Alix 2D13 256MB systems, if I have more than 2 OpenVPN servers+clients, then I get "killed - out of swap space" messages during the boot. From what I can see, when an OpenVPN client or server is actually bringing up its link it seems to use a lot of real memory for maybe 30 seconds or so. Occasionally at boot something important is killed (like dnsmasq or dhcpd) and has to be restarted from the WebGUI, but usually the last OpenVPN to come up causes filter reload etc etc things to happen anyway, and it all works.
    If I ever had spare time, I would like to sort out how to control the boot on low-memory systems to do less things at once and avoid "out of swap space".
    If you disable all your OpenVPN clients and servers does it boot clean?
    If you enable them 1 at a time and reboot, how many can you have and still get a clean boot?



  • Good Suggestion.

    I've tried to disable openvpn, but it doesn't change anything.

    It does change that dnsmasq doesn't get killed anymore… which is one less process to get killed

    dhcp & racoon still get killed, but dhcp is a different problem.

    Jan 25 09:32:24 kernel: pid 284 (php), uid 0, was killed: out of swap space
    Jan 25 09:32:24 kernel: pid 9925 (php), uid 0, was killed: out of swap space
    Jan 25 09:32:24 kernel: pid 12319 (php), uid 0, was killed: out of swap space
    Jan 25 09:32:24 kernel: pid 35311 (php), uid 0, was killed: out of swap space
    Jan 25 09:32:24 kernel: pid 13497 (php), uid 0, was killed: out of swap space
    Jan 25 09:32:24 kernel: pid 13842 (php), uid 0, was killed: out of swap space

    is there a way to make things more sequential our wait on certain actions before starting new onces ?  or invercing startup orders of services ?



  • 128 MB RAM is light if you want to run anything beyond the defaults, though if you're on more recent snapshots they're more memory-friendly than earlier ones. Still not going to get a whole lot out of 128 MB though.



  • Hi All,

    I have a similar issue.  I've installed v2.0.2 on an oldish laptop Dell (D610) with 1GB RAM and 20GB HDD.  Its been running fine for a while however it started going wrong yesterday.

    I installed squid, but did not configure it.  This seems to be related as I've found a few othe rposts that reference squid as a possible culprit.

    My symptoms are that it reports "out of swap space" on the console and http access does not work.

    Regards,
    Wikus



  • Anyone interested can you try this change on your system

    UPDATED:

    diff –git a/etc/rc b/etc/rc
    index a2f8cf3..68726d0 100755
    --- a/etc/rc
    +++ b/etc/rc
    @@ -389,8 +389,6 @@ if [ -f /etc/rc.custom_boot_early ]; then
           echo "Done"
    fi

    -/usr/bin/nice -n20 /usr/local/sbin/check_reload_status

    let the PHP-based configuration subsystem set up the system now

    echo -n "Launching the init system…"
    /bin/rm -f /cf/conf/backup/backup.cache
    @@ -405,6 +403,8 @@ if [ -f "/tmp/donotbootup" ]; then
           exit 0
    fi

    +/usr/bin/nice -n20 /usr/local/sbin/check_reload_status

    echo -n "Starting CRON… "
    cd /tmp && /usr/sbin/cron -s 2>/dev/null
    echo "done."



  • Alix 2D13 256MB real memory
    2.1-BETA1 (i386)
    built on Mon Mar 11 20:50:42 EDT 2013
    4 OpenVPN shared key servers
    2 OpenVPN Peer to Peer SSL/TLS clients
    bandwidthd package

    When connected to a real internet, so that the OpenVPN servers and/or clients start getting/making connections as soon as they start, "Killed" messages appear on the console late in the boot (I guess when enough stuff is all going that it runs out of memory). led_normalise does not happen (the LEDs stay flashing forever after bootup). The file systems are left RW.
    Booted twice to make sure the problem repeats.

    Made Ermal's change to /etc/rc - move check_reload_status down to after the main startup code in /etc/rc.bootup

    Bootup does not have any "Killed" messages. LEDs normalise at end of boot process. Filesystem is RO.
    Booted 3 times - all successful.

    Put the code back the way it was before - "Killed" problems come back.

    Made the change again - booted and it comes up clean.
    Looks like a good thing - I'll keep this change!

    Console output before the change:

    Welcome to pfSense 2.1-BETA1  ...
    
    Creating symlinks......done.
    External config loader 1.0 is now starting... ad0s3
    Launching the init system... done.
    Initializing............................. done.
    Starting device manager (devd)...done.
    Loading configuration......done.
    Updating configuration...done.
    Cleaning backup cache........done.
    Setting up extended sysctls...done.
    Setting timezone...done.
    Configuring loopback interface...done.
    Starting syslog...done.
    Starting Secure Shell Services...done.
    Setting up polling defaults...done.
    Setting up interfaces microcode...done.
    Configuring loopback interface...done.
    Creating wireless clone interfaces...done.
    Configuring LAGG interfaces...done.
    Configuring VLAN interfaces...done.
    Configuring QinQ interfaces...done.
    Configuring WAN interface...done.
    Configuring LAN interface...done.
    Syncing OpenVPN settings...route: writing to routing socket: No such process
    route: writing to routing socket: No such process
    route: writing to routing socket: No such process
    route: writing to routing socket: No such process
    route: writing to routing socket: No such process
    route: writing to routing socket: No such process
    route: writing to routing socket: No such process
    route: writing to routing socket: No such process
    route: writing to routing socket: No such process
    route: writing to routing socket: No such process
    done.
    Configuring firewall.route: writing to routing socket: No such process
    .DUMMYNET 0 with IPv6 initialized (100409)
    load_dn_sched dn_sched FIFO loaded
    load_dn_sched dn_sched QFQ loaded
    load_dn_sched dn_sched RR loaded
    load_dn_sched dn_sched WF2Q+ loaded
    load_dn_sched dn_sched PRIO loaded
    Bump sched buckets to 256 (was 0)
    Bump sched buckets to 256 (was 0)
    ....done.
    Starting PFLOG...done.
    Setting up gateway monitors...done.
    Synchronizing user settings...done.
    Starting webConfigurator...done.
    Configuring CRON...done.
    Starting DNS forwarder...done.
    Starting NTP time client...done.
    Starting DHCP service...done.
    Configuring firewall.route: writing to routing socket: No such process
    .Bump sched buckets to 256 (was 0)
    Bump sched buckets to 256 (was 0)
    ....done.
    Generating RRD graphs...Killed
    Starting CRON... done.
    Killed
    Bootup complete
    
    

    and after:

    Welcome to pfSense 2.1-BETA1  ...
    
    Creating symlinks......done.
    External config loader 1.0 is now starting... ad0s3
    Launching the init system... done.
    Initializing............................. done.
    Starting device manager (devd)...done.
    Loading configuration......done.
    Updating configuration...done.
    Cleaning backup cache........done.
    Setting up extended sysctls...done.
    Setting timezone...done.
    Configuring loopback interface...done.
    Starting syslog...done.
    Starting Secure Shell Services...done.
    Setting up polling defaults...done.
    Setting up interfaces microcode...done.
    Configuring loopback interface...done.
    Creating wireless clone interfaces...done.
    Configuring LAGG interfaces...done.
    Configuring VLAN interfaces...done.
    Configuring QinQ interfaces...done.
    Configuring WAN interface...done.
    Configuring LAN interface...done.
    Syncing OpenVPN settings...route: writing to routing socket: No such process
    route: writing to routing socket: No such process
    route: writing to routing socket: No such process
    route: writing to routing socket: No such process
    route: writing to routing socket: No such process
    route: writing to routing socket: No such process
    route: writing to routing socket: No such process
    route: writing to routing socket: No such process
    route: writing to routing socket: No such process
    route: writing to routing socket: No such process
    done.
    Configuring firewall.route: writing to routing socket: No such process
    .DUMMYNET 0 with IPv6 initialized (100409)
    load_dn_sched dn_sched FIFO loaded
    load_dn_sched dn_sched QFQ loaded
    load_dn_sched dn_sched RR loaded
    load_dn_sched dn_sched WF2Q+ loaded
    load_dn_sched dn_sched PRIO loaded
    Bump sched buckets to 256 (was 0)
    Bump sched buckets to 256 (was 0)
    ....done.
    Starting PFLOG...done.
    Setting up gateway monitors...done.
    Synchronizing user settings...done.
    Starting webConfigurator...done.
    Configuring CRON...done.
    Starting DNS forwarder...done.
    Starting NTP time client...done.
    Starting DHCP service...done.
    Configuring firewall.route: writing to routing socket: No such process
    .Bump sched buckets to 256 (was 0)
    Bump sched buckets to 256 (was 0)
    ....done.
    Generating RRD graphs...Bump sched buckets to 256 (was 0)
    Bump sched buckets to 256 (was 0)
    Bump sched buckets to 256 (was 0)
    Bump sched buckets to 256 (was 0)
    done.
    Starting syslog...done.
    Starting CRON... done.
     Starting package Cron...done.
     Starting package bandwidthd...done.
    Bootup complete
    
    

    Note: An issue for another thread - what are all the "route: writing to routing socket: No such process" messages about? I think they started coming a month or 2 ago?


Locked