Virtualized pfsense 2.1.5 - out of swap space
-
Hi,
Im getting strange behaviour on my virtualized pfsense 2.1.5 (qemu 2.0.0+dfsg-2ubuntu1.9 and libvirt 1.2.2-0ubuntu13.1.7).
I'm getting those messages:
Jan 5 19:55:14 pfsense kernel: swap_pager_getswapspace(16): failed Jan 5 19:55:14 pfsense kernel: swap_pager_getswapspace(16): failed Jan 5 19:55:14 pfsense kernel: swap_pager_getswapspace(16): failed Jan 5 19:55:14 pfsense kernel: swap_pager_getswapspace(16): failed Jan 5 19:55:14 pfsense kernel: swap_pager_getswapspace(12): failed Jan 5 19:55:14 pfsense kernel: swap_pager_getswapspace(16): failed Jan 5 19:55:14 pfsense kernel: swap_pager_getswapspace(12): failed Jan 5 19:55:14 pfsense kernel: pid 22576 (php), uid 0, was killed: out of swap space
Sometimes my radius seems to get shot in the head by this.
But my machine has 1 GB of RAM and as normally it's using hardly any RAM, but look for yourself. I have no idea why the gaps.
I experimented with squid, but it's not even installed any more.There also seem to be a lot of php processes:
root 1792 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn HE_IPV6_TUNNELV6 root 2763 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn WAN_PPPOE root 4023 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn WAN_PPPOE root 4056 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn WAN_PPPOE root 4457 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn HE_IPV6_TUNNELV6 root 5368 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn WAN_PPPOE root 6851 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn WAN_PPPOE root 8260 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn HE_IPV6_TUNNELV6 root 10798 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn HE_IPV6_TUNNELV6 root 12744 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn WAN_PPPOE root 14130 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn WAN_PPPOE root 14656 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn HE_IPV6_TUNNELV6 root 16917 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn HE_IPV6_TUNNELV6 root 22529 0.0 1.5 147776 19848 ?? Is 7:55PM 0:00.06 /usr/local/bin/php root 24054 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn HE_IPV6_TUNNELV6 root 25582 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn WAN_PPPOE root 27940 0.0 3.1 156224 39672 ?? I 7:55PM 0:01.52 /usr/local/bin/php root 31934 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn HE_IPV6_TUNNELV6 root 32892 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn HE_IPV6_TUNNELV6 root 34593 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn WAN_PPPOE root 34632 0.0 1.6 147776 20176 ?? Is 8:16PM 0:00.06 /usr/local/bin/php root 34654 0.0 3.1 151872 39856 ?? I 8:16PM 0:01.25 /usr/local/bin/php root 36972 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn HE_IPV6_TUNNELV6 root 43906 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn WAN_PPPOE root 46206 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn HE_IPV6_TUNNELV6 root 48809 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn WAN_PPPOE root 49553 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn HE_IPV6_TUNNELV6 root 50333 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn HE_IPV6_TUNNELV6 root 53829 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn WAN_PPPOE root 54447 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn WAN_PPPOE root 54938 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn HE_IPV6_TUNNELV6 root 55750 0.0 2.9 160064 38020 ?? IN 4:30PM 0:00.16 /usr/local/bin/php -f /etc/rc.openvpn WAN_PPPOE root 57091 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn WAN_PPPOE root 58447 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn HE_IPV6_TUNNELV6 root 58782 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn HE_IPV6_TUNNELV6 root 59429 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn WAN_PPPOE root 59467 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn HE_IPV6_TUNNELV6 root 61061 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn WAN_PPPOE root 62477 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn HE_IPV6_TUNNELV6 root 64928 0.0 2.9 160064 38116 ?? IN 7:45PM 0:00.13 /usr/local/bin/php -f /etc/rc.openvpn WAN_PPPOE root 65593 0.0 2.9 160064 38116 ?? IN 7:45PM 0:00.13 /usr/local/bin/php -f /etc/rc.openvpn HE_IPV6_TUNNELV6 root 67485 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn WAN_PPPOE root 73511 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn WAN_PPPOE root 74311 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn HE_IPV6_TUNNELV6 root 74356 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn HE_IPV6_TUNNELV6 root 81334 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn WAN_PPPOE root 81873 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn WAN_PPPOE root 82196 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn HE_IPV6_TUNNELV6 root 82861 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn HE_IPV6_TUNNELV6 root 88261 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn WAN_PPPOE root 89119 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn HE_IPV6_TUNNELV6 root 89412 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn HE_IPV6_TUNNELV6 root 95295 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn WAN_PPPOE root 97302 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn WAN_PPPOE root 99899 0.0 0.0 160064 0 ?? IWN - 0:00.00 /usr/local/bin/php -f /etc/rc.openvpn WAN_PPPOE
-
Swap space is usually on the hard drive and not in memory, right?
-
What makes you think I don't know the difference between physical and virtual memory?
My experience actually shows me that swap is only getting rare, when phyical memory is getting rare, too. -
how much swap space is available when you are getting these messages. (is it a bad error message / or is swap really full ?)
it appears you are using 70% of RAM at the end of the RRD-graph. can you find out what processes are using it ? (using top or something)
-
That's current top:
last pid: 98703; load averages: 0.05, 0.17, 0.99 up 0+01:51:55 09:50:35
45 processes: 1 running, 44 sleeping
CPU: 0.0% user, 0.0% nice, 3.6% system, 0.4% interrupt, 96.1% idle
Mem: 35M Active, 19M Inact, 128M Wired, 1856K Cache, 40M Buf, 1993M Free
Swap: 512M Total, 81M Used, 431M Free, 15% InuseEdit:
I just gave the virtual machine 2 gig of RAM because I'm desperate. -
Strange. I run 2.1.5 in production under ESXi 5.5 w/2GB and 2.2 in my home lab under VirtualBox w/512MB. Never had a swap error, and that's with multiple packages installed and acrobats on the trapeze and everything.
-
Something threw openvpn's reload script into a loop and consumed all your RAM. Does it recur after a reboot? The reboot (or just a "killall php") will clean up the mess it's gotten itself into. It's not clear what or why, can't say I've seen anything like that.