Memory usage steadily rises, pfsense crashes when it runs out.
-
Bit of background:
I've been running 2.0RC1 for the past couple of months with only the expected slight hiccups. Before that, I was running 1.2.3 rock stable.
Hardware: Neoware CA2, 800Mhz, 512MB flash disk, 256MB RAM, 2 NICs. I'm running 512MB embedded.
Last week, I started getting limited responsiveness and then complete unresponsiveness from the firewall… It would happen over a period of time (generally within 24 hours of boot) and at different times of day. It seemed to start at about the same time that I set up the firewall as an OpenVPN server (I currently have a Linux box performing that function behind the firewall). I removed the OpenVPN server and the client export package, but the problems persisted through multiple snapshots.
Last night, I rolled back (erased and reflashed) to 1.2.3, expecting my previous level of stability. I haven't got it. The firewall became unresponsive to all commands after just a few hours (the serial console allowed me to reboot, but I couldn't run commands in the shell) because it ran out of memory, and as we know, the embedded systems do not use swap. I rebooted about 4 hours ago, and have had to reboot again as the memory usage creeped up from 22% to nearly 80%. I'm not on-site, so I rebooted preemptively this time.
I'm not running any packages except for Dashboard. I moved logging to a remote server, but that doesn't seem to affect anything. I do have a couple of Ipsec tunnels, but they are disabled most of the time... otherwise, nothing out of the ordinary. I restored a config file from last October, when my uptime was in the hundreds of days on this same hardware.
Any ideas?
-Matt
-
Think I found the culprit….
I noticed that the number of processes was steadily increasing over time as well... Turned off NAT reflection, and things seem back to normal.
The start of this problem also coincided with my use of Wuala (a great service) which requires UDP port forwards in the ~7000 range to work properly.
Hope this helps someone else. It's worth noting that I experienced the same problem on 2.0RC1 and 1.2.3RELEASE.
-Matt