Random IPsec error
Ever since upgrading to 2.2.4, I've been getting this error periodically in System Alerts when I log in:
10-01-15 15:55:34 [ There were error(s) loading the rules: /tmp/rules.debug:130: macro IPsec not defined - The line in question reads : pass out on $IPsec all tracker 1000000961 tracker 1000000962 keep state label IPsec internal host to host]
Yes, IPsec is enabled and we use it actively.
Also, possibly related….since upgrading to 2.2.4, the machine has been locking up periodically with this error:
[zone: mbuf] kern.ipc.nmbufs limit reached
When this happens I can't access pfsense via the web gui or SSH. I have to view the physical console. Even then, it is completely unresponsive and i have to power cycle.
I previously increased kern.ipc.nmbclusters to 131072, but that didn't seem to help. I've actually never seen mbuf usage go over 1500-1600.
# cat /boot/loader.conf.local kern.ipc.nmbclusters="131072"
built on Sat Jul 25 19:57:37 CDT 2015
Just an update to this…
So the issue started creeping up again this morning. Getting the same IPsec errors, the web gui was unreachable, and I was getting the mbufs error at the console. The load average was astronomical (500-900) and memory was nearly exhausted.
After experimenting with killing different services via console, I finally found killing squid returned pfsense to a usable state: pfSsh.php playback svc stop squid
Are there any known issues with squid on 2.2.4? Did not have this issue on 2.2.3 and earlier.
EDIT: After researching, I see there are lots of issues with Squid3 and pfsense 2.2.4.
The issues people have with Squid are generally it not starting because of PBI problems. If it runs, you're not having the same issue.
kern.ipc.nmbufs is different from nmbclusters. You might need to bump kern.ipc.nmbufs separately in that case. Run 'sysctl kern.ipc.nmbufs', what's that set to?