PfSense crashing once a week
-
We have a pfSense running as an internal firewall. It is virtualized in VmWare and running pfSense 2.3.4 release.
No additional packages are installed.
Approximately one a week the firewall crashes and reboots.
There is always the same thread marked as curthread in the crashlog: "em6 taskq"
The special thing about this interface em6 is that there is, for routing reasons, quite number of NAT reflections in place that can get hit with substantial traffic (50 MB/s).
Is there anythobg known that NAT reflection can crash the pfSense?Regards
Greyhat -
You have not provided nearly enough information to speculate about a cause.
If you can, post the full crash dump as a text attachment here. Or at least post the backtrace and the last hundred or so lines of the message buffer.
-
Attached is a complete crash dump. The NAT is a combination of 1:1 NAT and Outgoing.
The NAT definitions are like this:
<onetoone><external>10.49.209.0</external><interface>opt5</interface>
<source><address>10.144.0.0/24</address>
<destination><any></any></destination>
<natreflection>disable</natreflection></onetoone><onetoone><external>10.49.211.0</external>
<interface>opt5</interface>
<source><address>10.144.2.0/26</address>
<destination><any></any></destination>
<natreflection>disable</natreflection></onetoone><onetoone><external>10.49.211.64</external>
<interface>opt5</interface>
<source><address>10.144.2.64/26</address>
<destination><any></any></destination>
<natreflection>disable</natreflection></onetoone><onetoone><external>10.49.211.128</external>
<interface>opt5</interface>
<source><address>10.144.128.0/27</address>
<destination><any></any></destination>
<natreflection>disable</natreflection></onetoone><outbound><mode>advanced</mode>
<rule><source>
<network>any</network><target>10.49.8.1</target>
<targetip></targetip>
<targetip_subnet></targetip_subnet>
<interface>opt5</interface><destination><address>10.144.0.0/24</address></destination>
<updated><time>1471849596</time>
<username>admin@192.168.0.43</username></updated>
<created><time>1471849596</time>
<username>admin@192.168.0.43</username></created></rule>
<rule><source>
<network>any</network><target>10.49.8.1</target>
<targetip></targetip>
<targetip_subnet></targetip_subnet>
<interface>opt5</interface><destination><address>10.144.2.0/24</address></destination>
<updated><time>1471849606</time>
<username>admin@192.168.0.43</username></updated>
<created><time>1471849606</time>
<username>admin@192.168.0.43</username></created></rule>
<rule><source>
<network>any</network><target>10.49.8.1</target>
<targetip></targetip>
<targetip_subnet></targetip_subnet>
<interface>opt5</interface><destination><address>10.144.128.0/24</address></destination>
<updated><time>1471849615</time>
<username>admin@192.168.0.43</username></updated>
<created><time>1471849615</time>
<username>admin@192.168.0.43</username></created></rule></outbound>[2017_07_18_pfIntern1 Crash.txt](/public/imported_attachments/1/2017_07_18_pfIntern1 Crash.txt)
-
That looks like a problem that is usually solved by setting the NIC queues to 1, though it's early this morning and I can't remember if em has a knob for that. igb does.
You would be better off using vmxnet3 NICs if you can. At least until pfSense 2.4.x is shipping on FreeBSD 11.1 which will be quite soon.
-
Thanks for the hint.
It does not seem that there is a switch to tune the queue length.
I do not suppose you mean values in system tunables.
I will try and set the adapters to vmxnet3 and see what happens.
