Suricata memory usage very high
-
Hi,
I'm facing a strange issue (misconfiguration ?) with Suricata.
When I play a video hosted on my SMB server, memory usage of suricata thread increase continuously . You can check screenshot bellow.
If I pause video, memory usage stop increase and when I press play again memory is usage is increase again. If I continue to play video Suricata use all memory and restart (with network issue). When a close video player, suricata free memory after few second.
It feels like Suricata is buffering a copy of file to inspect it when "download"/"playing" is complete.
Technical information :
Pfsense 2.4.5.r.20200318.0600
Suricata 5.0.2 INLINE mode
vtnet1 is local LAN on VLAN 10 with my SMB server
vtnet3 is local LAN on VLAN 30 with my client
VLAN 30 is configure as EXTERNAL_NET on Suricata vtnet1 instance.Any idea to fix this ?
-
You are likely hitting the SMB parser memory leak bug in Suricata 5.x. You can find out about it on the upstream Suricata Redmine bug reporting site here: https://redmine.openinfosecfoundation.org/projects/suricata. The only way to fix it until the next Suricata release will be to stop using the SMB parser.
If this is your home network, then there is really very little reason at all to run the SMB parser. In fact, the majority of such parsers could be disabled saving both resources and potential issues from various bugs that have crept into the Suricata code with the recent upstream decision to switch over to Rust instead of the original C code. In my humble opinion, that was a very bad idea for upstream.
-
Does anyone know the actual bug report for this issue? I'm not finding it in the Suricata Redmine site. Thanks.
-
@opoplawski said in Suricata memory usage very high:
Does anyone know the actual bug report for this issue? I'm not finding it in the Suricata Redmine site. Thanks.
Here is one of the SMB bugs. This particular one was about crashing when scanning SMB data: https://redmine.openinfosecfoundation.org/issues/3342. Maybe that is the one I remembered and was falsely conflating it with memory leaks.
There are a number of small memory leak bugs logged for Suricata 5.x and 6.x BETA. Some of them have been closed recently and will show up in the next Suricata release which will be 5.0.3.
There were also some SMB flow-related bugs listed that should be fixed in the upcoming Suricata release from upstream. You might be hitting one of those.
-
FWIW - I'm still seeing suricata periodically (actually, pretty regularly every other day or so) consume all memory and get killed. This is with 6.0.4_1 on pfSense 2.6.0.
-
@opoplawski said in Suricata memory usage very high:
FWIW - I'm still seeing suricata periodically (actually, pretty regularly every other day or so) consume all memory and get killed. This is with 6.0.4_1 on pfSense 2.6.0.
You may be seeing the result of this open bug: https://redmine.openinfosecfoundation.org/issues/5363. There are a few upstream bug reports of memory leaks in Suricata. The pfSense package is at the mercy of the upstream team in this area, though. These leaks are inside the running binary code.
Here is another from the Suricata Redmine site: https://redmine.openinfosecfoundation.org/issues/5368.
And another one: https://redmine.openinfosecfoundation.org/issues/5204.Just to be clear so everyone understands, what folks see as a Suricata package on pfSense is actually two distinct pieces of code. Under the hood is the Suricata binary engine that comes from upstream. That binary does the actual loading of rules and inspection of the packets and triggering of alerts. The other piece of the package that users see and interact with is a GUI front-end written in PHP. All that front-end does is give you a GUI for manipulating the configuration parameters that end up getting fed to that running binary discussed earlier. Unless you are actively interacting with Suricata in the pfSense GUI, that PHP piece of the package is not even running. Only the binary runs 100% of the time.
There is a 6.0.5 version of the binary posted, but I've held off updating the Suricata package for two reasons. First, I wanted to make sure the new version was relatively free of serious issues. And second, not a ton of things changed between 6.0.4 and 6.0.5, so I saw no urgency.
-
@bmeeks The first defect affects version 6.0.5, and the last two have Fixed version "TBD". So even if we update to 6.0.5 it will not fix it. Regardless of being a fan of Suricata, they don't test on FreeBSD much, only on Linux, and on Windows. Last time you found a pretty critical bug.
-
@le_bleu said in Suricata memory usage very high:
Pfsense 2.4.5.r.20200318.0600
This is a beta release of 2.4.5 and more than 2 years old. You should upgrade to 2.5.2 at this point or at least the RELEASE version of 2.4.5.
