snort using high amount of ram crashing pfsense
-
I'm having this issue with snort where it eat up a large amount of the ram, and then remote access to the the machine stops but I can still move traffic pass it. I see error messages pertaining to swap memory failing and these messages just fold the console. I've configured the VM with 16GB and you will see the usage increase continuously and then the machine crashes.
Also all software is running the latest version and pfsense is running the latest stable version. Vmware tools are installed.
[2.4.4-RELEASE][admin@pfSense.localdomain]/root: ps aux -m
USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND
root 76331 0.0 74.7 12537264 12490732 - Ss 19:15 0:00.78 /usr/local/bin/snort -R 40421 -D -q --suppress-config-log -l /var/log/snort/snort_em040421 --pid-path /var/run --nolock-pidfile -G 40421 -c root 77097 0.0 0.3 103812 56864 - Ss 19:15 0:00.02 /usr/local/bin/snort -R 35190 -D -q --suppress-config-log -l /var/log/snort/snort_em335190 --pid-path /var/run --nolock-pidfile -G 35190 -c root 95276 0.0 0.0 11912 5176 - IC 19:09 0:00.01 /usr/local/libexec/sshg-parser
root 335 0.0 0.2 94232 38652 - I 19:09 0:08.19 php-fpm: pool nginx (php-fpm)
root 334 0.0 0.2 93972 25148 - Ss 19:09 0:00.03 php-fpm: master process (/usr/local/lib/php-fpm.conf) (php-fpm)
root 336 0.0 0.2 94168 39032 - I 19:09 0:05.05 php-fpm: pool nginx (php-fpm)
root 58908 0.0 0.2 94168 38668 - I 19:10 0:02.14 php-fpm: pool nginx (php-fpm)
dhcpd 66481 0.0 0.0 12580 7748 - Ss 19:09 0:00.05 /usr/local/sbin/dhcpd -user dhcpd -group _dhcp -chroot /var/dhcpd -cf /etc/dhcpd.conf -pf /var/run/dhcpd.pid em1 em2 em3
unbound 54020 0.0 0.2 55208 35172 - Is 19:09 0:00.71 /usr/local/sbin/unbound -c /var/unbound/unbound.conf
root 59343 0.0 0.0 21548 7024 - Is 19:09 0:00.00 nginx: master process /usr/local/sbin/nginx -c /var/etc/nginx-webConfigurator.conf (nginx)
root 59421 0.0 0.0 21548 7500 - I 19:09 0:00.00 nginx: worker process (nginx) -
Have you changed any of the defaults in the Snort GUI? In particular, have you changed the Fast Multipattern Matcher algorithm? It should really never be touched.
Are you sure Snort is the culprit? Do you have any other packages installed on the box?
Unless you have changed the pattern matcher algorithm from the default value, it would be extremely rare for Snort to consume 16 GB of RAM and cause you to utilize swap.
Have you used the FreeBSD CLI tools to see which running processes are consuming RAM and how much each is using?
The only other scenario that could cause Snort to consume that much RAM would be if you had a large number of configured interfaces with all of them running very large enabled rule sets.
-
As I stated before Snort is running the latest version and pfsense is running the latest stable version. I only had my wan interface turn on with the level of security set to balanced. As shown above you see snort consuming 74.7% of 16GB memory. That ps command is sorted by memory utilization with the -m option.
-
@lugwitz said in snort using high amount of ram crashing pfsense:
As I stated before Snort is running the latest version and pfsense is running the latest stable version. I only had my wan interface turn on with the level of security set to balanced. As shown above you see snort consuming 74.7% of 16GB memory. That ps command is sorted by memory utilization with the -m option.
What you posted is non-formatted and a little difficult to read. I quickly glanced at it and thought is was just a log snippet to be honest. When you post things like command outputs it makes it easier to see if you use the </> tags to wrap the text so that the forum software correctly formats it as machine output.
Anyway, I still need an answer to the question about whether or not you changed the Fast Pattern Matcher algorithm selection or not. That will definitely cause Snort to eat memory. The FPM should be set to AC-BNFA or AC-BNFA-HQ and nothing else. Other than that I can't imagine what the issue might be as a single interface, even if you enabled every rule possible, is unlikely to consume that much RAM.
I have an SG-5100 with Snort running on the LAN using the IPS-Balanced security policy and my memory usage is 14% of 4GB (and that's for every running process including Snort).
-
I had ACS pattern matching. I don't remember why I did that. I'm trying to switch it to the default for now.
-
@lugwitz said in snort using high amount of ram crashing pfsense:
I had ACS pattern matching.
Bingo! There you go. Don't mess with the pattern matcher. It is super picky and to be honest, pretty much useless in terms of using the other options. I have been tempted to remove them over the years but left them in place because the underlying binary supports them. But everyone here that has ever changed the value from the default gets themselves into an "out of RAM" situation.
-
@lugwitz said in snort using high amount of ram crashing pfsense:
I don't remember why I did that. I'm trying to switch it to the default for now.
When you change it back, you will have to restart Snort, of course.
-
It's weird that setting supposed to be low memory.
https://forum.netgate.com/topic/4753/faq-into-snort-modes/6
-
@lugwitz said in snort using high amount of ram crashing pfsense:
It's weird that setting supposed to be low memory.
https://forum.netgate.com/topic/4753/faq-into-snort-modes/6
Yeah, but the reality has never matched what the old docs say. Folks have posted here in the past (if I am recalling correctly) with 32 GB RAM boxes being brought to their knees with changes to the pattern matcher algorithm.
-
Thank you for the help! It's must appreciated.