Snort Behavior

DefenderLLC

@defenderllc There's no need, the patches are for pfSense not packages. There was another update to the package a week or so ago, however, with other patches.

It could be the ARC memory cache then, especially if you have a lot of disk activity for updates.

All of the updates were stabilized before I started effing with Snort. I was just too aggressive with my rulesets. The spike occurred when there was an influx of SIP traffic on Monday morning. I just thought it was weird that blocking was still happening on even after I deleted the Snort interfaces. Alerts and blocking kept coming back until I rebooted and the service remained stopped. I'll start over now that things have returned to normal.

Thanks for the prompt responses.

SteveITS

@defenderllc said in Snort Behavior:

Alerts and blocking kept coming back until I rebooted and the service remained stopped

I've seen posts about that, if there is an extra/zombie Snort process that is still running. That could also explain extra memory usage. It's not supposed to happen but does apparently in some circumstances. Search for "bmeeks" and "zombie" or similar.

bmeeks

@defenderllc said in Snort Behavior:

I just thought it was weird that blocking was still happening on even after I deleted the Snort interfaces. Alerts and blocking kept coming back until I rebooted and the service remained stopped.

A common misconception among Snort and Suricata users is that if they stop the service "all blocks are removed". That is not correct when Legacy Blocking Mode is enabled (and that mode is the default).

Legacy Blocking Mode works by putting the IP addresses to be blocked into an internal pf (packet filter) firewall table called snort2c. That table is created by pfSense at boot up and lives in RAM. Addresses placed into that table will continue to be blocked by a hidden firewall rule until the addresses are removed from the snort2c table. The addresses are NOT removed by simply stopping the Snort process. You must manually clear the table using GUI tools. Rebooting the firewall will also result in clearing the table because the table lives in RAM and will be wiped out with the reboot.

It is also possible, under rare circumstances, for duplicate Snort instances to get started on the same physical interface. The GUI will lose track of the first instance if a second one starts. You won't be able to control the now "zombie" first instance nor will that instance respond to any changes made in the GUI. You can manually kill the zombie instance, or you can reboot the firewall to remove it.

DefenderLLC

@bmeeks said in Snort Behavior:

@defenderllc said in Snort Behavior:

I just thought it was weird that blocking was still happening on even after I deleted the Snort interfaces. Alerts and blocking kept coming back until I rebooted and the service remained stopped.

A common misconception among Snort and Suricata users is that if they stop the service "all blocks are removed". That is not correct when Legacy Blocking Mode is enabled (and that mode is the default).

Legacy Blocking Mode works buy putting the IP addresses to be blocked into an internal pf (packet filter) firewall table called snort2c. That table is created by pfSense at boot up and lives in RAM. Addresses placed into that table will continue to be blocked by a hidden firewall rule until the addresses are removed from the snort2c table. The addresses are NOT removed by simply stopping the Snort process. You must manually clear the table using GUI tools. Rebooting the firewall will also result in clearing the table because the table lives in RAM will be wiped out with the reboot.

Thanks for the clarification. I kept deleting the blocks from the Snort UI and they just kept coming back. I did not know about the hidden rule. Which table were you referring to?

All is well now though. I will start over and gradually add more rulesets.

bmeeks

@defenderllc said in Snort Behavior:

@bmeeks said in Snort Behavior:

@defenderllc said in Snort Behavior:

I just thought it was weird that blocking was still happening on even after I deleted the Snort interfaces. Alerts and blocking kept coming back until I rebooted and the service remained stopped.

A common misconception among Snort and Suricata users is that if they stop the service "all blocks are removed". That is not correct when Legacy Blocking Mode is enabled (and that mode is the default).

Legacy Blocking Mode works buy putting the IP addresses to be blocked into an internal pf (packet filter) firewall table called snort2c. That table is created by pfSense at boot up and lives in RAM. Addresses placed into that table will continue to be blocked by a hidden firewall rule until the addresses are removed from the snort2c table. The addresses are NOT removed by simply stopping the Snort process. You must manually clear the table using GUI tools. Rebooting the firewall will also result in clearing the table because the table lives in RAM will be wiped out with the reboot.

Thanks for the clarification. I kept deleting the blocks from the Snort UI and they just kept coming back. I did not know about the hidden rule. Which table were you referring to?

All is well now though. I will start over and gradually add more rulesets.

You can't see the hidden rule in the GUI. It is created by pfSense during boot. You can see a text file dump of all the rules, including the hidden ones, by examining /tmp/rules.debug on the firewall.

If you had traffic repeatedly hitting a given rule, then simply clearing the block is not enough as the next packet through will simply trigger another block. In that instance, you should consider either disabling or suppressing that rule using the GUI icons on the ALERTS tab.

You can see all the pf tables under DIAGNOSTICS > TABLES in the pfSense menu.

SteveITS

@bmeeks said in Snort Behavior:

the next packet through will simply trigger another block

He's saying it was triggering alerts while the service/interface was showing stopped in the GUI. Hence my comment about a zombie process.

DefenderLLC

@steveits said in Snort Behavior:

@bmeeks said in Snort Behavior:

the next packet through will simply trigger another block

He's saying it was triggering alerts while the service/interface was showing stopped in the GUI. Hence my comment about a zombie process.

It was still blocking with the interfaces completely deleted from Snort. I had to reboot to get this to stop.

bmeeks

@steveits said in Snort Behavior:

@bmeeks said in Snort Behavior:

the next packet through will simply trigger another block

He's saying it was triggering alerts while the service/interface was showing stopped in the GUI. Hence my comment about a zombie process.

If he deleted the interfaces, there would not have anything to see under the ALERTS tab as with no defined interfaces nothing would show up.

What some users do is leave the setting for "log default firewall rules" set to "on". Then, when looking at the firewall logs, they can see snort2c blocks showing up. Perhaps that is what the user was doing ??

bmeeks

@defenderllc said in Snort Behavior:

@steveits said in Snort Behavior:

@bmeeks said in Snort Behavior:

the next packet through will simply trigger another block

He's saying it was triggering alerts while the service/interface was showing stopped in the GUI. Hence my comment about a zombie process.

It was still still blocking with the interfaces completely deleted from Snort. I had to reboot to get this to stop.

You should look for Snort alerts under the ALERTS tab of the Snort GUI. You can see blocks under the BLOCKS tab.

DefenderLLC

@bmeeks said in Snort Behavior:

@defenderllc said in Snort Behavior:

@steveits said in Snort Behavior:

@bmeeks said in Snort Behavior:

the next packet through will simply trigger another block

He's saying it was triggering alerts while the service/interface was showing stopped in the GUI. Hence my comment about a zombie process.

It was still still blocking with the interfaces completely deleted from Snort. I had to reboot to get this to stop.

You should look for Snort alerts under the ALERTS tab of the Snort GUI. You can see blocks under the BLOCKS tab.

Here is what was happening. An excessive amount of SIP traffic triggered a bunch of blocks that pushed memory utilization to nearly 100% and the firewall was inaccessible for about 10 minutes.

I was finally able to get in via console cable and then eventually the GUI. The Snort LAN interface was stopped which was fine with me so I could make some adjustments. The memory remained at nearly 70% and I was super busy with work, so I tried to stop the bleeding by simply stopping the service. That did not work because it would just restart within seconds even without it being configured in Service Watchdog.

The next step was just to delete all of the interfaces in Snort, clear all alerts and blocks... I did this over and over and it did not matter because they just kept coming back despite a lack of any interfaces configured. Memory utilization was still high.

A reboot was what it took to clear everything - include the memory utilization. #LessonLearned