Snort - not starting anymore

Guest

It's not the download alone (between 16 and 100 Mbit/s download), afterwards the building of sig maps for all three interfaces et. pp…. The hardware is dualcore but not the very latest, so overall it's about half an hour till then snort instances are stopped and finally restarted.

Here an example from another box:


Nov 11 13:03:05 domain php: snort_check_for_rule_updates.php: [Snort] Snort VRT rules are up to date...
Nov 11 13:03:08 domain php: snort_check_for_rule_updates.php: [Snort] There is a new set of Snort GPLv2 Community Rules posted. Downloading community-rules.tar.gz...
Nov 11 13:03:11 domain php: snort_check_for_rule_updates.php: [Snort] Snort GPLv2 Community Rules file update downloaded successfully
Nov 11 13:03:12 domain php: snort_check_for_rule_updates.php: [Snort] There is a new set of Emerging Threats Open rules posted. Downloading emerging.rules.tar.gz...
Nov 11 13:03:16 domain php: snort_check_for_rule_updates.php: [Snort] Emerging Threats Open rules file update downloaded successfully
Nov 11 13:03:52 domain php: snort_check_for_rule_updates.php: [Snort] Updating rules configuration for: WAN ...
Nov 11 13:04:17 domain kernel: pid 46476 (snort), uid 0, was killed: out of swap space
Nov 11 13:04:17 domain kernel: re2: promiscuous mode disabled
Nov 11 13:08:57 domain php: snort_check_for_rule_updates.php: [Snort] Enabling any flowbit-required rules for: WAN...
Nov 11 13:09:21 domain php: snort_check_for_rule_updates.php: [Snort] Building new sig-msg.map file for WAN...
Nov 11 13:13:50 domain php: snort_check_for_rule_updates.php: [Snort] Updating rules configuration for: LAN ...
Nov 11 13:19:27 domain php: snort_check_for_rule_updates.php: [Snort] Enabling any flowbit-required rules for: LAN...
Nov 11 13:20:00 domain php: snort_check_for_rule_updates.php: [Snort] Building new sig-msg.map file for LAN...
Nov 11 13:25:35 domain php: snort_check_for_rule_updates.php: [Snort] Updating rules configuration for: OPT1 ...
Nov 11 13:31:04 domain php: snort_check_for_rule_updates.php: [Snort] Enabling any flowbit-required rules for: OPT1...
Nov 11 13:31:28 domain php: snort_check_for_rule_updates.php: [Snort] Building new sig-msg.map file for OPT1...
Nov 11 13:36:26 domain SnortStartup[12651]: Snort STOP for interface(50118_pppoe0)...
Nov 11 13:36:26 domain snort[3398]: *** Caught Term-Signal
Nov 11 13:36:26 domain kernel: pppoe0: promiscuous mode disabled
Nov 11 13:36:31 domain SnortStartup[14732]: Snort STOP for LAN(12330_re2)...
Nov 11 13:36:33 domain SnortStartup[15615]: Snort STOP for OPT1 (54662_re0)...
Nov 11 13:36:34 domain snort[47686]: *** Caught Term-Signal
Nov 11 13:36:34 domain kernel: re0: promiscuous mode disabled
Nov 11 13:36:47 domain php: snort_check_for_rule_updates.php: [Snort] Snort has restarted with your new set of rules...
Nov 11 13:36:47 domain php: snort_check_for_rule_updates.php: [Snort] The Rules update has finished.
Nov 11 13:36:47 domain SnortStartup[42078]: Snort START for interface(50118_pppoe0)...
Nov 11 13:36:50 domain check_reload_status: Syncing firewall
Nov 11 13:38:11 domain kernel: pppoe0: promiscuous mode enabled
Nov 11 13:38:14 domain SnortStartup[67697]: Snort START for LAN(12330_re2)...
Nov 11 13:40:57 domain kernel: re2: promiscuous mode enabled
Nov 11 13:40:59 domain SnortStartup[29775]: Snort START for OPT1 (54662_re0)...
Nov 11 13:42:35 domain kernel: re0: promiscuous mode enabled

bmeeks

OK, I found the problem. It's not what I initially thought it was. It had nothing to do with moved code. Instead, a version string did not get updated in the last release. That string is really not necessary any more, so I will remove it in the next update. For you guys with RAM disks for /tmp and /var, including those running on Nano installs, here is a quick fix you can do yourself if you want to:

1. Go to Diagnostics…Edit File in the pfSense menu and then browse to and open the file /usr/local/pkg/snort.xml:

2. Scroll down to near the bottom of the file and find this section of text


	 <custom_php_resync_config_command>if ($GLOBALS['pfSense_snort_version'] == "3.1.3")
		sync_snort_package_config();
		]]></custom_php_resync_config_command>

3. Edit it so it looks like this and then save the change:


	 <custom_php_resync_config_command>sync_snort_package_config();
		]]></custom_php_resync_config_command>

Notice that the entire line containing the "if()" statement was removed.

This should fix the problem of Snort failing to restart on a reboot. I will soon post a fix for the pfSense developers to merge to production, but you can make the edit above yourself if need the fix sooner.

EDIT: here is the posted Pull Request for review by the pfSense developers.

https://github.com/pfsense/pfsense-packages/pull/725

Bill

bmeeks

@chemlud:

It's not the download alone (between 16 and 100 Mbit/s download), afterwards the building of sig maps for all three interfaces et. pp…. The hardware is dualcore but not the very latest, so overall it's about half an hour till then snort instances are stopped and finally restarted.

Here an example from another box:


Nov 11 13:03:05 domain php: snort_check_for_rule_updates.php: [Snort] Snort VRT rules are up to date...
Nov 11 13:03:08 domain php: snort_check_for_rule_updates.php: [Snort] There is a new set of Snort GPLv2 Community Rules posted. Downloading community-rules.tar.gz...
Nov 11 13:03:11 domain php: snort_check_for_rule_updates.php: [Snort] Snort GPLv2 Community Rules file update downloaded successfully
Nov 11 13:03:12 domain php: snort_check_for_rule_updates.php: [Snort] There is a new set of Emerging Threats Open rules posted. Downloading emerging.rules.tar.gz...
Nov 11 13:03:16 domain php: snort_check_for_rule_updates.php: [Snort] Emerging Threats Open rules file update downloaded successfully
Nov 11 13:03:52 domain php: snort_check_for_rule_updates.php: [Snort] Updating rules configuration for: WAN ...
Nov 11 13:04:17 domain kernel: pid 46476 (snort), uid 0, was killed: out of swap space
Nov 11 13:04:17 domain kernel: re2: promiscuous mode disabled
Nov 11 13:08:57 domain php: snort_check_for_rule_updates.php: [Snort] Enabling any flowbit-required rules for: WAN...
Nov 11 13:09:21 domain php: snort_check_for_rule_updates.php: [Snort] Building new sig-msg.map file for WAN...
Nov 11 13:13:50 domain php: snort_check_for_rule_updates.php: [Snort] Updating rules configuration for: LAN ...
Nov 11 13:19:27 domain php: snort_check_for_rule_updates.php: [Snort] Enabling any flowbit-required rules for: LAN...
Nov 11 13:20:00 domain php: snort_check_for_rule_updates.php: [Snort] Building new sig-msg.map file for LAN...
Nov 11 13:25:35 domain php: snort_check_for_rule_updates.php: [Snort] Updating rules configuration for: OPT1 ...
Nov 11 13:31:04 domain php: snort_check_for_rule_updates.php: [Snort] Enabling any flowbit-required rules for: OPT1...
Nov 11 13:31:28 domain php: snort_check_for_rule_updates.php: [Snort] Building new sig-msg.map file for OPT1...
Nov 11 13:36:26 domain SnortStartup[12651]: Snort STOP for interface(50118_pppoe0)...
Nov 11 13:36:26 domain snort[3398]: *** Caught Term-Signal
Nov 11 13:36:26 domain kernel: pppoe0: promiscuous mode disabled
Nov 11 13:36:31 domain SnortStartup[14732]: Snort STOP for LAN(12330_re2)...
Nov 11 13:36:33 domain SnortStartup[15615]: Snort STOP for OPT1 (54662_re0)...
Nov 11 13:36:34 domain snort[47686]: *** Caught Term-Signal
Nov 11 13:36:34 domain kernel: re0: promiscuous mode disabled
Nov 11 13:36:47 domain php: snort_check_for_rule_updates.php: [Snort] Snort has restarted with your new set of rules...
Nov 11 13:36:47 domain php: snort_check_for_rule_updates.php: [Snort] The Rules update has finished.
Nov 11 13:36:47 domain SnortStartup[42078]: Snort START for interface(50118_pppoe0)...
Nov 11 13:36:50 domain check_reload_status: Syncing firewall
Nov 11 13:38:11 domain kernel: pppoe0: promiscuous mode enabled
Nov 11 13:38:14 domain SnortStartup[67697]: Snort START for LAN(12330_re2)...
Nov 11 13:40:57 domain kernel: re2: promiscuous mode enabled
Nov 11 13:40:59 domain SnortStartup[29775]: Snort START for OPT1 (54662_re0)...
Nov 11 13:42:35 domain kernel: re0: promiscuous mode enabled

This error is a little strange:


Nov 11 13:04:17 domain kernel: pid 46476 (snort), uid 0, was killed: out of swap space

How much RAM did you say was in the box?

Bill

Guest

1 GB only. It's the first time I see this ;-) …copied just from the most recent log... Would have to do some research if this happened in the past. Overall this box is normally doing fine. All boxes are not for high-throughput, more for "controlled access areas" with very limited number of users.

bmeeks

@chemlud:

1 GB only. It's the first time I see this ;-) …copied just from the most recent log... Would have to do some research if this happened in the past. Overall this box is normally doing fine. All boxes are not for high-throughput, more for "controlled access areas" with very limited number of users.

This box might be on the ragged edge in terms of RAM with Snort. Depending on the number of enabled rules, Snort can eat a lot of memory. I would recommend bumping this box up to 2GB if possible.

Bill

Guest

no, not possible. Maybe I should remove some rules, especially from the WAN interface, iirc what is frequently recommended…

bmeeks

@chemlud:

no, not possible. Maybe I should remove some rules, especially from the WAN interface, iirc what is frequently recommended…

Yes, you certainly would not want any duplicate rules (same ones on more than one interface) in a box with limited RAM. Also make sure you have the pattern matcher set to AC-BNFA-NQ. That is the most memory and speed efficient setting.

The rules update process can consume up to 200 MB of memory itself on a temporary basis because it needs to hold a lot of data in some in-memory arrays while building the rules files. So the combination of that memory consumption combined with what the Snort binary process is already using could put your 1GB box over the top. That might explain why it sometimes dies during an update. The rules update process will only restart Snort if it is detected as running during the update process.

Bill

Guest

I have AC-BNFA-NQ as standard..

"The rules update process will only restart Snort if it is detected as running during the update process."

That's what I expected, therefore I controlled this box some minutes ago, but all three snort-interfaces were up and running, strange indeed…