Snort 2.9.4.1 pkg v.2.5.8

dwood

In this release the icons were reverted to match the style elsewhere in pfSense. Namely "green" now means RUNNING and "red" means STOPPED. The tooltip text was changed to reflect this when hovering over the start/stop icons on the Snort GUI page. Is that perhaps what you are seeing?

Bill

Makes sense..thanks Bill

bmeeks

@ccb056:

Great work Bill and Marcello!! Just updated my install.

Does this latest version fix the whitelist/FQDN problem??

It fixes the problem with Locally-Attached networks not being in the default whitelist, and it fixes the issue of the entire WAN subnet being whitelisted. It does not allow the use of FQDN Aliases. That is still on the TO-DO list.

Bill

bmeeks

@gogol:

Thanx again guys for a great update.

For now I noticed that nothing was blocked. After some investigation it seems that:

Snort generates an alert according to my settings
20 seconds thereafter I get this in my logs:
May 31 13:16:52	check_reload_status: updating dyndns WAN_DHCP
May 31 13:16:52	check_reload_status: Restarting ipsec tunnels
May 31 13:16:52	check_reload_status: Restarting OpenVPN tunnels/interfaces
May 31 13:16:52	check_reload_status: Reloading filter
May 31 13:16:55	php: : phpDynDNS (xxx.xxx.xxx): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
And then 12 seconds later:
May 31 13:17:07	check_reload_status: updating dyndns WAN_DHCP
May 31 13:17:07	check_reload_status: Restarting ipsec tunnels
May 31 13:17:07	check_reload_status: Restarting OpenVPN tunnels/interfaces
May 31 13:17:07	check_reload_status: Reloading filter
May 31 13:17:11	php: : phpDynDNS (xxx.xxx.xxx): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
So check_reload_status is called twice and "snort2c" table gets cleared.

Edit: to be clear, this happens after every generated alert.

I guess Bill knows an answer ;)

What kind of ISP connection do you have? Is this a home network or a commercial one? I'm asking because the error messages seem to indicate a DHCP lease expired and was auto-renewed. I believe under pfSense that triggers all the restart activity you see. Snort itself is not even capable of restarting all those services. You are correct that other pfSense processes (in particular, a firewall service restart or reload) will clear the snort2c blocking table. Again, Snort itself has no control over that.

Bill

gogol

@bmeeks:

@gogol:
Thanx again guys for a great update.

For now I noticed that nothing was blocked. After some investigation it seems that:

Snort generates an alert according to my settings
20 seconds thereafter I get this in my logs:
May 31 13:16:52	check_reload_status: updating dyndns WAN_DHCP
May 31 13:16:52	check_reload_status: Restarting ipsec tunnels
May 31 13:16:52	check_reload_status: Restarting OpenVPN tunnels/interfaces
May 31 13:16:52	check_reload_status: Reloading filter
May 31 13:16:55	php: : phpDynDNS (xxx.xxx.xxx): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
And then 12 seconds later:
May 31 13:17:07	check_reload_status: updating dyndns WAN_DHCP
May 31 13:17:07	check_reload_status: Restarting ipsec tunnels
May 31 13:17:07	check_reload_status: Restarting OpenVPN tunnels/interfaces
May 31 13:17:07	check_reload_status: Reloading filter
May 31 13:17:11	php: : phpDynDNS (xxx.xxx.xxx): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
So check_reload_status is called twice and "snort2c" table gets cleared.

Edit: to be clear, this happens after every generated alert.

I guess Bill knows an answer ;)
What kind of ISP connection do you have? Is this a home network or a commercial one? I'm asking because the error messages seem to indicate a DHCP lease expired and was auto-renewed. I believe under pfSense that triggers all the restart activity you see. Snort itself is not even capable of restarting all those services. You are correct that other pfSense processes (in particular, a firewall service restart or reload) will clear the snort2c blocking table. Again, Snort itself has no control over that.

Bill

Bill,

I reverted to an old backup of 2.1 RC0 of May 28 and everything was back to normal with version 2.5.7. Previously I updated to version 2.5.8 while doing a firmware update.
So now I only updated the Snort package to version 2.5.8 and same problems happened again. In another thread I read about someone who has problems with NIC. Maybe this is related, but I definitely think that something in the 2.5.8 code is causing this behavior.
I also tested with disabled dynamic DNS, but that made no difference.
I will report back and do some more reading and testing, because I really appreciate your efforts with this package and I want it to be a very stable package.

I believe the process check_reload_status is checking the /tmp directory, so maybe it is happening there.

bmeeks

@gogol:

I believe the process check_reload_status is checking the /tmp directory, so maybe it is happening there.

I just updated a 2.1 virtual machine I have from 2.1-BETA to 2.1RC0. I'm using Snort with the "IPS Connectivity" policy enabled with no problems. The WAN interface is set to get its IP address via DHCP. This VM is running on VMware Workstation 9. So far I am not seeing any issues with the VM, but I will leave it running to check it out thoroughly.

As for the "check_reload_status" script, that is a native pfSense binary that does a number of things. I am not sure what exactly it does. One of the Core Team developers might could tell you. The "check_reload_status" is not part of the Snort package.

Bill

bmeeks

@jflsakfja:

Since there is no other thread about v2.5.8, I'll go ahead and post a bug in here.

Selecting edit interface>rules> and under category, IPS Policy - Security takes me to https://xxx/snort/snort_rules.php?id=0&openruleset=IPS%20Policy%20-%20Security (xxx added to hide ip and port), which is a completely blank page.

TIA

Have you actually downloaded the Snort VRT rule set? If you enable the Snort VRT rules on the Global Settings tab, then you can select a policy on the Rules tab to view. However, if you did not actually go to the Updates tab and download the Snort VRT rule set, then there will be nothing to display on the Rules tab and thus you get the blank page.

I just tested again on two of my boxes to be sure it works, and it does. I do not get the blank page. Double-check and be sure you actually have downloaded the Snort VRT rules. Go to the Updates tab and try to refresh them. Also, look first on that tab for the MD5 hash for the Snort VRT MD5 file. If it is currently blank, then you have not downloaded the rule set.

Bill

A Former User

VRT was enabled and downloaded. Worked OK before the update and md5 shows up in updates tab. Same thing on both of my boxes.
Forced an update and it found an update to VRT rules (although I'm pretty sure it passed through 2 auto updates so far…). Checked IPS Policy - Security list and it comes up as blank. Restarted snort through dashboard, restarted the interface after it finished and it's still blank.

Went into the interface setttings and selected Balanced as the policy. Restarted the Interface and the list was populated as it should. Went back and changed it to security and restarted the interface yet again and it's still blank.

Edit:
To define blank, here's the html code of the page that comes up:

ccb056

@jflsakfja:

VRT was enabled and downloaded. Worked OK before the update and md5 shows up in updates tab. Same thing on both of my boxes.
Forced an update and it found an update to VRT rules (although I'm pretty sure it passed through 2 auto updates so far…). Checked IPS Policy - Security list and it comes up as blank. Restarted snort through dashboard, restarted the interface after it finished and it's still blank.

Went into the interface setttings and selected Balanced as the policy. Restarted the Interface and the list was populated as it should. Went back and changed it to security and restarted the interface yet again and it's still blank.

Edit:
To define blank, here's the html code of the page that comes up:

That html is pretty malformed, within the …..

Have you tried completely uninstalling snort and re-installing?
I've been doing all my upgrades this way since before bmeeks took over because there were so many problems with the other programmer on updates, this was the best way to do them.

A Former User

I had an error in writing down the code since I couldn't copy it directly. Fixed in my post above. Looks normal but empty from my point of view. I'll try a reinstall and report back

Removing the package and re-installing didn't help (had "settings will not be removed during reinstall" checked, since I would basically have to fire myself first thing Monday morning if those settings got lost).

What's the difference between the 3 categories? Does it just enable/disable rules in the list? Will selecting balance and enable rules make it "security" in a way? If that's the case I'll go with balanced since I'm enabling all rules by hand anyway (that's why I requested the carp sync and fell in love with it).

edit: nothing seems to fix it. Tried disabling and enabling everything I could think of, reinstalling snort and even the tried and tested method of threatening the pc with a short flight out of the window didn't work (don't ask, it works everytime).
Can someone please tell me which file to check for corruption? I'm thinking either the rules don't get loaded into the file they should, or the file that gets read to display the page is messed up (not sure if they are the same file). A bit of info, when I select balanced all my yellow rules are there. Why it does not carry over to the security one is beyond me.
If I'm not making sense it's because I've been up all night messing with it. I'm not frustrated at anyone, don't get me wrong. I really appreciate all your help and if I wasn't in such a tight financial situation I would have donated money already.
If anyone needs more info, please let me know. If I don't respond it means I fell asleep on the keyboard.
TIA

Nukama

Hey Bill,

thanks for this great package!

The alert tab in this version overlaps the IPv6 addresses over columns.

gogol

@bmeeks:

@gogol:

I believe the process check_reload_status is checking the /tmp directory, so maybe it is happening there.

I just updated a 2.1 virtual machine I have from 2.1-BETA to 2.1RC0. I'm using Snort with the "IPS Connectivity" policy enabled with no problems. The WAN interface is set to get its IP address via DHCP. This VM is running on VMware Workstation 9. So far I am not seeing any issues with the VM, but I will leave it running to check it out thoroughly.

As for the "check_reload_status" script, that is a native pfSense binary that does a number of things. I am not sure what exactly it does. One of the Core Team developers might could tell you. The "check_reload_status" is not part of the Snort package.

I tested a Virtual Machine which gave me the same errors. I even installed a fresh system on this VM and restored a configuration file. Same result. Both on WAN DHCP. I even compared both configuration files of 2.5.7 and 2.5.8 but could not see a problem there.

So, package 2.5.7 works on both systems and package 2.5.8 does not. I really like the options you made in 2.5.8 but for now I can't use it. Do you think there a way I can update the firmware and install the old package 2.5.7?

I hope there will be others that have the same experience ;)

Supermule

I dont see that at all here.

2.5.8 works like a charm.

bmeeks

@jflsakfja:

VRT was enabled and downloaded. Worked OK before the update and md5 shows up in updates tab. Same thing on both of my boxes.
Forced an update and it found an update to VRT rules (although I'm pretty sure it passed through 2 auto updates so far…). Checked IPS Policy - Security list and it comes up as blank. Restarted snort through dashboard, restarted the interface after it finished and it's still blank.

Went into the interface setttings and selected Balanced as the policy. Restarted the Interface and the list was populated as it should. Went back and changed it to security and restarted the interface yet again and it's still blank.

How much memory do you have in the box, and are you waiting for the process to completely finish? I ask these questions because the Security Policy is very large and it takes several seconds to read and then construct the table via PHP for the web page. I have tested all three Snort IPS Policies in my 2.1RC0 virtual machine, and they all work. Balanced and Security take longer to populate than Connectivity, with Security taking the longer by far. Attached is a screenshot showing the IPS Security policy rules being displayed.

One other question – what browser are you using? I've done the majority of my testing with Internet Explorer 10, but also checked things out with the latest Firefox and Chrome browsers.

IPS-SecurityPolicy.jpg_thumb

bmeeks

@gogol:

I tested a Virtual Machine which gave me the same errors. I even installed a fresh system on this VM and restored a configuration file. Same result. Both on WAN DHCP. I even compared both configuration files of 2.5.7 and 2.5.8 but could not see a problem there.

So, package 2.5.7 works on both systems and package 2.5.8 does not. I really like the options you made in 2.5.8 but for now I can't use it. Do you think there a way I can update the firmware and install the old package 2.5.7?

I hope there will be others that have the same experience ;)

There was a change in the 2.5.8 code in the way it selected the gateways and WAN IPs to add to the default Whitelist and HOME_NET collections, but I can't for the life of me see how that could translate to what you are seeing. Not saying it can't be related, but how it might be related escapes me at the moment. I will keep testing my own VMs to see if I can replicate this.

As for running 2.5.7 on the new firmware, that is relatively easy to pull off if you are willing to copy some files around. Here is what to do:

Rollback to your 2.5.7 setup and copy the following files off to a set of directories in /tmp.

First, create the following directories in /tmp: www and pkg.

Next, navigate to /usr/local/pkg/snort and copy the files there to the /tmp/pkg directory created earlier.

Repeat the step above except the source directory is /usr/local/www/snort and the destination is /tmp/www.

Now update the firmware. This will also update Snort to 2.5.8, but that is OK. After the reboot from the update and the package reinstallation completes, copy the saved files back to their original locations. Copy the /tmp/pkg files to /usr/local/pkg/snort, and copy the /tmp/www files to /usr/local/www/snort.

Stop and restart Snort, and you should have 2.5.7 running on the new firmware. It will say "2.5.8" on the Packages screens because that is what PBI thinks it installed, but the header on the actual Snort pages should say 2.5.7.

Bill

bmeeks

@Nukama:

Hey Bill,

thanks for this great package!

The alert tab in this version overlaps the IPv6 addresses over columns.

Hmm…I am having problems getting the various browsers to line-break properly. Thank you for the feedback on this problem.

The issue with the Alerts tab is needing to display more information than there really is available column width. Will take another crack at it. Have not found a way yet to signal the browser to break on say the colon ":" in the IPv6 address. If anyone has a suggestion, I'm all ears.

Bill

A Former User

@bmeeks:

How much memory do you have in the box, and are you waiting for the process to completely finish? I ask these questions because the Security Policy is very large and it takes several seconds to read and then construct the table via PHP for the web page. I have tested all three Snort IPS Policies in my 2.1RC0 virtual machine, and they all work. Balanced and Security take longer to populate than Connectivity, with Security taking the longer by far. Attached is a screenshot showing the IPS Security policy rules being displayed.

I have 2GB ram in those boxes, and RAM never gets above 40% (on the days they get hammered, usually around 30%).
Waiting for it doesn't help either.It's not that the browser is waiting for the page, it gets the page, but it's a completely blank page (see a couple of my posts back for the html code).
I'm enabling all rules anyway, is there a difference between balanced and security after doing that, or does security load something external (something other that the rules listed on the rule enable/disable page)?

bmeeks

@jflsakfja:

I have 2GB ram in those boxes, and RAM never gets above 40% (on the days they get hammered, usually around 30%).
Waiting for it doesn't help either.It's not that the browser is waiting for the page, it gets the page, but it's a completely blank page (see a couple of my posts back for the html code).
I'm enabling all rules anyway, is there a difference between balanced and security after doing that, or does security load something external (something other that the rules listed on the rule enable/disable page)?

The three policies enable different sets of rules. These are pre-defined by the Snort VRT. The policy membership of a given rule is contained within a metadata statement in the text. Use grep and search the Snort rules files for these phrases to see which rules belong to which policy:

policy connectivity-ips, policy balanced-ips, and policy security-ips

Many of the rules containing one of these metadata values will be disabled by default (have the semicolon in front of the text). What Snort does when you select a policy is find all the rules containing the target policy in their metadata, then it enables those rules by removing any semicolons if present. So selecting all the Rule Categories on the Categories tab will only get you part of the way there. You will still have to individually enable the commented-out rules. That's why using the policy option in the drop-down is better. I am really puzzled why it is not working for you.

As one last troubleshooting attempt, have you tried force flushing the cache on your browser? Or how about a forced refresh while the blank page is displayed?

Bill

Supermule

This is exactly why I wanted the widescreen package back. It needs to be able to scale the width of any column.

A Former User

@bmeeks:

The three policies enable different sets of rules. These are pre-defined by the Snort VRT. The policy membership of a given rule is contained within a metadata statement in the text. Use grep and search the Snort rules files for these phrases to see which rules belong to which policy:

policy connectivity-ips, policy balanced-ips, and policy security-ips

Many of the rules containing one of these metaata values will be disabled by default (have the semicolon in front of the text). What Snort does when you select a policy is find all the rules containing the target policy in their metadata, then it enables those rules by removing any semicolons if present. So selecting all the Rule Categories on the Categories tab will only get you part of the way there. You will still have to individually enable the commented-out rules. That's why using the policy option in the drop-down is better. I am really puzzled why it is not working for you.

As one last troubleshooting attempt, have you tried force flushing the cache on your browser? Or how about a forced refresh while the blank page is displayed?

Bill

Already tried forcing refresh, clearing cache, restarting browser… nothing. It was working absolutely perfect before the update, but I'm not going back, carp sync is more important.
Is there anyway to force snort to rewrite the "security" part? like delete a file and restart or something? Deselecting VRT rules, restarting snort and then reenabling them doesn't work.

Supermule

Deinstall, reboot, install and see if it goes away.