Snort 2.9.4.1 pkg v.2.5.8

VRT was enabled and downloaded. Worked OK before the update and md5 shows up in updates tab. Same thing on both of my boxes.
Forced an update and it found an update to VRT rules (although I'm pretty sure it passed through 2 auto updates so far…). Checked IPS Policy - Security list and it comes up as blank. Restarted snort through dashboard, restarted the interface after it finished and it's still blank.

Went into the interface setttings and selected Balanced as the policy. Restarted the Interface and the list was populated as it should. Went back and changed it to security and restarted the interface yet again and it's still blank.

Edit:
To define blank, here's the html code of the page that comes up:

ccb056

@jflsakfja:

VRT was enabled and downloaded. Worked OK before the update and md5 shows up in updates tab. Same thing on both of my boxes.
Forced an update and it found an update to VRT rules (although I'm pretty sure it passed through 2 auto updates so far…). Checked IPS Policy - Security list and it comes up as blank. Restarted snort through dashboard, restarted the interface after it finished and it's still blank.

Went into the interface setttings and selected Balanced as the policy. Restarted the Interface and the list was populated as it should. Went back and changed it to security and restarted the interface yet again and it's still blank.

Edit:
To define blank, here's the html code of the page that comes up:

That html is pretty malformed, within the …..

Have you tried completely uninstalling snort and re-installing?
I've been doing all my upgrades this way since before bmeeks took over because there were so many problems with the other programmer on updates, this was the best way to do them.

I had an error in writing down the code since I couldn't copy it directly. Fixed in my post above. Looks normal but empty from my point of view. I'll try a reinstall and report back

Removing the package and re-installing didn't help (had "settings will not be removed during reinstall" checked, since I would basically have to fire myself first thing Monday morning if those settings got lost).

What's the difference between the 3 categories? Does it just enable/disable rules in the list? Will selecting balance and enable rules make it "security" in a way? If that's the case I'll go with balanced since I'm enabling all rules by hand anyway (that's why I requested the carp sync and fell in love with it).

edit: nothing seems to fix it. Tried disabling and enabling everything I could think of, reinstalling snort and even the tried and tested method of threatening the pc with a short flight out of the window didn't work (don't ask, it works everytime).
Can someone please tell me which file to check for corruption? I'm thinking either the rules don't get loaded into the file they should, or the file that gets read to display the page is messed up (not sure if they are the same file). A bit of info, when I select balanced all my yellow rules are there. Why it does not carry over to the security one is beyond me.
If I'm not making sense it's because I've been up all night messing with it. I'm not frustrated at anyone, don't get me wrong. I really appreciate all your help and if I wasn't in such a tight financial situation I would have donated money already.
If anyone needs more info, please let me know. If I don't respond it means I fell asleep on the keyboard.
TIA

Nukama

Hey Bill,

thanks for this great package!

The alert tab in this version overlaps the IPv6 addresses over columns.

gogol

@bmeeks:

@gogol:

I believe the process check_reload_status is checking the /tmp directory, so maybe it is happening there.

I just updated a 2.1 virtual machine I have from 2.1-BETA to 2.1RC0. I'm using Snort with the "IPS Connectivity" policy enabled with no problems. The WAN interface is set to get its IP address via DHCP. This VM is running on VMware Workstation 9. So far I am not seeing any issues with the VM, but I will leave it running to check it out thoroughly.

As for the "check_reload_status" script, that is a native pfSense binary that does a number of things. I am not sure what exactly it does. One of the Core Team developers might could tell you. The "check_reload_status" is not part of the Snort package.

I tested a Virtual Machine which gave me the same errors. I even installed a fresh system on this VM and restored a configuration file. Same result. Both on WAN DHCP. I even compared both configuration files of 2.5.7 and 2.5.8 but could not see a problem there.

So, package 2.5.7 works on both systems and package 2.5.8 does not. I really like the options you made in 2.5.8 but for now I can't use it. Do you think there a way I can update the firmware and install the old package 2.5.7?

I hope there will be others that have the same experience ;)

Supermule

I dont see that at all here.

2.5.8 works like a charm.

bmeeks

@jflsakfja:

VRT was enabled and downloaded. Worked OK before the update and md5 shows up in updates tab. Same thing on both of my boxes.
Forced an update and it found an update to VRT rules (although I'm pretty sure it passed through 2 auto updates so far…). Checked IPS Policy - Security list and it comes up as blank. Restarted snort through dashboard, restarted the interface after it finished and it's still blank.

Went into the interface setttings and selected Balanced as the policy. Restarted the Interface and the list was populated as it should. Went back and changed it to security and restarted the interface yet again and it's still blank.

How much memory do you have in the box, and are you waiting for the process to completely finish? I ask these questions because the Security Policy is very large and it takes several seconds to read and then construct the table via PHP for the web page. I have tested all three Snort IPS Policies in my 2.1RC0 virtual machine, and they all work. Balanced and Security take longer to populate than Connectivity, with Security taking the longer by far. Attached is a screenshot showing the IPS Security policy rules being displayed.

One other question – what browser are you using? I've done the majority of my testing with Internet Explorer 10, but also checked things out with the latest Firefox and Chrome browsers.

IPS-SecurityPolicy.jpg_thumb

bmeeks

@gogol:

I tested a Virtual Machine which gave me the same errors. I even installed a fresh system on this VM and restored a configuration file. Same result. Both on WAN DHCP. I even compared both configuration files of 2.5.7 and 2.5.8 but could not see a problem there.

So, package 2.5.7 works on both systems and package 2.5.8 does not. I really like the options you made in 2.5.8 but for now I can't use it. Do you think there a way I can update the firmware and install the old package 2.5.7?

I hope there will be others that have the same experience ;)

There was a change in the 2.5.8 code in the way it selected the gateways and WAN IPs to add to the default Whitelist and HOME_NET collections, but I can't for the life of me see how that could translate to what you are seeing. Not saying it can't be related, but how it might be related escapes me at the moment. I will keep testing my own VMs to see if I can replicate this.

As for running 2.5.7 on the new firmware, that is relatively easy to pull off if you are willing to copy some files around. Here is what to do:

Rollback to your 2.5.7 setup and copy the following files off to a set of directories in /tmp.

First, create the following directories in /tmp: www and pkg.

Next, navigate to /usr/local/pkg/snort and copy the files there to the /tmp/pkg directory created earlier.

Repeat the step above except the source directory is /usr/local/www/snort and the destination is /tmp/www.

Now update the firmware. This will also update Snort to 2.5.8, but that is OK. After the reboot from the update and the package reinstallation completes, copy the saved files back to their original locations. Copy the /tmp/pkg files to /usr/local/pkg/snort, and copy the /tmp/www files to /usr/local/www/snort.

Stop and restart Snort, and you should have 2.5.7 running on the new firmware. It will say "2.5.8" on the Packages screens because that is what PBI thinks it installed, but the header on the actual Snort pages should say 2.5.7.

Bill

bmeeks

@Nukama:

Hey Bill,

thanks for this great package!

The alert tab in this version overlaps the IPv6 addresses over columns.

Hmm…I am having problems getting the various browsers to line-break properly. Thank you for the feedback on this problem.

The issue with the Alerts tab is needing to display more information than there really is available column width. Will take another crack at it. Have not found a way yet to signal the browser to break on say the colon ":" in the IPv6 address. If anyone has a suggestion, I'm all ears.

Bill

@bmeeks:

How much memory do you have in the box, and are you waiting for the process to completely finish? I ask these questions because the Security Policy is very large and it takes several seconds to read and then construct the table via PHP for the web page. I have tested all three Snort IPS Policies in my 2.1RC0 virtual machine, and they all work. Balanced and Security take longer to populate than Connectivity, with Security taking the longer by far. Attached is a screenshot showing the IPS Security policy rules being displayed.

I have 2GB ram in those boxes, and RAM never gets above 40% (on the days they get hammered, usually around 30%).
Waiting for it doesn't help either.It's not that the browser is waiting for the page, it gets the page, but it's a completely blank page (see a couple of my posts back for the html code).
I'm enabling all rules anyway, is there a difference between balanced and security after doing that, or does security load something external (something other that the rules listed on the rule enable/disable page)?

bmeeks

@jflsakfja:

I have 2GB ram in those boxes, and RAM never gets above 40% (on the days they get hammered, usually around 30%).
Waiting for it doesn't help either.It's not that the browser is waiting for the page, it gets the page, but it's a completely blank page (see a couple of my posts back for the html code).
I'm enabling all rules anyway, is there a difference between balanced and security after doing that, or does security load something external (something other that the rules listed on the rule enable/disable page)?

The three policies enable different sets of rules. These are pre-defined by the Snort VRT. The policy membership of a given rule is contained within a metadata statement in the text. Use grep and search the Snort rules files for these phrases to see which rules belong to which policy:

policy connectivity-ips, policy balanced-ips, and policy security-ips

Many of the rules containing one of these metadata values will be disabled by default (have the semicolon in front of the text). What Snort does when you select a policy is find all the rules containing the target policy in their metadata, then it enables those rules by removing any semicolons if present. So selecting all the Rule Categories on the Categories tab will only get you part of the way there. You will still have to individually enable the commented-out rules. That's why using the policy option in the drop-down is better. I am really puzzled why it is not working for you.

As one last troubleshooting attempt, have you tried force flushing the cache on your browser? Or how about a forced refresh while the blank page is displayed?

Bill

Supermule

This is exactly why I wanted the widescreen package back. It needs to be able to scale the width of any column.

@bmeeks:

The three policies enable different sets of rules. These are pre-defined by the Snort VRT. The policy membership of a given rule is contained within a metadata statement in the text. Use grep and search the Snort rules files for these phrases to see which rules belong to which policy:

policy connectivity-ips, policy balanced-ips, and policy security-ips

Many of the rules containing one of these metaata values will be disabled by default (have the semicolon in front of the text). What Snort does when you select a policy is find all the rules containing the target policy in their metadata, then it enables those rules by removing any semicolons if present. So selecting all the Rule Categories on the Categories tab will only get you part of the way there. You will still have to individually enable the commented-out rules. That's why using the policy option in the drop-down is better. I am really puzzled why it is not working for you.

As one last troubleshooting attempt, have you tried force flushing the cache on your browser? Or how about a forced refresh while the blank page is displayed?

Bill

Already tried forcing refresh, clearing cache, restarting browser… nothing. It was working absolutely perfect before the update, but I'm not going back, carp sync is more important.
Is there anyway to force snort to rewrite the "security" part? like delete a file and restart or something? Deselecting VRT rules, restarting snort and then reenabling them doesn't work.

Supermule

Deinstall, reboot, install and see if it goes away.

@Supermule:

Deinstall, reboot, install and see if it goes away.

Already tried that, just tried it again with no luck. I'm off to bed for now. Thank you all for your help so far.

Nukama

@bmeeks:

@Nukama:

Hey Bill,

thanks for this great package!

The alert tab in this version overlaps the IPv6 addresses over columns.

Hmm…I am having problems getting the various browsers to line-break properly. Thank you for the feedback on this problem.

The issue with the Alerts tab is needing to display more information than there really is available column width. Will take another crack at it. Have not found a way yet to signal the browser to break on say the colon ":" in the IPv6 address. If anyone has a suggestion, I'm all ears.

Bill

Try the status_dhcpv6_leases.php, it outgrows the border to fit in information.

bmeeks

@jflsakfja:

@Supermule:

Deinstall, reboot, install and see if it goes away.

Already tried that, just tried it again with no luck. I'm off to bed for now. Thank you all for your help so far.

I think you said "Security" was the only policy that did not display correctly. Is that right? So for instance, Connectivity and Balanced display OK?

Is your locale set to display the screens in English or another language?

Do you have more than one interface enabled for Snort? If so, do all interfaces exhibit the same problem?

Sorry to pepper you with questions, but I really want to try and figure out what the problem is.

Edit: Oh, and one other thing you can check to see if the "Security" policy is actually in place. It could just be a display issue. Follow the steps below to investigate.

1. Under Diagnostics choose Edit File.

2. Navigate to /usr/pbi/snort__{arch}_/etc/snort/ (where {arch} is either "amd64" or "i386" depending on your CPU)

3. In the directory will be one or more snort_xxxx_xxx sub-directories. They correspond to each configured Snort interface. If you have Snort only on the WAN, then there will be only one of these additional snort_xxxx_xxx sub-directories. The "xxxx_xxx" will be a random UUID created for the interface followed by the NIC driver name and number. For example, on my VM test box the directory is called snort_33226_em0.

4. Navigate down into the snort_xxxx_xxx directory for the interface giving you the problem. In the directory is a rules directory. Navigate down into it and click the snort.rules rule to view it. It should be full of rule text lines, all uncommented and all containing "policy security-ips" in the metadata. If the file is present and contains the correct rules, then Snort is actually using the policy defined and your problem is purely a display issue. If the snort.rules file is empty, then we definitely have something strange going on.

Bill

gogol

@bmeeks:

@gogol:

I tested a Virtual Machine which gave me the same errors. I even installed a fresh system on this VM and restored a configuration file. Same result. Both on WAN DHCP. I even compared both configuration files of 2.5.7 and 2.5.8 but could not see a problem there.

So, package 2.5.7 works on both systems and package 2.5.8 does not. I really like the options you made in 2.5.8 but for now I can't use it. Do you think there a way I can update the firmware and install the old package 2.5.7?

I hope there will be others that have the same experience ;)

There was a change in the 2.5.8 code in the way it selected the gateways and WAN IPs to add to the default Whitelist and HOME_NET collections, but I can't for the life of me see how that could translate to what you are seeing. Not saying it can't be related, but how it might be related escapes me at the moment. I will keep testing my own VMs to see if I can replicate this.

As for running 2.5.7 on the new firmware, that is relatively easy to pull off if you are willing to copy some files around. Here is what to do:

Rollback to your 2.5.7 setup and copy the following files off to a set of directories in /tmp.

First, create the following directories in /tmp: www and pkg.

Next, navigate to /usr/local/pkg/snort and copy the files there to the /tmp/pkg directory created earlier.

Repeat the step above except the source directory is /usr/local/www/snort and the destination is /tmp/www.

Now update the firmware. This will also update Snort to 2.5.8, but that is OK. After the reboot from the update and the package reinstallation completes, copy the saved files back to their original locations. Copy the /tmp/pkg files to /usr/local/pkg/snort, and copy the /tmp/www files to /usr/local/www/snort.

Stop and restart Snort, and you should have 2.5.7 running on the new firmware. It will say "2.5.8" on the Packages screens because that is what PBI thinks it installed, but the header on the actual Snort pages should say 2.5.7.

Thank you, that worked! Although I see some strange file permissions (077).

I did some further testing. Deinstalled snort without saving configuration and configured it from scratch: same problems.
Then I installed a fresh pfSense RC0 Fri May 31 from memorystick to hard disk and restored configuration file and rebooted: same problems
An alert and then "check_reload_status" as I mentioned before.

I am out of options now (well I can install pfSense system from scratch but then I need a day or so) and need an answer from the developers. Very weird!

bmeeks

@gogol:

I did some further testing. Deinstalled snort without saving configuration and configured it from scratch: same problems.
Then I installed a fresh pfSense RC0 Fri May 31 from memorystick to hard disk and restored configuration file and rebooted: same problems
An alert and then "check_reload_status" as I mentioned before.

Just so I'm clear, does the 2.5.8 package start and run fine until the first Alert, and then it starts going haywire? And one more question. Does it go haywire on the first Alert, or the first Alert with a Block? I'm wondering if something is weird with Spoink, the snort2c table, and the check_reload_status() tool.

You could help me test this by configuring a VM and not checking the "Block Offenders" option. Let it record Alerts, but tell it not to block on them. Let's see if that keeps it stable. Trying to isolate if the problem is with Snort itself, or if it might be related to Spoink.

Bill

bmeeks

@Nukama:

Try the status_dhcpv6_leases.php, it outgrows the border to fit in information.

Thanks for the tip, but I wound up finding a different trick using a zero-width space character after each colon in an IPv6 address. That seems to fix the column overrun on my test systems using IE10, Chrome and Firefox as browsers.

This fix has been submitted via a Pull Request to the Core Team for review and approval. The Snort Package Version number will not increment, though. I will post back when the update has been pushed to the Packages repository.

Bill