Snort 2.9.4.1 pkg v.2.5.8
-
@jflsakfja:
VRT was enabled and downloaded. Worked OK before the update and md5 shows up in updates tab. Same thing on both of my boxes.
Forced an update and it found an update to VRT rules (although I'm pretty sure it passed through 2 auto updates so far…). Checked IPS Policy - Security list and it comes up as blank. Restarted snort through dashboard, restarted the interface after it finished and it's still blank.Went into the interface setttings and selected Balanced as the policy. Restarted the Interface and the list was populated as it should. Went back and changed it to security and restarted the interface yet again and it's still blank.
How much memory do you have in the box, and are you waiting for the process to completely finish? I ask these questions because the Security Policy is very large and it takes several seconds to read and then construct the table via PHP for the web page. I have tested all three Snort IPS Policies in my 2.1RC0 virtual machine, and they all work. Balanced and Security take longer to populate than Connectivity, with Security taking the longer by far. Attached is a screenshot showing the IPS Security policy rules being displayed.
One other question – what browser are you using? I've done the majority of my testing with Internet Explorer 10, but also checked things out with the latest Firefox and Chrome browsers.
-
I tested a Virtual Machine which gave me the same errors. I even installed a fresh system on this VM and restored a configuration file. Same result. Both on WAN DHCP. I even compared both configuration files of 2.5.7 and 2.5.8 but could not see a problem there.
So, package 2.5.7 works on both systems and package 2.5.8 does not. I really like the options you made in 2.5.8 but for now I can't use it. Do you think there a way I can update the firmware and install the old package 2.5.7?
I hope there will be others that have the same experience ;)
There was a change in the 2.5.8 code in the way it selected the gateways and WAN IPs to add to the default Whitelist and HOME_NET collections, but I can't for the life of me see how that could translate to what you are seeing. Not saying it can't be related, but how it might be related escapes me at the moment. I will keep testing my own VMs to see if I can replicate this.
As for running 2.5.7 on the new firmware, that is relatively easy to pull off if you are willing to copy some files around. Here is what to do:
Rollback to your 2.5.7 setup and copy the following files off to a set of directories in /tmp.
First, create the following directories in /tmp: www and pkg.
Next, navigate to /usr/local/pkg/snort and copy the files there to the /tmp/pkg directory created earlier.
Repeat the step above except the source directory is /usr/local/www/snort and the destination is /tmp/www.
Now update the firmware. This will also update Snort to 2.5.8, but that is OK. After the reboot from the update and the package reinstallation completes, copy the saved files back to their original locations. Copy the /tmp/pkg files to /usr/local/pkg/snort, and copy the /tmp/www files to /usr/local/www/snort.
Stop and restart Snort, and you should have 2.5.7 running on the new firmware. It will say "2.5.8" on the Packages screens because that is what PBI thinks it installed, but the header on the actual Snort pages should say 2.5.7.
Bill
-
Hey Bill,
thanks for this great package!
The alert tab in this version overlaps the IPv6 addresses over columns.
Hmm…I am having problems getting the various browsers to line-break properly. Thank you for the feedback on this problem.
The issue with the Alerts tab is needing to display more information than there really is available column width. Will take another crack at it. Have not found a way yet to signal the browser to break on say the colon ":" in the IPv6 address. If anyone has a suggestion, I'm all ears.
Bill
-
How much memory do you have in the box, and are you waiting for the process to completely finish? I ask these questions because the Security Policy is very large and it takes several seconds to read and then construct the table via PHP for the web page. I have tested all three Snort IPS Policies in my 2.1RC0 virtual machine, and they all work. Balanced and Security take longer to populate than Connectivity, with Security taking the longer by far. Attached is a screenshot showing the IPS Security policy rules being displayed.
I have 2GB ram in those boxes, and RAM never gets above 40% (on the days they get hammered, usually around 30%).
Waiting for it doesn't help either.It's not that the browser is waiting for the page, it gets the page, but it's a completely blank page (see a couple of my posts back for the html code).
I'm enabling all rules anyway, is there a difference between balanced and security after doing that, or does security load something external (something other that the rules listed on the rule enable/disable page)? -
@jflsakfja:
I have 2GB ram in those boxes, and RAM never gets above 40% (on the days they get hammered, usually around 30%).
Waiting for it doesn't help either.It's not that the browser is waiting for the page, it gets the page, but it's a completely blank page (see a couple of my posts back for the html code).
I'm enabling all rules anyway, is there a difference between balanced and security after doing that, or does security load something external (something other that the rules listed on the rule enable/disable page)?The three policies enable different sets of rules. These are pre-defined by the Snort VRT. The policy membership of a given rule is contained within a metadata statement in the text. Use grep and search the Snort rules files for these phrases to see which rules belong to which policy:
policy connectivity-ips, policy balanced-ips, and policy security-ips
Many of the rules containing one of these metadata values will be disabled by default (have the semicolon in front of the text). What Snort does when you select a policy is find all the rules containing the target policy in their metadata, then it enables those rules by removing any semicolons if present. So selecting all the Rule Categories on the Categories tab will only get you part of the way there. You will still have to individually enable the commented-out rules. That's why using the policy option in the drop-down is better. I am really puzzled why it is not working for you.
As one last troubleshooting attempt, have you tried force flushing the cache on your browser? Or how about a forced refresh while the blank page is displayed?
Bill
-
This is exactly why I wanted the widescreen package back. It needs to be able to scale the width of any column.
-
The three policies enable different sets of rules. These are pre-defined by the Snort VRT. The policy membership of a given rule is contained within a metadata statement in the text. Use grep and search the Snort rules files for these phrases to see which rules belong to which policy:
policy connectivity-ips, policy balanced-ips, and policy security-ips
Many of the rules containing one of these metaata values will be disabled by default (have the semicolon in front of the text). What Snort does when you select a policy is find all the rules containing the target policy in their metadata, then it enables those rules by removing any semicolons if present. So selecting all the Rule Categories on the Categories tab will only get you part of the way there. You will still have to individually enable the commented-out rules. That's why using the policy option in the drop-down is better. I am really puzzled why it is not working for you.
As one last troubleshooting attempt, have you tried force flushing the cache on your browser? Or how about a forced refresh while the blank page is displayed?
Bill
Already tried forcing refresh, clearing cache, restarting browser… nothing. It was working absolutely perfect before the update, but I'm not going back, carp sync is more important.
Is there anyway to force snort to rewrite the "security" part? like delete a file and restart or something? Deselecting VRT rules, restarting snort and then reenabling them doesn't work. -
Deinstall, reboot, install and see if it goes away.
-
Deinstall, reboot, install and see if it goes away.
Already tried that, just tried it again with no luck. I'm off to bed for now. Thank you all for your help so far.
-
Hey Bill,
thanks for this great package!
The alert tab in this version overlaps the IPv6 addresses over columns.
Hmm…I am having problems getting the various browsers to line-break properly. Thank you for the feedback on this problem.
The issue with the Alerts tab is needing to display more information than there really is available column width. Will take another crack at it. Have not found a way yet to signal the browser to break on say the colon ":" in the IPv6 address. If anyone has a suggestion, I'm all ears.
Bill
Try the status_dhcpv6_leases.php, it outgrows the border to fit in information.
-
@jflsakfja:
Deinstall, reboot, install and see if it goes away.
Already tried that, just tried it again with no luck. I'm off to bed for now. Thank you all for your help so far.
I think you said "Security" was the only policy that did not display correctly. Is that right? So for instance, Connectivity and Balanced display OK?
Is your locale set to display the screens in English or another language?
Do you have more than one interface enabled for Snort? If so, do all interfaces exhibit the same problem?
Sorry to pepper you with questions, but I really want to try and figure out what the problem is.
Edit: Oh, and one other thing you can check to see if the "Security" policy is actually in place. It could just be a display issue. Follow the steps below to investigate.
1. Under Diagnostics choose Edit File.
2. Navigate to /usr/pbi/snort__{arch}_/etc/snort/ (where {arch} is either "amd64" or "i386" depending on your CPU)
3. In the directory will be one or more snort_xxxx_xxx sub-directories. They correspond to each configured Snort interface. If you have Snort only on the WAN, then there will be only one of these additional snort_xxxx_xxx sub-directories. The "xxxx_xxx" will be a random UUID created for the interface followed by the NIC driver name and number. For example, on my VM test box the directory is called snort_33226_em0.
4. Navigate down into the snort_xxxx_xxx directory for the interface giving you the problem. In the directory is a rules directory. Navigate down into it and click the snort.rules rule to view it. It should be full of rule text lines, all uncommented and all containing "policy security-ips" in the metadata. If the file is present and contains the correct rules, then Snort is actually using the policy defined and your problem is purely a display issue. If the snort.rules file is empty, then we definitely have something strange going on.
Bill
-
I tested a Virtual Machine which gave me the same errors. I even installed a fresh system on this VM and restored a configuration file. Same result. Both on WAN DHCP. I even compared both configuration files of 2.5.7 and 2.5.8 but could not see a problem there.
So, package 2.5.7 works on both systems and package 2.5.8 does not. I really like the options you made in 2.5.8 but for now I can't use it. Do you think there a way I can update the firmware and install the old package 2.5.7?
I hope there will be others that have the same experience ;)
There was a change in the 2.5.8 code in the way it selected the gateways and WAN IPs to add to the default Whitelist and HOME_NET collections, but I can't for the life of me see how that could translate to what you are seeing. Not saying it can't be related, but how it might be related escapes me at the moment. I will keep testing my own VMs to see if I can replicate this.
As for running 2.5.7 on the new firmware, that is relatively easy to pull off if you are willing to copy some files around. Here is what to do:
Rollback to your 2.5.7 setup and copy the following files off to a set of directories in /tmp.
First, create the following directories in /tmp: www and pkg.
Next, navigate to /usr/local/pkg/snort and copy the files there to the /tmp/pkg directory created earlier.
Repeat the step above except the source directory is /usr/local/www/snort and the destination is /tmp/www.
Now update the firmware. This will also update Snort to 2.5.8, but that is OK. After the reboot from the update and the package reinstallation completes, copy the saved files back to their original locations. Copy the /tmp/pkg files to /usr/local/pkg/snort, and copy the /tmp/www files to /usr/local/www/snort.
Stop and restart Snort, and you should have 2.5.7 running on the new firmware. It will say "2.5.8" on the Packages screens because that is what PBI thinks it installed, but the header on the actual Snort pages should say 2.5.7.
Thank you, that worked! Although I see some strange file permissions (077).
I did some further testing. Deinstalled snort without saving configuration and configured it from scratch: same problems.
Then I installed a fresh pfSense RC0 Fri May 31 from memorystick to hard disk and restored configuration file and rebooted: same problems
An alert and then "check_reload_status" as I mentioned before.I am out of options now (well I can install pfSense system from scratch but then I need a day or so) and need an answer from the developers. Very weird!
-
I did some further testing. Deinstalled snort without saving configuration and configured it from scratch: same problems.
Then I installed a fresh pfSense RC0 Fri May 31 from memorystick to hard disk and restored configuration file and rebooted: same problems
An alert and then "check_reload_status" as I mentioned before.Just so I'm clear, does the 2.5.8 package start and run fine until the first Alert, and then it starts going haywire? And one more question. Does it go haywire on the first Alert, or the first Alert with a Block? I'm wondering if something is weird with Spoink, the snort2c table, and the check_reload_status() tool.
You could help me test this by configuring a VM and not checking the "Block Offenders" option. Let it record Alerts, but tell it not to block on them. Let's see if that keeps it stable. Trying to isolate if the problem is with Snort itself, or if it might be related to Spoink.
Bill
-
Try the status_dhcpv6_leases.php, it outgrows the border to fit in information.
Thanks for the tip, but I wound up finding a different trick using a zero-width space character after each colon in an IPv6 address. That seems to fix the column overrun on my test systems using IE10, Chrome and Firefox as browsers.
This fix has been submitted via a Pull Request to the Core Team for review and approval. The Snort Package Version number will not increment, though. I will post back when the update has been pushed to the Packages repository.
Bill
-
I did some further testing. Deinstalled snort without saving configuration and configured it from scratch: same problems.
Then I installed a fresh pfSense RC0 Fri May 31 from memorystick to hard disk and restored configuration file and rebooted: same problems
An alert and then "check_reload_status" as I mentioned before.Just so I'm clear, does the 2.5.8 package start and run fine until the first Alert, and then it starts going haywire? And one more question. Does it go haywire on the first Alert, or the first Alert with a Block? I'm wondering if something is weird with Spoink, the snort2c table, and the check_reload_status() tool.
You could help me test this by configuring a VM and not checking the "Block Offenders" option. Let it record Alerts, but tell it not to block on them. Let's see if that keeps it stable. Trying to isolate if the problem is with Snort itself, or if it might be related to Spoink.
Bill
I found it!
This bug #2555 describes also what I discovered and this topic made me also think. I have Intel NICs (82574L) and I switched my WAN interface to an Realtek one. Problem gone!
It must be a driver issue although the Intel NICs are recommended and were stable.Fingers crossed!!!!
-
I found it!
This bug #2555 describes also what I discovered and this topic made me also think. I have Intel NICs (82574L) and I switched my WAN interface to an Realtek one. Problem gone!
It must be a driver issue although the Intel NICs are recommended and were stable.Fingers crossed!!!!
Great news! Another user in the thread topic you linked was also having a problem with a newer Intel NIC (using the fxp0) driver. He swapped out his card and his problems also went away.
Bill
-
Yes I referred to that post but mangled my link. Corrected that now.
Just wanted 2.5.8 so badly that I spent a day looking for solutions ;D
But I can't change the driver for my Virtual Machine >:(
-
Another user in the thread topic you linked was also having a problem with a newer Intel NIC (using the fxp0) driver.
He swapped out his card and his problems also went away.Unless something changed very recently, the fxp(4) FreeBSD driver is used for ancient (mid-1990s) 100Mbps Intel NICs.
For best results it is recommended to use relatively recent (less than 10 years old) Intel GbE NICs such as those supported by the em(4) driver, and IMHO avoid wasting developers' time troubleshooting 20 year old hardware …
PS: bmeeks keep up your fine work !!!
-
But I can't change the driver for my Virtual Machine >:(
If it's VMware, yes you can. Simply choose e1000 in the NIC options (for ESXi). For Workstation you can manually edit the vmx file. Do a quick Google search. VMware will usually by default configure the e1000 NIC driver for virtual machines. This driver will show up in pfSense as "em0".
Bill
-
We are going off-topic but I have a Mac and Parallels Desktop. Too bad :'(
