Snort 2.9.4.1 pkg v.2.5.8
-
Yea, I publicly apologize for not providing more specific info. I'm running 2.0.3 (latest stable?), 32 bit, 2GB RAM, 3.4HT p4 cpu, supermicro p4sci motherboard (2 onboard intel nics) and a pci-x dual intel card. Install is on a gmirror with 2 40GB sata drives running off the board. Don't forget that there are 2 systems arranged in a CARP cluster. Running AC-BNFA.
If you need more info please let me know. I'm not able to access the systems for a couple of days (out of town) but I'll try my best to answer from memory. As far I can say it's not that it's running out of memory, it's not reading/parsing a file needed for the rule edit page, hence the blank page. There are no out of memory errors or cannot access file or anything, on my installs, just the blank security page. I can verify this by looking at the various logs. I'll repeat myself, balanced works ok, other edit pages work ok (GPL, all ET categories) and I can disable/enable rules, but when security is selected the page is blank. I can still edit GPL and all ET categories, just not IPS Policy Security.
The same problem is replicated to the slave/backup system. -
@jflsakfja:
Yea, I publicly apologize for not providing more specific info. I'm running 2.0.3 (latest stable?), 32 bit, 2GB RAM, 3.4HT p4 cpu, supermicro p4sci motherboard (2 onboard intel nics) and a pci-x dual intel card. Install is on a gmirror with 2 40GB sata drives running off the board. Don't forget that there are 2 systems arranged in a CARP cluster. Running AC-BNFA.
If you need more info please let me know. I'm not able to access the systems for a couple of days (out of town) but I'll try my best to answer from memory. As far I can say it's not that it's running out of memory, it's not reading/parsing a file needed for the rule edit page, hence the blank page. There are no out of memory errors or cannot access file or anything, on my installs, just the blank security page. I can verify this by looking at the various logs. I'll repeat myself, balanced works ok, other edit pages work ok (GPL, all ET categories) and I can disable/enable rules, but when security is selected the page is blank. I can still edit GPL and all ET categories, just not IPS Policy Security.
The same problem is replicated to the slave/backup system.I was able to reliably reproduce the problem. It is caused by the way a global array was being "released" in the new code. In an attempt to use less memory overall, the 2.5.8 package was tweaked so that an initial list of all the rules was loaded once as a global array and then used several times during the enforcing rules building process. This same trick is also a part of the IPS Policy rule selection. Ironically, the new "optimization" was the cause of the memory problem. The new code was not properly releasing this global array when it was finished with it. Thus the PHP process itself would run up against the default 128 MB limit set in the php.ini file on pfSense and crash. This caused the blank page. Bumping that value up to 250 MB as marcelloc suggested was a workaround, but it did not really get at the root of the problem. I tried his suggestion first in testing and it worked, but then some Google research turned up the proper (and not well documented, by the way) procedure for deleting or releasing global variables. Instead of unset($my_var), you must use unset($GLOBALS['my_var']).
The problem is not running out of RAM on the firewall itself, but rather the PHP process that is running the current web page you are viewing runs up against the 128 MB process limit set for an individual PHP process. Once that happens, HTML page construction stops and you get the blank page. You should be able to look in /tmp/PHP_errors.log and see messages in that file about PHP not being able to allocate additional memory.
After making the change for releasing global variables, I no longer get the blank page even when leaving the PHP process memory limit at the default of 128 MB. I will submit a fix for this error later today for review and approval by the Core Team. Once they approve, it will be posted in the Packages repository.
To cause this problem, you have to select a large number of Rule Categories. If using just the Snort VRT policy with either none or very few ET rule categories enabled, you don't see it. It manifests itself when you really load up with all the Rule Categories as you suggested. Thanks for reporting this problem. Solving it helped me learn something new about PHP coding.. ;D
Bill
-
successfully installed snort, but to start, back the following error in system.log > "FATAL ERROR: /usr/local/etc/snort/snort_61463_msk0/snort.conf(87) Unknown config directive: enable_gtp"
tryed uncheck the option "Enable GTP Detection" in preprocessor, but dont start.
anyone experiencing this problem?
Thx
-
successfully installed snort, but to start, back the following error in system.log > "FATAL ERROR: /usr/local/etc/snort/snort_61463_msk0/snort.conf(87) Unknown config directive: enable_gtp"
tryed uncheck the option "Enable GTP Detection" in preprocessor, but dont start.
anyone experiencing this problem?
Thx
You are missing a required Preprocessor. Instead of "unchecking" the Enable GTP Detection preprocessor, "check it". In fact, you generally should enable ALL of the Preprocessors except for Sensitive Data and two SCADA ones.
This post has some general tips for getting Snort up and running. Proper matching of enabled Preprocessors and selected rules is key to getting Snort to run.
http://forum.pfsense.org/index.php/topic,61018.msg328717.html#msg328717
Bill
-
successfully installed snort, but to start, back the following error in system.log > "FATAL ERROR: /usr/local/etc/snort/snort_61463_msk0/snort.conf(87) Unknown config directive: enable_gtp"
tryed uncheck the option "Enable GTP Detection" in preprocessor, but dont start.
anyone experiencing this problem?
Thx
You are missing a required Preprocessor. Instead of "unchecking" the Enable GTP Detection preprocessor, "check it". In fact, you generally should enable ALL of the Preprocessors except for Sensitive Data and two SCADA ones.
This post has some general tips for getting Snort up and running. Proper matching of enabled Preprocessors and selected rules is key to getting Snort to run.
http://forum.pfsense.org/index.php/topic,61018.msg328717.html#msg328717
Bill
thx Bill, but I tried with the box checked and gave the same error, tried with the box unchecked and had the same error.
I read your post, very good post, the only difference is why not use snort VRT and tried to enable but I had the same error…
:(
-
thx Bill, but I tried with the box checked and gave the same error, tried with the box unchecked and had the same error.
I read your post, very good post, the only difference is why not use snort VRT and tried to enable but I had the same error…
:(
Oops…misread your original error message. It is not liking a value in the snort.conf file. I initially thought it said "rule option", but it says config directive in snort.conf. Tell me what rule categories you have enabled, and the version of pfSense. From the path given in your error message, I am guessing 2.0.3. Is it 32-bit or 64-bit?
Bill
-
thx Bill, but I tried with the box checked and gave the same error, tried with the box unchecked and had the same error.
I read your post, very good post, the only difference is why not use snort VRT and tried to enable but I had the same error…
:(
Oops…misread your original error message. It is not liking a value in the snort.conf file. I initially thought it said "rule option", but it says config directive in snort.conf. Tell me what rule categories you have enabled, and the version of pfSense. From the path given in your error message, I am guessing 2.0.3. Is it 32-bit or 64-bit?
Bill
Yes, my version is pfsense 2.0.3 64-bit.
My rules catagories :
snort_botnet-cnc.rules
snort_ddos.rules
snort_scan.rules
snort_virus.rulesother testes:
-Checked only snort_ddos.rules in rules catagories
-tested too with "Use IPS Policy" checked box and "IPS Policy" checkedBut same erro
-
Yes, my version is pfsense 2.0.3 64-bit.
Would you mind posting the contents of /usr/local/etc/snort/snort_61463_msk0/snort.conf for me? You can obscure IP addresses in it if you wish. I am thus far unable to replicate this problem, so I need a look at your snort.conf file.
Thanks,
Bill -
Hey Bill,
thanks for this great package!
The alert tab in this version overlaps the IPv6 addresses over columns.
The fix for this is now posted. The Snort Package version number was not incremented, but you can pick up the fix by simply reinstalling the GUI components. Go to System…Packages and then the Installed Packages tab. Click the XML icon next to Snort to reinstall the GUI files. That should do it.
Bill
-
Yes, my version is pfsense 2.0.3 64-bit.
Would you mind posting the contents of /usr/local/etc/snort/snort_61463_msk0/snort.conf for me? You can obscure IP addresses in it if you wish. I am thus far unable to replicate this problem, so I need a look at your snort.conf file.
Thanks,
BillThx Bill,
snort.conf atached.
-
I was able to reliably reproduce the problem. It is caused by the way a global array was being "released" in the new code. In an attempt to use less memory overall, the 2.5.8 package was tweaked so that an initial list of all the rules was loaded once as a global array and then used several times during the enforcing rules building process. This same trick is also a part of the IPS Policy rule selection. Ironically, the new "optimization" was the cause of the memory problem. The new code was not properly releasing this global array when it was finished with it. Thus the PHP process itself would run up against the default 128 MB limit set in the php.ini file on pfSense and crash. This caused the blank page. Bumping that value up to 250 MB as marcelloc suggested was a workaround, but it did not really get at the root of the problem. I tried his suggestion first in testing and it worked, but then some Google research turned up the proper (and not well documented, by the way) procedure for deleting or releasing global variables. Instead of unset($my_var), you must use unset($GLOBALS['my_var']).
The problem is not running out of RAM on the firewall itself, but rather the PHP process that is running the current web page you are viewing runs up against the 128 MB process limit set for an individual PHP process. Once that happens, HTML page construction stops and you get the blank page. You should be able to look in /tmp/PHP_errors.log and see messages in that file about PHP not being able to allocate additional memory.
After making the change for releasing global variables, I no longer get the blank page even when leaving the PHP process memory limit at the default of 128 MB. I will submit a fix for this error later today for review and approval by the Core Team. Once they approve, it will be posted in the Packages repository.
To cause this problem, you have to select a large number of Rule Categories. If using just the Snort VRT policy with either none or very few ET rule categories enabled, you don't see it. It manifests itself when you really load up with all the Rule Categories as you suggested. Thanks for reporting this problem. Solving it helped me learn something new about PHP coding.. ;D
Bill
Glad I could be of help ;D. I'll report back when I get back to the systems, could be a couple of days which should give time for the fix to be available anyways.
PS.
Did I mention how much I love the SYNC feature? If it was less than 5 times, I'll say it again. Love the new SYNC feature ;D -
Thx Bill,
snort.conf atached.
marcosasjr:
I found a line-break problem in the file attached to your last post, but it was not in the section where you were getting the error. It's further down at line 130 in the {http_methods} section. There is a newline in the file you posted at the end of the parameter CCM_POST in that section. It should not be there. I don't know if that happened during file transfer and attaching to the forum post, or if the original file on your firewall has the same issue.
Have you tried these steps? This will force a rebuild of the snort.conf file on your setup.
1. From the Snort Interfaces tab, click the green icon to toggle Snort "off". The icon will turn into a red X.
2. Click the icon to edit the Snort interface and then click the Save button on the interface edit tab.
3. This will return you to the Snort Interfaces tab. Click the red X to start Snort. If successful, it will turn green in a few seconds.
4. If that doesn't work, try an uninstall and reinstall. Click the Global Settings tab and be sure the checkbox near the bottom of the page is checked so Snort will save settings upon a deinstall. Next, go to the Installed Packages tab (under the System…Packages menu) and click the X icon to completely remove Snort. When the uninstall finishes, go to the Available Packages tab and install it again fresh.
5. You can also have Snort locally verify the configuration file. Run this command sequence from a console prompt on the firewall. These are two separate commands with ENTER pressed after typing each one:
cd /usr/local/etc/snort/snort_61463_msk0 /usr/local/bin/snort -T -c ./snort.confSnort will read and validate the configuration file. It will report any error and the line number where the error was.
Report back and let me know if any of the above worked.
Bill
-
Another user in the thread topic you linked was also having a problem with a newer Intel NIC (using the fxp0) driver.
He swapped out his card and his problems also went away.Unless something changed very recently, the fxp(4) FreeBSD driver is used for ancient (mid-1990s) 100Mbps Intel NICs.
For best results it is recommended to use relatively recent (less than 10 years old) Intel GbE NICs such as those supported by the em(4) driver, and IMHO avoid wasting developers' time troubleshooting 20 year old hardware …
PS: bmeeks keep up your fine work !!!
I wonder why my 82574L Intel NIC doesn't work and an old Realtek 8139 does with Snort version 2.5.8. Anyway I am going to replace the Realtek NIC with an Intel Pro 1000 GT card. Let us see what happens then.
I saw a change made to "check_reload_status" and decided to try again my Intel NIC with a snapshot "2.1-RC0 (i386) built on Mon Jun 3 08:27:21 EDT 2013" and Snort block works again!
Another riddle solved. -
Thx Bill,
snort.conf atached.
marcosasjr:
I found a line-break problem in the file attached to your last post, but it was not in the section where you were getting the error. It's further down at line 130 in the {http_methods} section. There is a newline in the file you posted at the end of the parameter CCM_POST in that section. It should not be there. I don't know if that happened during file transfer and attaching to the forum post, or if the original file on your firewall has the same issue.
Have you tried these steps? This will force a rebuild of the snort.conf file on your setup.
1. From the Snort Interfaces tab, click the green icon to toggle Snort "off". The icon will turn into a red X.
2. Click the icon to edit the Snort interface and then click the Save button on the interface edit tab.
3. This will return you to the Snort Interfaces tab. Click the red X to start Snort. If successful, it will turn green in a few seconds.
4. If that doesn't work, try an uninstall and reinstall. Click the Global Settings tab and be sure the checkbox near the bottom of the page is checked so Snort will save settings upon a deinstall. Next, go to the Installed Packages tab (under the System…Packages menu) and click the X icon to completely remove Snort. When the uninstall finishes, go to the Available Packages tab and install it again fresh.
5. You can also have Snort locally verify the configuration file. Run this command sequence from a console prompt on the firewall. These are two separate commands with ENTER pressed after typing each one:
cd /usr/local/etc/snort/snort_61463_msk0 /usr/local/bin/snort -T -c ./snort.confSnort will read and validate the configuration file. It will report any error and the line number where the error was.
Report back and let me know if any of the above worked.
Bill
Hi Bill,
Thx for help,
I tried this steps and again now the step 4…
follows attached the archive "test snort.txt" from step 5 and snort.conf on "snort.txt".
[test snort.txt](/public/imported_attachments/1/test snort.txt)
snort.txt -
Hi Bill,
Thx for help,
I tried this steps and again now the step 4…
follows attached the archive "test snort.txt" from step 5 and snort.conf on "snort.txt".
I do not visually see anything wrong in the attached snort.conf file. This one is really puzzling. I will make a fresh 2.0.3 64-bit VM, import your file, and test with it to see if I can reproduce the problem. Give me a day for troubleshooting.
Thanks,
Bill -
Hi Bill,
Thx for help,
I tried this steps and again now the step 4…
follows attached the archive "test snort.txt" from step 5 and snort.conf on "snort.txt".
I do not visually see anything wrong in the attached snort.conf file. This one is really puzzling. I will make a fresh 2.0.3 64-bit VM, import your file, and test with it to see if I can reproduce the problem. Give me a day for troubleshooting.
Thanks,
BillHi Bill,
Thanks for great help,
I'll be watching the topic. -
Hi Bill,
Thanks for great help,
I'll be watching the topic.marcosasjr:
I created a clean 2.0.3 64-bit install on a VMware Workstation virtual machine and imported the snort.conf file you sent. I did have to edit the file to change the interface names, but I changed nothing anywhere else in the file. It validated fine, and Snort started just fine on the VM. I even selected the rule categories you listed in a previous post, and everything worked.
By the way, are you aware that those rule categories you had selected are basically empty of rules? You can see what I mean by viewing them on the RULES tab. Select each one, and any rules present in the file will show up below. The Snort VRT folks have restructured the rules files a bit and many of the old names are now empty shells. A number of those you listed are empty shells. That's not your problem with Snort not starting, but just pointing out that with those selected you have basically zero protection from Snort. Much better to check the "Use IPS Policy" checkbox and choose either "Connectivity, Balanced or Security". I suggest "Connectivity" if you are just starting out with Snort.
Now back to the topic and your problem – at this point I really have no idea what is wrong on your end. The file you sent me was fine. As far I know, you are the only user to experience this problem. So that indicates to me it is something unique with your installation.
What language setup do you have on your box and what browser are you using? Try taking the exact file you sent me and copying it back to the firewall. Download it fresh from your post here, then copy it back to the firewall on top of the original and try to start Snort. Instead of trying to start from the GUI, execute this command at a console prompt to start Snort. This sequence will not rebuild the snort.conf file prior to starting. The GUI start will always rebuild the snort.conf from scratch.
Command to start Snort without rebuilding snort.conf file:
/usr/local/etc/rc.d/snort.sh startAs a last resort, you can clear the box for Snort to save settings and then remove Snort from the firewall. Reboot the firewall, then install Snort fresh again. You will have to reconfigure all the Snort settings, but maybe that will fix it.
Bill
-
Hey Bill,
thanks for this great package!
The alert tab in this version overlaps the IPv6 addresses over columns.
The fix for this is now posted. The Snort Package version number was not incremented, but you can pick up the fix by simply reinstalling the GUI components. Go to System…Packages and then the Installed Packages tab. Click the XML icon next to Snort to reinstall the GUI files. That should do it.
Bill
Thanks Bill,
this patch does introduce line breaks in alert tab.
Copy and pasting is problematic when a parser is the next step.
Good work for this fixed width limitation. -
Thanks Bill,
this patch does introduce line breaks in alert tab.
Copy and pasting is problematic when a parser is the next step.
Good work for this fixed width limitation.Yeah, I knew copy and paste could be a problem given the inserted zero-width characters. It was the best option I could find. I never could find any tricks to tell the browser to break on something other than spaces or hyphens.
Bill
-
I am noticing that if i disable a few rules in current events, and then click on apply. It Disables the interface but the pid is still running. I then click start on the interface and it just makes a new one pid on top of the old one.
