Snort does not run in WAN interface in pfSense 2.1
-
Hi:
Installed Snort and activated to all the interfaces. Snort is enabled and running in both LAN and OPT1 interfaces, but not running in the WAN interface though enabled. Trying to run by clicking on the red x icon does not do anything.
Checking the snort log outputs segmentation fault.
clog /var/log/snort/snort_bge057263/alert Segmentation fault (core dumped) -
Hi:
Installed Snort and activated to all the interfaces. Snort is enabled and running in both LAN and OPT1 interfaces, but not running in the WAN interface though enabled. Trying to run by clicking on the red x icon does not do anything.
Checking the snort log outputs segmentation fault.
clog /var/log/snort/snort_bge057263/alert Segmentation fault (core dumped)Whoa! A seg-fault is pretty severe. First up, are all the interfaces the same hardware NIC? Just trying to eliminate any driver issues. Snort puts the interfaces in promiscuous mode.
Here are some suggested troubleshooting steps:
1. Try Snort on the WAN with no rules enabled at all. See if it starts up.
2. If that works, then add a very minimal set of rules like the Emerging Threats ET_CIARMY rules, and try to stop and restart again.
Post back with the results.
Bill
-
Whoa! A seg-fault is pretty severe. First up, are all the interfaces the same hardware NIC? Just trying to eliminate any driver issues. Snort puts the interfaces in promiscuous mode.
The WAN has broadcom gigabit (bge) and the rest two are intel gigabit (em). Two intel interfaces works fine but with bge it refuses to start. Necessary tweaks are applied for relevant network cards in /boot/loader.conf.local.
Here are some suggested troubleshooting steps:
1. Try Snort on the WAN with no rules enabled at all. See if it starts up.
Yes it did!
2. If that works, then add a very minimal set of rules like the Emerging Threats ET_CIARMY rules, and try to stop and restart again.
Post back with the results.
Bill
Yes, with just the et_ciarmy rules, it runs, but when I added more ET rules, it didn't. On the other hand, 'Snort Text rules' and 'Snort SO rules' are greyed preventing from selection, except 'Snort GPLv2 Community Rules (VRT certified)' which can be (un)checked.
-
Yes, with just the et_ciarmy rules, it runs, but when I added more ET rules, it didn't. On the other hand, 'Snort Text rules' and 'Snort SO rules' are greyed preventing from selection, except 'Snort GPLv2 Community Rules (VRT certified)' which can be (un)checked.
Unless you have an Oinkcode and have entered it on the Global Settings tab, then the Snort VRT selections will be grayed-out. You would also need to perform an update of the rules downloaded from the Updates tab once enabling the Snort VRT rules.
Is this a 2.0.x or 2.1 platform? If 2.0.x, you might consider trying the 2.1RC0 snapshot. It will have newer drivers for the NICs. I'm kind of thinking maybe the NIC driver is at fault here, but I'm not sure. The fact Snort runs OK on the Intel NICs suggests the basic Snort installation is fine. The exact same binary and libraries are used for each interface instance. It just launches an additional instance of the executable binary for each interface.
Are the Preprocessor settings configured the same on the interfaces?
Bill
-
I have oinkcode inserted. Preprocessors configured on the same interface. This is on pfSense 2.1 RC0 snapshot.
-
I have oinkcode inserted. Preprocessors configured on the same interface. This is on pfSense 2.1 RC0 snapshot.
Couple more questions. I am trying to figure out why the Snort VRT things are grayed-out for you. On the Categories tab, do you have "Use IPS Policy" checked? That will automatically disable the checkboxes for the Snort Text and SO Rules when checked. Also, I assume on the Global Settings tab that the Snort VRT rules are enabled (that box is checked)? This is global, so if you are using Snort VRT rules on the other interfaces, then never mind.
Have you tried configuring the exact same rules on the WAN as you have on a working interface just for testing purposes? That would help eliminate some other possibilities.
Bill
-
Thanks Bill for your explicit explanation.
Yes, the 'Use IPS policy' checked which is causing the snort rules to grey out.
I have the same configuration for DMZ (em) as WAN (bge), and it works flawlessly. Maybe it is a bge driver issue. In that case, the snort on bge interface works fine when I choose a single ET_CIARMY rule, but fails whenever other rules are chosen. Pretty weird, isn't it?!
-
Thanks Bill for your explicit explanation.
Yes, the 'Use IPS policy' checked which is causing the snort rules to grey out.
I have the same configuration for DMZ (em) as WAN (bge), and it works flawlessly. Maybe it is a bge driver issue. In that case, the snort on bge interface works fine when I choose a single ET_CIARMY rule, but fails whenever other rules are chosen. Pretty weird, isn't it?!
Yes, that is quite weird. When you say a "single ET_CIARMY rule", do you literally mean a single rule out of that file, or all the rules that are default enabled in that particular Category file? I assume it's the latter (all the default enabled rules in that single category).
If you have not tried this already, delete the WAN interface entirely from the Snort Interfaces tab, and then add it back and configure rules on it again. Just in case something got stored corrupted in the section of the config.xml file for the Snort WAN settings.
I am running out of ideas at this point.
Bill
-
Stupid question but have you enabled multiple snort interface mappings??
-
Stupid question but have you enabled multiple snort interface mappings??
pfSense GUI is so exhaustive (I am old-styled command line guy) that I could not figure out where to enable "multiple snort interface mappings"! Do you mean to check the checkbox against 'Enable' against each interfaces in snort gui? In that case, yes!
-
I thought that it has to do with the bge driver that Snort is not running on WAN as Bill suspected, and therefore, swapped the em driver running on LAN with the bge running on WAN. After the swapping, even with WAN with em (which is working in LAN) stopped to work and LAN with bge driver starts Snort (whereas WAN with bge was not working). Therefore, there is something that prevents WAN to run snort.
The only thing that was displayed on console was:
swap_pager: out of swap space
swap_pager_getswapspace(16):failedAnd when I checked system.log
It showed up:
Jun 25 22:22:19 pfSense0 kernel: swap_pager: out of swap space
Jun 25 22:22:19 pfSense0 kernel: swap_pager_getswapspace(16): failed
….
Jun 25 22:31:10 pfSense0 snort[50004]: FATAL ERROR: fpcreate.c(1555) Failed to compile port group patterns.Any inputs appreciated!
-
I thought that it has to do with the bge driver that Snort is not running on WAN as Bill suspected, and therefore, swapped the em driver running on LAN with the bge running on WAN. After the swapping, even with WAN with em (which is working in LAN) stopped to work and LAN with bge driver starts Snort (whereas WAN with bge was not working). Therefore, there is something that prevents WAN to run snort.
The only thing that was displayed on console was:
swap_pager: out of swap space
swap_pager_getswapspace(16):failedAnd when I checked system.log
It showed up:
Jun 25 22:22:19 pfSense0 kernel: swap_pager: out of swap space
Jun 25 22:22:19 pfSense0 kernel: swap_pager_getswapspace(16): failed
….
Jun 25 22:31:10 pfSense0 snort[50004]: FATAL ERROR: fpcreate.c(1555) Failed to compile port group patterns.Any inputs appreciated!
zenny:
Something is corrupt in the snortglobals part of config.xml for the WAN interface. If you feel comfortable with editing an XML file, here's what I would suggest.
1. Backup the existing config.xml (the configuration) by using the tool under Diagnostics…Backup/Restore. Save the file off somewhere not on the firewall. If you don't mind, I would like a copy of that file to see what might be wrong with it. I will PM you my e-mail address. Looking into the file will hopefully help me find the problem and see if any additional data scrubbing or validation might be needed in the code to prevent it from happening to others.
2. First try deleting the Snort WAN interface. Go to Snort Interfaces, check the checkbox beside the WAN interface, and click the (x) icon to remove it.
3. Add the interface back and start over with the WAN configuration in Snort.
If you have already tried steps #2 and #3 above and it still is broken, then proceed to #4 below.
4. Edit the config.xml file on the firewall by going to Diagnostics…Edit File and navigating to /conf/config.xml.
5. Scroll down towards the bottom of the file until you find the section <snortglobal>.
6. Within that section, there will be multiple <rule>sub-sections. Each will be delimited by <rule>at the start and</rule> at the end. Find the sub-section for the WAN interface. There is an <interface>element tag in the file with the interface name specified.
7. Delete the entire <rule>to</rule> section for the WAN. Be careful that you choose the correct start and end tags and do not go past them. It's easiest to stay scrolled all the way to the left-hand margin and just use SHIFT+PAGE DOWN to select entire lines. Delete the highlighted section.
8. Save the file, then go to the Snort Interfaces tab and add the WAN back.
Note that unless the config.xml file is severely messed up, simply deleting the interface from the Snort GUI (steps #2 and #3 above) should clean it up without needing to resort to the manual edit. It you mess it up, just restore the saved configuration.
Bill</interface></rule></snortglobal>
-
I thought that it has to do with the bge driver that Snort is not running on WAN as Bill suspected, and therefore, swapped the em driver running on LAN with the bge running on WAN. After the swapping, even with WAN with em (which is working in LAN) stopped to work and LAN with bge driver starts Snort (whereas WAN with bge was not working). Therefore, there is something that prevents WAN to run snort.
The only thing that was displayed on console was:
swap_pager: out of swap space
swap_pager_getswapspace(16):failedAnd when I checked system.log
It showed up:
Jun 25 22:22:19 pfSense0 kernel: swap_pager: out of swap space
Jun 25 22:22:19 pfSense0 kernel: swap_pager_getswapspace(16): failed
….
Jun 25 22:31:10 pfSense0 snort[50004]: FATAL ERROR: fpcreate.c(1555) Failed to compile port group patterns.Any inputs appreciated!
zenny:
Something is corrupt in the snortglobals part of config.xml for the WAN interface. If you feel comfortable with editing an XML file, here's what I would suggest.
1. Backup the existing config.xml (the configuration) by using the tool under Diagnostics…Backup/Restore. Save the file off somewhere not on the firewall. If you don't mind, I would like a copy of that file to see what might be wrong with it. I will PM you my e-mail address. Looking into the file will hopefully help me find the problem and see if any additional data scrubbing or validation might be needed in the code to prevent it from happening to others.
2. First try deleting the Snort WAN interface. Go to Snort Interfaces, check the checkbox beside the WAN interface, and click the (x) icon to remove it.
3. Add the interface back and start over with the WAN configuration in Snort.
If you have already tried steps #2 and #3 above and it still is broken, then proceed to #4 below.
4. Edit the config.xml file on the firewall by going to Diagnostics…Edit File and navigating to /conf/config.xml.
5. Scroll down towards the bottom of the file until you find the section <snortglobal>.
6. Within that section, there will be multiple <rule>sub-sections. Each will be delimited by <rule>at the start and</rule> at the end. Find the sub-section for the WAN interface. There is an <interface>element tag in the file with the interface name specified.
7. Delete the entire <rule>to</rule> section for the WAN. Be careful that you choose the correct start and end tags and do not go past them. It's easiest to stay scrolled all the way to the left-hand margin and just use SHIFT+PAGE DOWN to select entire lines. Delete the highlighted section.
8. Save the file, then go to the Snort Interfaces tab and add the WAN back.
Note that unless the config.xml file is severely messed up, simply deleting the interface from the Snort GUI (steps #2 and #3 above) should clean it up without needing to resort to the manual edit. It you mess it up, just restore the saved configuration.
Bill</interface></rule></snortglobal>
Tried with step 4-8, didn't work!
Then tried step 2-3, yet didn't work!
Sending the config.xml to your email as per your PM!
Thanks again!
-
Tried with step 4-8, didn't work!
Then tried step 2-3, yet didn't work!
Sending the config.xml to your email as per your PM!
Thanks again!
Received your file and will take a look. This one has me puzzled for sure! Cleaning out the file should have fixed it.
Bill
-
Tried with step 4-8, didn't work!
Then tried step 2-3, yet didn't work!
Sending the config.xml to your email as per your PM!
Thanks again!
Received your file and will take a look. This one has me puzzled for sure! Cleaning out the file should have fixed it.
Bill
Nope even cleaning out the file didn't help! Anyway thanks! It could be something messy with either 2.1, don't know.
UPDATE: The culprit seems to be the 'IPS Policy' option which should not be. After I disabled IPS Policy and manually selected the ET and Snort rules, snort on WAN worked fine.
-
Nope even cleaning out the file didn't help! Anyway thanks! It could be something messy with either 2.1, don't know.
UPDATE: The culprit seems to be the 'IPS Policy' option which should not be. After I disabled IPS Policy and manually selected the ET and Snort rules, snort on WAN worked fine.
Thanks for the clue. Which policy were you selecting (Connectivity, Balanced or Security), and how much physical RAM is in your firewall? Snort can chew through RAM, especially with lots of enabled rules. I believe Supermule had an issue with running out of swap space with a large rule set on a firewall with either 1 GB or 2 GB of RAM in a virtual machine on VMware. He bumped up the RAM and the problem fixed itself. When the OS starts swapping to disk and even runs out of virtual RAM there, things go weird pretty quickly after that.
Bill
-
I had to upgrade the VM's to 4GB for it not to swap and run out of memory. It can easily be an issue there.
-
Nope even cleaning out the file didn't help! Anyway thanks! It could be something messy with either 2.1, don't know.
UPDATE: The culprit seems to be the 'IPS Policy' option which should not be. After I disabled IPS Policy and manually selected the ET and Snort rules, snort on WAN worked fine.
Thanks for the clue. Which policy were you selecting (Connectivity, Balanced or Security), and how much physical RAM is in your firewall? Snort can chew through RAM, especially with lots of enabled rules. I believe Supermule had an issue with running out of swap space with a large rule set on a firewall with either 1 GB or 2 GB of RAM in a virtual machine on VMware. He bumped up the RAM and the problem fixed itself. When the OS starts swapping to disk and even runs out of virtual RAM there, things go weird pretty quickly after that.
Bill
Whatever policy I slecect among Connectivity, Balanced or Security, it does not work. The machine has 1.5GB of physical RAM. However, I cannot confirm whether it is a RAM issue or pfSense issue.
-
2GB was not enough for me. Try disabling a lot of rules and use the policy again.
-
2GB was not enough for me. Try disabling a lot of rules and use the policy again.
Thanks for the tip. There is no way to upgrade the RAM at the moment because the machine (Compaq Deskpro SFF dc7100sff with dual core P4 machine with 3.2Ghz processors) uses old PC3200 SDRAM which is pretty expensive compared to new DDR3 RAM. Worth upgrading the machine instead. ;-)
Let me know if anyone knows of cheap vendors who has at least 4GB of PC3200 SDRAM so that I can upgrade. ;-)
Actually I need 8 of these: http://www.newegg.com/Product/Product.aspx?Item=N82E16820141317 but it is out of stock. :-(
