Snort 2.9.4.6 Pkg v 2.5.9

bmeeks

Yes, Internet here is 100% IP dynamic whatever I power on/off my xDSL modem.

2.1-BETA1 (i386)
built on Wed May 22 08:31:46 EDT 2013
FreeBSD 8.3-RELEASE-p8

Snort builds the whitelist during each startup sequence. When the WAN IP changes, pfSense usually does a good job of restarting things. When restarted, Snort will correctly detect the new WAN IP and modify the whitelist accordingly assuming WAN IP is checked in the whitelist config (that is the default if you do not change it). Maybe in the newer 2.1 snapshots something is not working quite right with the auto-restart of packages.

A workaround would be to manually enter an Alias containing the IP subnet that your ISP routinely issues WAN IPs to you from. Then add this Alias to a custom whitelist for the WAN interface. That way no matter what IP in the block you happen to get, it will be whitelisted. This is not ideal and really should only be used as a temp workaround. Hopefully this problem will disappear as the 2.1 snapshots continue to be tweaked. I can also take a look to see if there is anything that could be done within Snort itself to better detect a WAN IP change.

Bill

pfSenseRocks

I have IPS Policy ( i.e. Snort GPLv2 Community Rules + Emerging Threats rule set) enabled on the WAN. And, all rule set minus the Snort GPLv2 Community Rules + Emerging Threats rule set enabled on the LAN interface.

Should I see 2 snort processes in this configuration, i.e. one snort process per interface? If I have IPv4 and IPv6 enabled on both the interface should I expect to see 4 processes?
Thanks!

bmeeks

@pfSenseRocks:

I have IPS Policy ( i.e. Snort GPLv2 Community Rules + Emerging Threats rule set) enabled on the WAN. And, all rule set minus the Snort GPLv2 Community Rules + Emerging Threats rule set enabled on the LAN interface.

Should I see 2 snort processes in this configuration, i.e. one snort process per interface? If I have IPv4 and IPv6 enabled on both the interface should I expect to see 4 processes?
Thanks!

One Snort process per interface. So in your case you should see two Snort processes. There was an issue with the later 2.1 Snapshots where multiple Snort processes per interface were getting kicked off on reboots. That was the result of some changes going on with the pfSense Snapshot code, though. Nothing has changed in the Snort package for a while.

Bill

pfSenseRocks

Thanks Bill. There is certainly something wonky going on, on the latest 2.1 snapshots. I have reconfigured snort for just the WAN interface IPv4 (no IPv6). Further, I only have IPS Policy ( i.e. Snort GPLv2 Community Rules + Emerging Threats rule set) enabled on the WAN. I see four (4) snort processes consuming up to 90% of the 6GB RAM and over 60% of the 16GB swap space.

Anything I can do (provide logs, traces, additional information) to debug and resolve this issue?

gogol

@pfSenseRocks:

Anything I can do (provide logs, traces, additional information) to debug and resolve this issue?

You could read through this thread. I already made a note about this a few pages back ;)

pfSenseRocks

Thank you for the workaround. I was offering up any help I can provide (since I have a 100% & consistent repro) to debug this issue and solve it rather than just working around it.

bmeeks

@pfSenseRocks:

Thank you for the workaround. I was offering up any help I can provide (since I have a 100% & consistent repro) to debug this issue and solve it rather than just working around it.

I have some VMs I can test in. I have a July 4th 2.1 Snapshot that does not exhibit this behavior. I will "snapshot" that VM and then let it upgrade to the latest 2.1 RC snapshot and see what I can determine about the multiple Snort process starts.

I've been letting Snort cook for a while with no package updates for two reasons. First to see how things were performing for users, and to see if the FreeBSD port got updated to the 2.5.x Snort binary. I have a new version of the Snort package ready that implements multiple engine/server configurations for the FRAG3, STREAM5 and HTTP_INSPECT preprocessors.

Bill

bmeeks

@pfSenseRocks:

Thank you for the workaround. I was offering up any help I can provide (since I have a 100% & consistent repro) to debug this issue and solve it rather than just working around it.

pfSenseRocks:

I upgraded a test VM to the latest 2.1RC snapshot. I could not reproduce the multiple processes problem. I have Snort configured on two interfaces for the VM, and I only get two Snort processes. Now I am using my new 2.6.0 package code in the VM. I can try reverting a VM back to the current 2.5.9 package and try again.

Bill

pfSenseRocks

That is great news, Bill. Thanks for the update. Let me update to the latest snapshot as well and see if I can reproduce your success.

pfSenseRocks

Unfortunately, I still reproduce the problem. Usually occurs after snort restarts after downloading new rules.

[2.1-RC1][admin@sense.home]/root(1): ps -ax | grep snort
23405 ?? Ss 8:25.86 /usr/pbi/snort-amd64/bin/snort -R 56048 -E -q -l /var/log/snort/snort_em0_vlan1056048 –pid-path /var/run
24490 ?? SNLs 0:28.51 /usr/pbi/snort-amd64/bin/snort -R 56048 -D -q -l /var/log/snort/snort_em0_vlan1056048 --pid-path /var/run
45765 ?? SNs 0:29.51 /usr/pbi/snort-amd64/bin/snort -R 56048 -D -q -l /var/log/snort/snort_em0_vlan1056048 --pid-path /var/run
46524 ?? Ss 0:03.79 /usr/pbi/snort-amd64/bin/snort -R 40477 -D -q -l /var/log/snort/snort_em0_vlan1140477 --pid-path /var/run
47171 ?? SNs 0:03.70 /usr/pbi/snort-amd64/bin/snort -R 40477 -D -q -l /var/log/snort/snort_em0_vlan1140477 --pid-path /var/run
47645 ?? SNs 0:03.76 /usr/pbi/snort-amd64/bin/snort -R 40477 -D -q -l /var/log/snort/snort_em0_vlan1140477 --pid-path /var/run
52671 0 S+ 0:00.00 grep snort

Version 2.1-RC1 (amd64)
built on Mon Aug 19 16:16:39 EDT 2013
FreeBSD 8.3-RELEASE-p9

bmeeks

@pfSenseRocks:

Unfortunately, I still reproduce the problem. Usually occurs after snort restarts after downloading new rules.

[2.1-RC1][admin@sense.home]/root(1): ps -ax | grep snort
23405 ?? Ss 8:25.86 /usr/pbi/snort-amd64/bin/snort -R 56048 -E -q -l /var/log/snort/snort_em0_vlan1056048 –pid-path /var/run
24490 ?? SNLs 0:28.51 /usr/pbi/snort-amd64/bin/snort -R 56048 -D -q -l /var/log/snort/snort_em0_vlan1056048 --pid-path /var/run
45765 ?? SNs 0:29.51 /usr/pbi/snort-amd64/bin/snort -R 56048 -D -q -l /var/log/snort/snort_em0_vlan1056048 --pid-path /var/run
46524 ?? Ss 0:03.79 /usr/pbi/snort-amd64/bin/snort -R 40477 -D -q -l /var/log/snort/snort_em0_vlan1140477 --pid-path /var/run
47171 ?? SNs 0:03.70 /usr/pbi/snort-amd64/bin/snort -R 40477 -D -q -l /var/log/snort/snort_em0_vlan1140477 --pid-path /var/run
47645 ?? SNs 0:03.76 /usr/pbi/snort-amd64/bin/snort -R 40477 -D -q -l /var/log/snort/snort_em0_vlan1140477 --pid-path /var/run
52671 0 S+ 0:00.00 grep snort

Version 2.1-RC1 (amd64)
built on Mon Aug 19 16:16:39 EDT 2013
FreeBSD 8.3-RELEASE-p9

Looks like you have multiple VLANs on a single interface. I did not test that way. I have just single IP blocks on each of my three interfaces, and I get only single instances of Snort per interface.

I have a theory about what could be happening. Unfortunately, if my theory is correct, this may be a hard bug to quash. Let me ponder on it and maybe also set up a VLAN configuration similar to yours. Without giving away too much private information, can you post a high-level description of how your Snort interfaces are configured in terms of VLANs (number per interface, etc.)?

Bill

Hello,
I have a small feature request. Would it be possible for the alerts tab to have a DNS lookup button under IPs shown (both source and destination) that opens a new tab and performs the same function as looking up an IP in Diagnostics>DNS lookup and displaying the results? Performing DNS lookups for all IPs showing up on alerts is not wanted or encouraged, just specific IPs. Saves me having to manually copy+paste the IP in DNS lookup.

Thank you.

Supermule

If I may add a feature for DNS lookup. A country flag next to the IP in the alerts and blocked tab…

Making it real easy to see where its coming from?

@jflsakfja:

Hello,
I have a small feature request. Would it be possible for the alerts tab to have a DNS lookup button under IPs shown (both source and destination) that opens a new tab and performs the same function as looking up an IP in Diagnostics>DNS lookup and displaying the results? Performing DNS lookups for all IPs showing up on alerts is not wanted or encouraged, just specific IPs. Saves me having to manually copy+paste the IP in DNS lookup.

Thank you.

@Supermule:

If I may add a feature for DNS lookup. A country flag next to the IP in the alerts and blocked tab…

Making it real easy to see where its coming from?

I believe that would require to perform the lookups in advance for all IPs, which could overload some low bandwidth connections. I'm getting hundreds of alerts per hour for example. Personally I don't think that is a good idea. If there is a way to store the country IPs in RAM and perform the country lookup there, I'd be fine with that.

Edit: completely missed my mind: The functionality wanted is the exact same functionality offered by the "blue i" button next to IPs in the firewall logs

kilthro

@Supermule:

If I may add a feature for DNS lookup. A country flag next to the IP in the alerts and blocked tab…

Making it real easy to see where its coming from?

@jflsakfja:

Hello,
I have a small feature request. Would it be possible for the alerts tab to have a DNS lookup button under IPs shown (both source and destination) that opens a new tab and performs the same function as looking up an IP in Diagnostics>DNS lookup and displaying the results? Performing DNS lookups for all IPs showing up on alerts is not wanted or encouraged, just specific IPs. Saves me having to manually copy+paste the IP in DNS lookup.

Thank you.

While i understand on a high traffic network with alot of alerts this may not be wanted but to have the option would be fantastic. Maybe something that is enabled or disabled.. Good idea anyways. :-D

An option to display all IP's country that can be enabled and the "blue i" button next to the IP in the alerts/blocked tabs disappears when the option is enabled, when it is disabled, the "blue i" button is shown next to IPs (to prevent flooding the network with lookups)? Everybody is happy then ;D

bmeeks

I will take a look and see what's possible with regards to the DNS lookups on the Alerts and Blocked tabs. I like the idea of the blue icon and then a pop-up window containing the lookup results when clicked. That is the least I/O intensive procedure.

Bill

Cino

@bmeeks:

@pfSenseRocks:

Unfortunately, I still reproduce the problem. Usually occurs after snort restarts after downloading new rules.

[2.1-RC1][admin@sense.home]/root(1): ps -ax | grep snort
23405 ?? Ss 8:25.86 /usr/pbi/snort-amd64/bin/snort -R 56048 -E -q -l /var/log/snort/snort_em0_vlan1056048 –pid-path /var/run
24490 ?? SNLs 0:28.51 /usr/pbi/snort-amd64/bin/snort -R 56048 -D -q -l /var/log/snort/snort_em0_vlan1056048 --pid-path /var/run
45765 ?? SNs 0:29.51 /usr/pbi/snort-amd64/bin/snort -R 56048 -D -q -l /var/log/snort/snort_em0_vlan1056048 --pid-path /var/run
46524 ?? Ss 0:03.79 /usr/pbi/snort-amd64/bin/snort -R 40477 -D -q -l /var/log/snort/snort_em0_vlan1140477 --pid-path /var/run
47171 ?? SNs 0:03.70 /usr/pbi/snort-amd64/bin/snort -R 40477 -D -q -l /var/log/snort/snort_em0_vlan1140477 --pid-path /var/run
47645 ?? SNs 0:03.76 /usr/pbi/snort-amd64/bin/snort -R 40477 -D -q -l /var/log/snort/snort_em0_vlan1140477 --pid-path /var/run
52671 0 S+ 0:00.00 grep snort

Version 2.1-RC1 (amd64)
built on Mon Aug 19 16:16:39 EDT 2013
FreeBSD 8.3-RELEASE-p9

Looks like you have multiple VLANs on a single interface. I did not test that way. I have just single IP blocks on each of my three interfaces, and I get only single instances of Snort per interface.

I have a theory about what could be happening. Unfortunately, if my theory is correct, this may be a hard bug to quash. Let me ponder on it and maybe also set up a VLAN configuration similar to yours. Without giving away too much private information, can you post a high-level description of how your Snort interfaces are configured in terms of VLANs (number per interface, etc.)?

Bill

I have a similar issue. If there rc.start_packages is called, snort doesn't restart correctly. It will create new instances of snort… I've maxed out of resources on my box because of this..


[2.1-RC1][/root(1): ps -ax | grep snort
11617  ??  SNs    0:19.21 /usr/pbi/snort-i386/bin/snort -R 63656 -D -q -l /var/log/snort/snort_em0_vlan563656 --pid-path /var/run --nolock-pidfile -G 63656 -c /usr/pbi/snort-i386/etc
12256  ??  SNs    9:30.06 /usr/pbi/snort-i386/bin/snort -R 60770 -D -q -l /var/log/snort/snort_em360770 --pid-path /var/run --nolock-pidfile -G 60770 -c /usr/pbi/snort-i386/etc/snort
18390  ??  SNs    7:23.96 /usr/pbi/snort-i386/bin/snort -R 5622 -D -q -l /var/log/snort/snort_em25622 --pid-path /var/run --nolock-pidfile -G 5622 -c /usr/pbi/snort-i386/etc/snort/sn
42825  ??  SNs    4:17.50 /usr/pbi/snort-i386/bin/snort -R 60770 -D -q -l /var/log/snort/snort_em360770 --pid-path /var/run --nolock-pidfile -G 60770 -c /usr/pbi/snort-i386/etc/snort
56893  ??  SNs    1:41.06 /usr/pbi/snort-i386/bin/snort -R 60770 -D -q -l /var/log/snort/snort_em360770 --pid-path /var/run --nolock-pidfile -G 60770 -c /usr/pbi/snort-i386/etc/snort
67712  ??  SNs    1:26.93 /usr/pbi/snort-i386/bin/snort -R 63656 -D -q -l /var/log/snort/snort_em0_vlan563656 --pid-path /var/run --nolock-pidfile -G 63656 -c /usr/pbi/snort-i386/etc
74458  ??  SNs    0:17.27 /usr/pbi/snort-i386/bin/snort -R 59292 -D -q -l /var/log/snort/snort_em359292 --pid-path /var/run --nolock-pidfile -G 59292 -c /usr/pbi/snort-i386/etc/snort
76099  ??  SNs    3:40.18 /usr/pbi/snort-i386/bin/snort -R 5622 -D -q -l /var/log/snort/snort_em25622 --pid-path /var/run --nolock-pidfile -G 5622 -c /usr/pbi/snort-i386/etc/snort/sn
90876  ??  SNs    1:26.13 /usr/pbi/snort-i386/bin/snort -R 5622 -D -q -l /var/log/snort/snort_em25622 --pid-path /var/run --nolock-pidfile -G 5622 -c /usr/pbi/snort-i386/etc/snort/sn
93617  ??  SNs    0:05.95 /usr/pbi/snort-i386/bin/snort -R 63656 -D -q -l /var/log/snort/snort_em0_vlan563656 --pid-path /var/run --nolock-pidfile -G 63656 -c /usr/pbi/snort-i386/etc
63880   0  S+     0:00.02 grep snort
[2.1-RC1][root@pfsense.cino.homeip.net]/root(2):

pfSenseRocks

can you post a high-level description of how your Snort interfaces are configured in terms of VLANs (number per interface, etc.)?

Sorry about the tardy response, Bill. I have been traveling with intermittent to no internet connectivity.

Here's my config on pfSense:

VLAN10 WAN
VLAN11 LAN
VLAN12 GAN // Guest LAN

snort:
WAN IPS security profile
LAN All other categories that aren't included in security profile

Both interfaces are configured for AC.

Sorry about the incomplete info. I am reciting from memory.

bmeeks

Cino and pfSenseRocks:

Thank you for the feedback on the multiple instances problem. I will be tied up the next few days on some business and then personal stuff, so it will be after the U.S. Labor Day Holiday (September 2nd) before I can devote a lot of time to researching this issue. The hint about rc.start_packages is helpful. I will see if I can get to the bottom of the problem, though. Different manifestations of what are probably the same underlying bug have shown up over the last year with Snort and restarts. Sometimes it seems to be fixed, and then it pops up again. Obviously we have not yet found the true root cause.

Bill