Snort 2.9.2.3 pkg v. 2.5.0 Issues

[mdima]

Hello,
    just uninstalled/reinstalled this morning, the package works fine (Sensitive Data Processor disabled), but it still blocks the WAN addresses… so I am still running with blocking off.

Thanks,
Michele

[Fesoj]

mdima,

do you mean with WAN address the WAN side of your box or any outside address? If it is the WAN address of your box, then you need to add the ip to the whitelist of the interface.

[mdima]

@Fesoj: It's the same, with or without whitelist… and by default it should exclude local IP addresses and so on.

Probably this can help: my current HomeNet is an Alias that contains sub-aliases of local and remote IP addresses that should not be detected/blocked by Snort...

Thanks,
Michele

[Fesoj]

mdima,

HOME_NET != WHITELIST. On the WAN side, I have the default HOME_NET, but a special whitelist (including the IP address on the WAN side (I don't know what to do in case of an ISP supplied address using DHCP)) and this works fine. It's just the way things are setup. As I said before, it is now easier than ever to shoot oneself in the foot.

[Fesoj]

snort[57587]: FATAL ERROR: /usr/local/etc/snort/snort_56964_re1/preproc_rules/sensitive-data.rules(1) Unknown ClassType: sdf

has been observed here several times. If you have installed only the ET rules, the file classification.config does not contain the required entry. The rules from Snort.org do have the definition. Both rule sets contain their own classification.config file. I think only 1 of them gets installed (but I may be wrong).

ermal, could you please have a look at this issue?

UPDATE: with package vs. 2.5.1 this is obsolete

[mdima]

@Fesoj:

mdima,

HOME_NET != WHITELIST. On the WAN side, I have the default HOME_NET, but a special whitelist (including the IP address on the WAN side (I don't know what to do in case of an ISP supplied address using DHCP)) and this works fine. It's just the way things are setup. As I said before, it is now easier than ever to shoot oneself in the foot.

Fesoj, I already specified as Whitelist the Alias where my WAN network is, and anyway it gets blocked. Before this didn't happen, I think it has something to do with the new features of the whitelist management…

[miles267]

@Fesoj:

snort[57587]: FATAL ERROR: /usr/local/etc/snort/snort_56964_re1/preproc_rules/sensitive-data.rules(1) Unknown ClassType: sdf

has been observed here several times. If you have installed only the ET rules, the file classification.config does not contain the required entry. The rules from Snort.org do have the definition. Both rule sets contain their own classification.config file. I think only 1 of them gets installed (but I may be wrong).

ermal, could you please have a look at this issue?

I too am now consistently receiving this same error in my system log and am unable to start my interface manually.  I wish snort still auto-updated snort rules and successfully restarted after the rules update.  Has since become a manual process recently.

snort[57587]: FATAL ERROR: /usr/local/etc/snort/snort_56964_re1/preproc_rules/sensitive-data.rules(1) Unknown ClassType: sdf

[Fesoj]

I currently have no problems and everything works as expected.

Did you have a look at
(1) http://forum.pfsense.org/index.php/topic,51493.msg275905.html#msg275905
(2) http://forum.pfsense.org/index.php/topic,51493.msg276246.html#msg276246
(3) http://forum.pfsense.org/index.php/topic,51493.msg276317.html#msg276317
?

What happens when you deactivate all rules? Which IPs are blocked (whitelisting works on my machine)?

UPDATE: with package vs. 2.5.1 this is obsolete

Update: Also http://forum.pfsense.org/index.php/topic,51493.msg276334.html#msg276334 matters.

[miles267]

Fesoj, I reviewed the hyperlinks you suggested and didn't see a fix for the FATAL ERROR (sdf issue).  If anything, someone said there isn't a fix for the sdf issue. I will try to disable all rules and see whether snort will start on the interface.  Thanks.

EDIT: deselecting all rules for the interface and attempting to manually start the snort on the interface fails. still returns same (sdf) error.

[Fesoj]

miles267,

first the files classification.config are incomplete. Secondly the sed commands in snort.inc are srambled. You need to take care of both issues.

… I am currently working on a patch to allow automatic updates (of the rules---not the package, because you'll need patched sources).

[Fesoj]

miles267,

here's the source patch for snort.inc: https://github.com/Fesoj/pfsense-packages/compare/patch-2

You need to edit 4 places by hand. Good luck.

Update: If someone feels uncomfortable with that, I could post my patched file…

UPDATE: with package vs. 2.5.1 this is obsolete

[miles267]

@Fesoj:

miles267,

here's the source patch for snort.inc: https://github.com/Fesoj/pfsense-packages/compare/patch-2

You need to edit 4 places by hand. Good luck.

Update: If someone feels uncomfortable with that, I could post my patched file…

Fesoj, thank you SO MUCH.  Edited the snort.inc file with the (4) lines of code you suggested (delete/add lines) and the interface started without issue.  Appreciate your help and attention to detail.

[Fesoj]

miles267, it aint't over yet. Make sure you patch the various classification.config files as well. The manual patches to classification.config won't survive any rule updates…

[miles267]

@Fesoj:

miles267, it aint't over yet. Make sure you patch the various classification.config files as well. The manual patches to classification.config won't survice any rule updates…

Fesoj, understood.  Can you point me to info on how to patch the classification.config file(s) also? Sorry I must've overlooked that detail.  Thanks for your patience.

[Fesoj]

(1) Find all the classification.config files with find /usr/local -name 'classification.config*'. If you have n interfaces defined for snort you should get at least n+1 copies.

(2) Append config classification: sdf,Sensitive Data was Transmitted Across the Network,2 to the end of each file if the sdf classification is missing.

This is a long story short.

UPDATE: with package vs. 2.5.1 this is obsolete

[Fesoj]

Hi,

I pretty sure that I have a quick patch for the update problem, though I am not sure whether I should post it.

If others can confirm my recent patches, i.e. snort with sensible data preproc enabled, I could go on with this, otherwise it simply does not make any sense. Please reply if you have some time left.

[CS]

@mdima:

@Fesoj:

mdima,

HOME_NET != WHITELIST. On the WAN side, I have the default HOME_NET, but a special whitelist (including the IP address on the WAN side (I don't know what to do in case of an ISP supplied address using DHCP)) and this works fine. It's just the way things are setup. As I said before, it is now easier than ever to shoot oneself in the foot.

Fesoj, I already specified as Whitelist the Alias where my WAN network is, and anyway it gets blocked. Before this didn't happen, I think it has something to do with the new features of the whitelist management…

Hi mdima, hi Fesoj,

I have the same problem with blocking my WAN gateway (local IP, not public). I left the default setting for Home_Net and I use a Whitelist with all my subnets (internal, external). Using the same config (blocking src and dst) I had no problem with the previous snort version but I have with this one. It is clear that it blocks the WAN IP which is included in the Whitelist.

/CS

[Fesoj]

Can you publish a screen dump of your whitelist? Just to make sure that you are using aliases. If you have upgraded from a previous version there might still be the old style list (which probably no longer works).

[CS]

No I created a new Alias under the Firewall tab and I copied there my old IPs/Subnets. Then I used this Alias on the Whitelist settings.

Maybe after all these re-installations of snort it has now been fixed. I will turn blocking ON again and I will update my post. :)

[Fesoj]

CS, look also for failed system/shell commands like sed, cp, etc. in the system logs. They might point to the real reason of failure.

[Fesoj]

Hi,

to be sure, I just attacked my pfSense box from outside and started a p2p program from inside with the appropriate rules enabled. This time I have activated blocking for both sides. The offending machines from the WAN sides were blocked as well as my LAN client that started the p2p program. But the ips of the pfsense box nics were not blocked.

I am using the latest package with the patches described earlier.

[CS]

@Fesoj

I get errors about the command /usr/bin/sed (invalid command code) and my WAN gateway is still blocked.

[miles267]

@Fesoj:

(1) Find all the classification.config files with find /usr/local -name 'classification.config*'. If you have n interfaces defined for snort you should get at least n+1 copies.

(2) Append config classification: sdf,Sensitive Data was Transmitted Across the Network,2 to the end of each file if the sdf classification is missing.

This is a long story short.

Fesoj, I followed your instructions here to the letter and it worked PERFECTLY as expected.  Snort seems to be running steadily for me now.

Would you mind posting screen captures (or e-mailin) of your IF SETTINGS and PREPROCESSOR settings for both your LAN and WAN interface?  Of course anything sensitive masked.  I just want to be sure I have mine setup correctly with regards to whitelists, home net, sensitive data settings, advanced pass thru settings, etc.

[Fesoj]

CS,

have a look at http://forum.pfsense.org/index.php/topic,51493.msg276317.html#msg276317. Since I haven't seen your system log, I don't know the exact reason of the failure, but I guess patching the sed calls in snort.inc will help.

[Fesoj]

miles267,

settings do not really matter, but which rules you enable could play a role. Also my pfsense box is not an edge router, so some issues cannot occur if you hook it up o an ISP directly and activate pfsense on the WAN NIC.

Feeling comfortable to tackle the  rules updated problem?

[mdima]

Hello,
Fesoj: I checked all the messages you send, and my installation is compliant to all that… what I think is that there is some problem with the alert_pf output plugin.

I made some test about that. The new version of the Snort package generate a whitelist for the box IPs like:
192.168.0.1/24 (referred to a single IP)
while in the old version (I just compared with a virtual machine I use as text that I didn't update) the IPs are written like:
192.168.0.1
.
So I run this test, I looked in the snort.conf file and I found:
output alert_pf: /usr/local/etc/snort/snort_35034_em2/default,snort2c,both,kill

I edited the /usr/local/etc/snort/snort_35034_em2/default file in order to change the format (for the single IPs) from:
IP/mask
to
IP
and I restarted Snort (without saving anything in order to avoid the "default" file to be recreated).
After some minutes, and some alerts generated, the only IPs that got blocked were the ones NOT included in the whitelist (as it should be and as it was before the latest package changes).
I made some investigation on the spoink plug-in, and yes, in its documentation the IPs in the whitelist are written without mask.

So, I think there is something to fix for the alert_pf package, probably the procedure that generate its whitelist.

Thanks,
Michele

[Fesoj]

Michele,

I haven't looked at the details of alert_pf yet. I am still busy with the update problem. It would be helpful if ermal finds some time to validate the latest patches so a standard installation would be at the current "patch level". Too many changes at the same time are typically a recipe for disaster.

As far as whitelisting goes, I would need to do some reading first. Currently I can cope with blocking quite easily, but I could imagine some scenarios where it gets more difficult. Before writing some code I'd like to suggest to collect some reference configurations that include the net topology, the snort interfaces, and the rules (actually sources and destinations would suffice). What I mean is, if you have a rule with ANY as the source and HOME_NET as the destination and you are blocking on the destination side, then this makes not always any sense.

PS: If you search the snort forums, you'll find several places where people have difficulties with IP ranges in whitelists, but I have not looked at the details whether this applies here, or whether these issues have been resolved, or whether the issues were due to a misunderstanding how snort works.

[Cino]

Whitelist has always been single IPs. It would be nice to use CDIR but it wasn't built that way. Ever since this change to the snort binary, https://github.com/bsdperimeter/pfsense-tools/commit/34fe38d61ba1f858f3c5bcdec6fa583a74e328a4, snort blocks everything unless its in a whitelist as single IP.

HOME_NET IP/Subnets should not be blocked at all.. If you want the system to be able to block an IP in your HOME_NET NETLIST then create a custom NETLIST that doesn't include IMHO.

But by default, HOME_NET should be exempt from blocking.. But since the binary change, this is not the case!

As far as the other issues, that has to do with the php code that Fesoj seems to be fixing, thanks Fesoj!… I'll test when i'm fully online later this week.

[Fesoj]

Cino,

But by default, HOME_NET should be exempt from blocking..

it actually depends. For example, if there is a company policy that forbids eDonkey and friends, then it could make sense to block machines in the HOME_NET. Sometimes this may be better than reporting the fellow to the management (actually 2 weeks ago I needed to attended a court case, where somebody got fired because of a "Facebook" issue) and sometimes, especially in some European countries, it might even be necessary to avoid legal actions because of excessive liability for interference claims. It all boils down to treat your colleagues or clients like little children, but this is a social problem.

[mdima]

@Fesoj:

PS: If you search the snort forums, you'll find several places where people have difficulties with IP ranges in whitelists, but I have not looked at the details whether this applies here, or whether these issues have been resolved, or whether the issues were due to a misunderstanding how snort works.

@Cino:

Whitelist has always been single IPs. It would be nice to use CDIR but it wasn't built that way. Ever since this change to the snort binary,

ok, we understood what the problem is. Before white lists were generated as "IP lists", now even the "default" whitelist is generated as a "network list" and it does not work anymore.

I am now looking at the Spoink code to see what we can do to restore this functionality… but for the moment I have to turn off the IP blocking or everything gets stuck in few minutes!

Ciao,
Michele

[Fesoj]

Michel,

after reading Cino's comment, that is in line with what I had in mind, your analysis is right. CIDR entries in the default file seem to be wrong and don't seem to work as expected. For my box the generated entries contain
(1) the loopback address
(2) the WAN address of the box (which is static in my case, so that's fine) [currently the subnet]
(3) the LAN address of the box [currently the subnet]
(4) and the DNS servers

I am wondering what happens if under point 2 the WAN address is dynamic. Would this mean that when blocking is enabled, you cannot generally avoid blocking yourself? Maybe there is an update mechanism.

[mdima]

@Fesoj:

Michel,
after reading Cino's comment, that is in line with what I had in mind, your analysis is right. CIDR entries in the default file seem to be wrong and don't seem to work as expected. For my box the generated entries contain
I am wondering what happens if under point 2 the WAN address is dynamic. Would this mean that when blocking is enabled, you cannot generally avoid blocking yourself? Maybe there is an update mechanism.

Thanks! But consider, I have about 80 VIPs, 5 NICs, it would be unconfortable to mantain a IP list aligned for the local IPs…
I mean, managing the HOME_NET with an Alias is awesome, just the whitelists should be managed as before, with an "host alias" for the part the user can decide (probably this should be noticed in the interface), and for the "default" part of the white list as single IP addresses without CIR.

Now that the problem is known and identified, I trust in Ermal to fix it... ;) Thanks!!

Michele

[Cino]

Imho I think the default home_net list should be exempt. If you need blocking, create a custom netlist.

[Fesoj]

Cino,

understood. As long as a custom list is possible.

[Fesoj]

If you see the following error message in your system log:

FATAL ERROR: /usr/local/etc/snort/snort_56964_re1/preproc_rules/sensitive-data.rules(1) Unknown ClassType: sdf

you can either add the type declaration by hand (see some of the earlier posts), or try the following patch:

https://github.com/bsdperimeter/pfsense-packages/pull/288

The reference classification.config file gets extracted from the rule set package (Snort.org and/or Emergingthreats.net), where the last download overwrites the first one. Currently the Snort.org rules are downloaded and unpacked before the ET rules, where the ET rules don't declare the sdf type. Hence the fatal error. The patch reverses the order of unpacking the archives since the Snort.org rules seem to have all the necessary declarations.

Happy patching.

UPDATE: with package vs. 2.5.1 this is obsolete

[eri--]

@Fesoj:

mdima,

HOME_NET != WHITELIST. On the WAN side, I have the default HOME_NET, but a special whitelist (including the IP address on the WAN side (I don't know what to do in case of an ISP supplied address using DHCP)) and this works fine. It's just the way things are setup. As I said before, it is now easier than ever to shoot oneself in the foot.

That is why the option of add wan ips is there.
It will get the right address when theip changes. at least on 2.0+

[Cino]

It's been part of the GUI for years and is still there. For custom Netlist.

[Fesoj]

It will get the right address when theip changes. at least on 2.0+

I checked that and the IP of the WAN nic does not get blocked.

[Fesoj]

ermal,

That is why the option of add wan ips is there.

part of the discussion today was about whether the IP ranges actually work in the whitelist (alert_pf). Essentially, they don't work, but I might be wrong.

[eri--]

I pushed some fixes to snort to prevent issues with so rules.
Also changed a bit the upgrade process.

Please test again.

Fesoj,

I tested the whitelist features and they work with ranges.
Have you installed teh latest binary?