Snort 2.9.4.1 Package with 03/20/13 Snapshot

rudger_wolvram

I just noticed today that there was an update to snort from 2.9.3.1 (whatever the latest was 5 days ago) to 2.9.4.1.

So I upgrade, but snort won't start anymore. I check the logs and find:
snort[pid]: FATAL ERROR: Failed to initialize dynamic preprocessor: SF_DCERPC2 (IPV6) version 1.0.3 (-2)

So my first thought it is to disable the preprocessor and then disable the rules that complain that need it, but to my suprise, when I try to start snort after disabling that preprocessor, I get the same exact error, not an error that says "could not load X rule because I don't know what dce/rpc is", which is what I was expecting.

I got into the shell and start rooting around for where the snort conf/rules are stored. I finally find them, but notice that preprocessor rules that I have disabled are not commented out in the snort.conf in /usr/local/etc/snort/. So now I'm either confused or have found a bug.

I would be confused if I'm looking in the most obvious place for the configuration and that actually is not it, and it's a reference file for the actual config.
Or
I found a bug where the GUI does not comment out or uncomment the appropriate sections of the snort config in 2.9.4.1.

A bug would explain why I'm getting the error because snort rules from 2.9.not4.X would not be entirely compatible with the newest version, but why the GUI isn't doing what I would expect it to is somewhat perplexing.

So my question, am I confused and I am looking in the wrong place to clean up old rules, or is it a bug?

P.S./Sidenote
Another odd thing I noticed, before I upgraded snort and after I upgraded to the latest snapshot (03/20/13).
I started getting snort alerts for IPv6 addresses, I ran snort for a solid month without restarting, I have never recieved an IPv6 alert.
I thought it might have been my ISP keeping a record of what would be my IPv6 address, even though I don't have it enabled on the interface, and had routed a malicous packet accordingly.
I don't know if something was enabled in the Wed snapshot and I missed it, or something goofy just happened to occur after a snapshot upgrade.

Gradius

Snort were running fine, until today's update (03/23).

Snort 2.9.4.1 pkg v2.5.4:

Mar 23 01:01:08 	php: /status_services.php: The command '/usr/local/etc/rc.d/snort.sh stop' returned exit code '1', the output was ''
Mar 23 01:01:10 	snort[91928]: FATAL ERROR: The dynamic detection library "/usr/local/lib/snort/dynamicrules/web-misc.so" version 1.0 compiled with dynamic engine library version 1.15 isn't compatible with the current dynamic engine library "/usr/local/lib/snort/dynamicengine/libsf_engine.so" version 1.17.

Gradius

Tried to disable web-misc.so, but no luck, I'm still getting same error.

Time to fix "/usr/local/lib/snort/dynamicrules/web-misc.so" to be compatible with "/usr/local/lib/snort/dynamicengine/libsf_engine.so" version 1.17.

-rwxr-xr-x  1 root  wheel   62244 Mar 21 00:04 web-misc.so

Looks "updated" but someone forgot to compile it to be compatible with libsf_engine.so v1.17 ::)

miles267

Snort.org rules don't even appear to download at all now for some reason. Not quite sure what changed. Only emerging rules are downloading.

bmeeks

@miles267:

Snort.org rules don't even appear to download at all now for some reason. Not quite sure what changed. Only emerging rules are downloading.

The new Snort binary requires the 2.9.4.1 rules tarball because the precompiled Shared Object rules need a newer library. Initially the new Snort package contained code to download the 2.9.4.0 rules tarball, but if you enabled any Shared Object rules then Snort would not start. The code was changed to download the 2.9.4.1 rules and that solved the Shared Object problem. Unfortunately, the Snort VRT model for rules downloads requires rules to be 30 days old before they "transition" over to the free "Registered Users" from the paying "Subscribers" (those that pay the annual Snort VRT subscription for the newest rules). So the 2.9.4.1 rules tarball will only download from Snort.org for folks whose Oinkcode qualifies them for the latest "paid rules" instead of the older "free rules".

This is a Snort.org issue and not directly a pfSense issue. There are two solutions. One is to not use the Snort VRT rules until the 2.9.4.1 version rules become "free". This should happen in about 30 days. The other solution, if the Snort VRT rules are really important to you, is to pay the annual subscription and upgrade your Oinkcode to one that has access to the paid rules. For personal use, I think it's $29 per year.

RonpfS

Could you include the rules in the package install for about '30 days' ? ;D ::)

bmeeks

@RonpfS:

Could you include the rules in the package install for about '30 days' ? ;D ::)

Only if we liked to be sued by copyright lawyers… ;D

Gradius

Is $29.99/year for personal use:
http://www.snort.org/vrt/buy-a-subscription/

I don't mind to pay that but this looks like they are pushing into more subscribes by force (not cool). ::)

Besides, they won't accept PayPal only CC directly by them. :o

rudger_wolvram

I've deselected everything but a handful of emerging threat rules, no snort rules are selected. The DCE/RPC preproc still fails to load. I'm guessing when the package was upgraded, it left behind some prior version settings that the new package isn't seeing. e.g. unselecting the DCE/RPC preproc is removing/commenting the config for it for the latest version, but the previous versions config is still in the same file, hence it breaks it. At least that's my best guess. I'm trying to avoid rebuilding and reinstalling my snort installation. That would be the last thing I try.

Also, I noticed there are symlinks to some config files, the links are RO while the actual files are RW. I'm assuming that if I wanted to modify the config outside of the gui, I would modify the RW. However, I don't know where the snort install is actually reading it's config from or where it's getting it's rules. I can only assume the RW snort.conf and rules folder are the primary, and the symlinks are for the package to read from. The main reason I'm not sure is, I would watch the files and see what changed as I selected/deselected and saved preroc settings, I did not see the DCE/RPC get commented/uncommented in the snort.conf I was looking at when changing the setting.

Files referenced:
/usr/local/etc/snort/ - where the RO symlinks are
/usr/pbi/snort-i386/etc/snort/ - where the RW files are, also where I assume i need to modify files.

rudger_wolvram

After some more digging I'm even more confused.
It seems that my config has been lost across package installs. I'm guessing because of the below line, the oddly named snort_34124_re1 says to me, custom config storage.
php: /pkg_mgr_install.php: The dir for /usr/pbi/snort-i386/etc/snort/snort_34124_re1/threshold.conf does not exist. Cannot add symlink to /usr/local/etc/snort/snort_34124_re1/threshold.conf.

After looking at the symlinks i noticed something odd for snort.conf
Inside /usr/pbi/snort-i386/etc/snort/
snort.conf -> /usr/local/etc/snort/snort.conf
Inside /usr/local/etc/snort/
snort.conf -> /usr/pbi/snort-i386/etc/snort/snort.conf-sample

So it's just a loop, well almost. But that it's pointing to a sample config really sets off alarms saying that what is in the GUI is not what is actually happening.

rudger_wolvram

And a litle more testing. It seems once I disable a pre-processor, in order to really disable it, I have to reinstall snort.
I fixed the DCE problem after reinstalling, then the SSL preproc started having fits, I disabled it, saved, and got the same error again, except for the SSL preproc, not the DCE/RPC. I reinstalled snort, with the SSL pre-proc disabled, it picked up on the FTP preproc complaing. So I disabled all preprocs. reinstalled, now it "acts" normal, however, it terminates with a sig 11.

rudger_wolvram

And it's now fixed.

I deselected all snort categories I had enabled. (Since I don't have a subscription)
Left my emerging threat rules alone.
Deselected all pre-processors.
Reinstalled snort.
Deleted everything under all dynamic folders under:
/usr/local/lib/snort/dynamic* (cleared each folders files, I didn't just wipe the folders out)
Started up snort, started getting alerts and blocks from Russia (normal)
Everything seems to be working fine now.

P.S.
I did lose network for a moment while snort started, like 10 seconds worth. Not sure if that was because it set the NIC to promiscuous mode then decided to rebuild everything, meanwhile the nic was in a weird state.

novacoresystems

Alright, so I was having the same issue as you since I update to the newest snapshot almost every day. I've been struggling with this for the past 3 days. I saw your post and what you did just now and decided to just delete the directory you specified above and reinstall snort. That fixed it lol.

Simple as that. Now one thing to note is that this also happens on a fresh install of pfsense using the newest snapshots from the past 3-4 days. So even on a fresh install it happens… something is broken and should be addressed. Looks like those dynamic files shouldn't be there? They seem to be causing issues...

Supermule

I agree :)

bmeeks

I've done most of my work and testing on 2.0.2 and not 2.1-BETA. I do have a 2.1-BETA virtual machine for testing, and it has some weirdness with removing and reinstalling Snort.

I have not studied the "install and remove" code in the Snort package enough to understand how it works (at least not completely). I agree it could definitely be improved, though. The higher priority item for me at the moment is the WAN IP blocking problem caused by the whitelist parsing bug in the Spoink plug-in code. Because of this bug, I have not yet upgraded my own production Snort install with the 2.9.4.1 port. I have just been working with the new version in VMware virtual machines to test it.

I am working on completing my pfSense package repository in VMware so I can compile packages on my own and run test installs and removes. That way, I can learn the system and perhaps make some improvements in the behavior of Snort when upgrading.

Bill

Gradius

My issue was fixed here:
http://forum.pfsense.org/index.php/topic,59473.15.html

Waiting for WAN (whitelist) fix now.

Gradius

Can you please let us know when the fix would be available?

TIA

bmeeks

@Gradius:

Can you please let us know when the fix would be available?

TIA

The WAN IP blocking problem should be fixed in the new binary recently posted. Do the typical package remove and reinstall to pickup the fix.

Bill

novacoresystems

@bmeeks:

I've done most of my work and testing on 2.0.2 and not 2.1-BETA. I do have a 2.1-BETA virtual machine for testing, and it has some weirdness with removing and reinstalling Snort.

I have not studied the "install and remove" code in the Snort package enough to understand how it works (at least not completely). I agree it could definitely be improved, though. The higher priority item for me at the moment is the WAN IP blocking problem caused by the whitelist parsing bug in the Spoink plug-in code. Because of this bug, I have not yet upgraded my own production Snort install with the 2.9.4.1 port. I have just been working with the new version in VMware virtual machines to test it.

I am working on completing my pfSense package repository in VMware so I can compile packages on my own and run test installs and removes. That way, I can learn the system and perhaps make some improvements in the behavior of Snort when upgrading.

Bill

Thanks Bill for all the hard work! I just wanted to make sure the issue didn't go unnoticed.

Gradius

@bmeeks:

The WAN IP blocking problem should be fixed in the new binary recently posted. Do the typical package remove and reinstall to pickup the fix.

Bill

Thanks Bill!