Snort 2.9.4.1 Pkg 2.5.4 – Fix for SO rules version mismatch and failed startup
-
There is a small typo in the name of the Snort VRT rules file in the new Snort 2.9.4.1 package released today. I will submit a Pull Request to the developers for a permanent fix, but in the meantime if you are experiencing startup failures caused by a Shared Objects rule version mismatch, here is a workaround.
Use the Diagnostics…Edit File menu option and browse to /usr/local/pkg/snort and open the file snort.inc in the editor.
Near the top of that file, find the line that reads as follows:
$snort_rules_file = "snortrules-snapshot-2940.tar.gz";
Change that line to read as follows instead:
$snort_rules_file = "snortrules-snapshot-2941.tar.gz";
Click Save in the dialog to save the change.
Go to the Snort Service and update the rules again. That should allow Snort to start. The error is caused by the new code downloading the rule package for 2.9.4.0 Snort instead of 2.9.4.1. The Shared Objects (SO) pre-compiled rules changed in 2.9.4.1 and were compiled with a newer library.
Bill
-
DAMN GOOD Bill!!
-
Getting this error still.. despite changing the snort.inc file.
snort[51278]: FATAL ERROR: The dynamic detection library "/usr/local/lib/snort/dynamicrules/web-misc.so" version 1.0 compiled with dynamic engine library version 1.15 isn't compatible with the current dynamic engine library "/usr/local/lib/snort/dynamicengine/libsf_engine.so" version 1.17.
-
Same problem however if you go to /usr/local/lib/snort/dynamicrules/ and delete everything it starts fine.
-
Same problem however if you go to /usr/local/lib/snort/dynamicrules/ and delete everything it starts fine.
Sorry not sure if I made it clear.
Delete the files
Change the code
Download the rules
Start snort
-
I just executed a Pull Request to the pfSense developers with a fix for this problem. As soon as one of them sees and accepts it, this error should be fixed.
UPDATE – This change has been pushed to production. If you are experiencing the "FATAL ERROR" problem with dynamic shared object rules, then reinstall the GUI components of the Snort package. After reinstalling the GUI components, THEN update your rules using the UPDATE tab in Snort.
Bill
-
Snort started fine after this mod (thanks!), however it is blocking my WAN connection so had to disable snort for now.
-
Still not working for me, Bill. Reinstalled and did the reinstall of GUI Components as well and I'm getting error during rule update. After update Installed Signature Ruleset for Snort.org still says N/A
Mar 21 22:26:15 php: /snort/snort_download_rules.php: The Rules update has finished... Mar 21 22:26:15 php: /snort/snort_download_rules.php: Snort has restarted with your new set of rules... Mar 21 22:26:13 SnortStartup[58102]: Snort START For HTTP Inspect(30901_em1)... Mar 21 22:26:13 snort[58095]: FATAL ERROR: The dynamic detection library "/usr/local/lib/snort/dynamicrules/web-misc.so" version 1.0 compiled with dynamic engine library version 1.15 isn't compatible with the current dynamic engine library "/usr/local/lib/snort/dynamicengine/libsf_engine.so" version 1.17. Mar 21 22:25:51 SnortStartup[56657]: Snort START For HTTP Inspect(30901_em1)... Mar 21 22:25:51 snort[56363]: FATAL ERROR: The dynamic detection library "/usr/local/lib/snort/dynamicrules/web-misc.so" version 1.0 compiled with dynamic engine library version 1.15 isn't compatible with the current dynamic engine library "/usr/local/lib/snort/dynamicengine/libsf_engine.so" version 1.17. Mar 21 22:25:47 php: /snort/snort_download_rules.php: Resolving and auto-enabling flowbit required rules for WAN... Mar 21 22:25:46 php: /snort/snort_download_rules.php: Checking for and disabling any rules dependent upon disabled preprocessors for WAN... Mar 21 22:25:45 php: /snort/snort_download_rules.php: Updating rules configuration for: WAN ... Mar 21 22:25:45 php: /snort/snort_download_rules.php: Emergingthreats rules file update downloaded succsesfully Mar 21 22:25:43 php: /snort/snort_download_rules.php: There is a new set of Emergingthreats rules posted. Downloading... Mar 21 22:25:42 php: /snort/snort_download_rules.php: Please wait... You may only check for New Rules every 15 minutes... Mar 21 22:25:42 php: /snort/snort_download_rules.php: Snort MD5 Attempts: 5 Mar 21 22:24:01 check_reload_status: Reloading filter Mar 21 22:24:01 check_reload_status: Syncing firewall Mar 21 22:23:58 php: /pkg_mgr_install.php: Resolving and auto-enabling flowbit required rules for WAN... Mar 21 22:23:57 php: /pkg_mgr_install.php: Checking for and disabling any rules dependent upon disabled preprocessors for WAN... Mar 21 22:23:56 php: /pkg_mgr_install.php: The dir for /usr/pbi/snort-amd64/etc/snort/snort_30901_em1/snort.conf does not exist. Cannot add symlink to /usr/local/etc/snort/snort_30901_em1/snort.conf. Mar 21 22:23:56 php: /pkg_mgr_install.php: The dir for /usr/pbi/snort-amd64/etc/snort/snort_30901_em1/threshold.conf does not exist. Cannot add symlink to /usr/local/etc/snort/snort_30901_em1/threshold.conf. Mar 21 22:23:53 php: /pkg_mgr_install.php: Resolving and auto-enabling flowbit required rules for WAN... Mar 21 22:23:51 php: /pkg_mgr_install.php: Checking for and disabling any rules dependent upon disabled preprocessors for WAN... Mar 21 22:23:50 php: /pkg_mgr_install.php: The Rules update has finished... Mar 21 22:23:50 php: /pkg_mgr_install.php: Snort has restarted with your new set of rules... Mar 21 22:23:28 php: /pkg_mgr_install.php: Resolving and auto-enabling flowbit required rules for WAN... Mar 21 22:23:26 php: /pkg_mgr_install.php: Checking for and disabling any rules dependent upon disabled preprocessors for WAN... Mar 21 22:23:25 php: /pkg_mgr_install.php: Updating rules configuration for: WAN ... Mar 21 22:23:25 php: /pkg_mgr_install.php: Emergingthreats rules file update downloaded succsesfully Mar 21 22:23:25 php: /pkg_mgr_install.php: There is a new set of Emergingthreats rules posted. Downloading... Mar 21 22:23:25 php: /pkg_mgr_install.php: Please wait... You may only check for New Rules every 15 minutes... Mar 21 22:23:25 php: /pkg_mgr_install.php: Snort MD5 Attempts: 5 Mar 21 22:21:49 php: /pkg_mgr_install.php: Beginning package installation for snort .
-
Snort.org ruleset not updating here as well. Service starts but gives a dozen of these in system logs.
Mar 21 22:32:12 snort[37838]: Non ip() parameter passed with white list, skipping…
Mar 21 22:32:12 snort[37838]: Non ip() parameter passed with white list, skipping…
Mar 21 22:32:12 snort[37838]: Non ip() parameter passed with white list, skipping…
-
OK, Removed Snort package, then ran "find /* | grep -i snort | xargs rm -rv" to remove all left over traces of Snort and now I'm up to speed with Asterix with the exact error and issue. Is this Snort.org's problem? Seems it takes forever to download the md5 file during the update process.
-
Rule updates have been slow for me as well. Could be a Snort.org problem. I am also getting the "non IP in whitelist" errors. Don't know yet what is causing them. I did not create the binary update to 2.9.4.1. I've just worked on the GUI parts. The "non IP in whitelist" error could be coming from either place. That is, it could be the new Snort binary itself, or an interaction with the GUI and the new binary. The GUI code is essentially unchanged from Snort 2.9.2.3 to 2.9.4.1.
I will try and determine exactly what is causing them. Very likely, if the whitelist is not being parsed correctly, that WAN IP addresses will get blocked by Snort.
Bill
-
Anyone else seeing WAN connections being blocked? I have a dual WAN setup, AMD64 2.0.2
-
In regards to the md5 file problem, I might be way off on this but on the Snort.org site, there are no 2.9.4.1 ruleset updates for Registered Users, only Subscribers. Wondering if that 30 day wait between Registered and Subscribed is the reason.
-
Yeah the WAN drop happened to me once but that was a few hours ago. I have an idea to why the snort.org aren't working. Snort tries to get the snapshot 2941. Subscribers can get v2941 but register users can only get v2940. Being as I am a registered user, can only get v2940
-
read my mind
In regards to the md5 file problem, I might be way off on this but on the Snort.org site, there are no 2.9.4.1 ruleset updates for Registered Users, only Subscribers. Wondering if that 30 day wait between Registered and Subscribed is the reason.
-
Good catch on the 2.9.4.1 snapshot versus 2.9.4.0. Subscribers can get 2.9.4.1, but Registered users can only get the 2.9.4.0. That may be a difficult nut to crack because it appears the 2.9.4.1 code must use the 2.9.4.1 rules tarball to work.
As for the other error on parsing the whitelist, I've identified the source of the error message as the Spoink plug-in that Snort on pfSense uses to actually do the blocking. This is a third-party module that Ermal heavily modified to work within pfSense. It is now, for some reason, apparently choking on parsing the whitelist file supplied to it. Don't know why yet. Sent Ermal a request just now to check it out and see if he sees something.
I will investigate options for the 2.9.4.0 versus 2.9.4.1 rules update issue. That one may not sort itself out until April 1 when the 2.9.4.1 rules officially become "30 days old".
Bill
-
@bmeeks
Are you sure its April 1st? Thought it might be April 19th with the datestamp on the current rulesets.@Shinzo
;D
-
WAN blocked a couple of times now. Uninstalling Snort for now.
-
Also, Snort continues to block WAN connections, even with interfaces disabled in the snort GUI. You need to uninstall it, or kill snort via command line.
-
Hello all
Steps worked for me :
Install Snort.org rules - Do NOT Install
Resolve Flowbits - Unchecked( So Emerging Threats rules only )
rm /usr/local/lib/snort/dynamicrules/*
Start. Working!
-
Also, Snort continues to block WAN connections, even with interfaces disabled in the snort GUI. You need to uninstall it, or kill snort via command line.
I submitted a request for the main developer to take a look at this problem. Apparently the Spoink plug-in used to perform the actual blocking on pfSense is not correctly parsing the whitelist file of IP addresses. At least that is the source of the error messages in the log upon startup saying a "…non IP parameter was detected and skipped..." in the whitelist.
Bill
-
Hello all
Steps worked for me :
Install Snort.org rules - Do NOT Install
Resolve Flowbits - Unchecked( So Emerging Threats rules only )
rm /usr/local/lib/snort/dynamicrules/*
Start. Working!For those of you that are NOT Snort VRT rule subscribers, the only fix for now is to NOT use the Snort rules and use just Emerging Threats until the 30-day window elapses and the VRT "registered user" rules version up to 2.9.4.1. I did some Google research, and this seems to be a common issue with each version update of the Snort binary. The Shared Object rules can get recompiled to work only with a newer dynamic rule library.
Bill
-
Aint that nice…............ :(
-
Aint that nice…............ :(
Yeah, but I guess from the point of view of the Snort VRT, this is a "carrot" to entice folks to buy subscriptions instead of using the free registered user rules. To stay current with the rules, they drive you to the subscription model. Can't really fault them for that.
There is not much that can be done on the pfSense side except to fall back to the older 2.9.4.0 binary. There are down sides to that as well.
-
Its understandable, but very frustrating from the enduser perspective…. :(
-
I found the bug that is causing the whitelist parsing failure in 2.9.4.1 and the error messages saying "…non IP() parameter found...".
A misplaced call to clear memory in the Spoink plug-in is the culprit. The buffer is being initialized AFTER being filled with data instead of BEFORE being filled with data to parse. The function in the Spoink plug-in then calls a Snort API to parse the IP address data. Because it inadvertently zeroes out the buffer prior to the Snort API call, then Snort returns a parsing error because there is nothing to parse. The end result is the whitelist does not get populated, and thus WAN IPs get blocked.
Here is the errant code snippet:
while((ret = s2c_parse_line(cad, wfile)) != 0) { memset(cad, 0, WLMAX); if (ret == 1) { ipw = malloc(sizeof(struct ipwlist)); if (ipw == NULL) { ErrorMessage("Could not allocate memory"); continue; } if (sfip_pton(cad, &ipw->waddr) != SFIP_SUCCESS) { ErrorMessage("Non ip(%s) parameter passed with white list, skipping...", cad); free(ipw); continue; } // else //printf("IP(%s) parsed succesfuly", cad);Notice the memset() call immediately after the while() statement. That is zeroing out the buffer containing the IP address to be parsed. The memset() function call should be BEFORE the while() statement instead of AFTER.
Bill
-
Have you sent this to Ermal?
-
Have you sent this to Ermal?
Yes. I have not received a reply yet, but he and I are in vastly different time zones and he may not have seen the note yet.
-
Allright mate!! You doing a hell of a job for the rest of the community!
-
bmeeks…very impressed with your debugging abilities :-) As a guy who coded a long time back, it's refreshing to see someone who can get under the hood and ID issues so quickly.
Cheers,
Dennis.
-
bmeeks…very impressed with your debugging abilities :-) As a guy who coded a long time back, it's refreshing to see someone who can get under the hood and ID issues so quickly.
Cheers,
Dennis.Thanks, but I must admit I stared at that code for like an hour and did not see the bug. I knew it had to be there somewhere, and finally I noticed the misplaced memset() call. After that it was like … Doh!! ... why didn't I see that first thing ?? :D
Bill
-
Please let us know when it will be fixed.
So we can pull new package.
Thanks a lot for all efforts
-
Please let us know when it will be fixed.
So we can pull new package.
Thanks a lot for all effortsI have submitted a Pull Request via GitHub that contains a fix for the whitelist parsing issue. The pfSense developers have to accept my patch into the packages repository and then compile it into the new binary for Snort. Changes to the binary are a bit more involved to publish than changes to the GUI code.
Bill
-
Any news on this ?
-
waiting as well for this.
-
Any news on this ?
My Pull Request for the Spoink patch was accepted, but so far it has not been incorporated into a new build of the binary as far as I can tell. I don't know what the process is nor the timeline for the binary side. On the GUI side, once a Pull Request is accepted by the Core Team it is immediately available for download. I know the binaries have to be built, but I don't know if that is automated (I think it is) or a human has to intervene.
Bill
-
Just uninstall , then install package.
It looks like is NOT rebuilt yet , I see my WAN blocked . :(
(Snort 2.9.4.1 Pkg 2.5.4 , Emergingthreats rules only )
Thanks
-
I reported that to Bmeeks some time ago since I saw my WAN blocked as well. It must be the implementation of Snort into PFsense that is causing this behaviour…
-
I found my WAN blocked yesterday. I removed the block, but it came right back. Restarted the service and it has been running fine since. Never saw this before upgrading to Snort 2.9.4.1.
If it matters, I do have the paid VRT rules. (Well worth $2.50/month. I think it's a good value and money worth spent)
-
I found my WAN blocked yesterday. I removed the block, but it came right back. Restarted the service and it has been running fine since. Never saw this before upgrading to Snort 2.9.4.1.
If it matters, I do have the paid VRT rules. (Well worth $2.50/month. I think it's a good value and money worth spent)
Until my latest bug fix is incorporated into the binary build of Snort on pfSense, you will see your WAN IP (and any other normally whitelisted IPs) get blocked. The blocking of offenders in Snort on pfSense is done with an optional output plugin.
Snort, natively, has no "blocking" capability. The Snort team leaves that to others. There are two popular methods in use: Snortsam and Spoink. The pfSense folks chose Spoink. This works as an optional output plugin compiled into the Snort binary. The Snort source code is patched during the pfSense package build process to incorporate the Spoink output plugin. This plugin receives each Alert from Snort as it is on the way to the log files. It compares the IP addresses in the Alert (SRC, DST or BOTH according to how you configure blocking) to the list of Whitelist IPs. If the offending IP is NOT in the whitelist, then an API call is made into the pfSense packet filter code to insert a blocking rule for that IP. The IP whitelist is just a text file in the same directory as the Snort configuration files. That file is created by the GUI code and then read at Snort startup by the Spoink plugin patched into Snort.
The bug that got introduced in 2.9.4.1 is in the Spoink plugin patch. During startup, when it reads the Whitelist file and stores the addresses in there into the in-memory table of whitelist IP addresses, it zeroes out the data it reads from the file just prior to parsing it! So it sees an "empty" whitelist file and thus blocks ALL alerting IP addresses. The intent of the zero-out call was to initialize the buffer with zeros prior to reading in the whitelist, but the memory clearing call was typed in the wrong spot such that it clears the buffer immediately after it was just filled with the file's data. I submitted a fix for this bug, but it has not made its way into the compiled binary package yet. Until it gets fixed, this bug will keep causing people issues with their WAN IP and other normally whitelisted IPs getting blocked.
Bill
