Snort 2.9.4.1 Pkg 2.5.4 – Fix for SO rules version mismatch and failed startup



  • read my mind

    @AhnHEL:

    In regards to the md5 file problem, I might be way off on this but on the Snort.org site, there are no 2.9.4.1 ruleset updates for Registered Users, only Subscribers.   Wondering if that 30 day wait between Registered and Subscribed is the reason.

    https://www.snort.org/snort-rules/



  • Good catch on the 2.9.4.1 snapshot versus 2.9.4.0.  Subscribers can get 2.9.4.1, but Registered users can only get the 2.9.4.0.  That may be a difficult nut to crack because it appears the 2.9.4.1 code must use the 2.9.4.1 rules tarball to work.

    As for the other error on parsing the whitelist, I've identified the source of the error message as the Spoink plug-in that Snort on pfSense uses to actually do the blocking.  This is a third-party module that Ermal heavily modified to work within pfSense.  It is now, for some reason, apparently choking on parsing the whitelist file supplied to it.  Don't know why yet.  Sent Ermal a request just now to check it out and see if he sees something.

    I will investigate options for the 2.9.4.0 versus 2.9.4.1 rules update issue.  That one may not sort itself out until April 1 when the 2.9.4.1 rules officially become "30 days old".

    Bill



  • @bmeeks
    Are you sure its April 1st?  Thought it might be April 19th with the datestamp on the current rulesets.

    @Shinzo
    ;D



  • WAN blocked a couple of times now. Uninstalling Snort for now.



  • Also, Snort continues to block WAN connections, even with interfaces disabled in the snort GUI.  You need to uninstall it, or kill snort via command line.



  • Hello all
    Steps worked for me :
    Install Snort.org rules - Do NOT Install
    Resolve Flowbits  - Unchecked

    ( So Emerging Threats rules only )

    rm /usr/local/lib/snort/dynamicrules/*
    Start. Working!



  • @dwood:

    Also, Snort continues to block WAN connections, even with interfaces disabled in the snort GUI.  You need to uninstall it, or kill snort via command line.

    I submitted a request for the main developer to take a look at this problem.  Apparently the Spoink plug-in used to perform the actual blocking on pfSense is not correctly parsing the whitelist file of IP addresses.  At least that is the source of the error messages in the log upon startup saying a "…non IP parameter was detected and skipped..." in the whitelist.

    Bill



  • @vizavi:

    Hello all
    Steps worked for me :
    Install Snort.org rules - Do NOT Install
    Resolve Flowbits  - Unchecked

    ( So Emerging Threats rules only )

    rm /usr/local/lib/snort/dynamicrules/*
    Start. Working!

    For those of you that are NOT Snort VRT rule subscribers, the only fix for now is to NOT use the Snort rules and use just Emerging Threats until the 30-day window elapses and the VRT "registered user" rules version up to 2.9.4.1.  I did some Google research, and this seems to be a common issue with each version update of the Snort binary.  The Shared Object rules can get recompiled to work only with a newer dynamic rule library.

    Bill


  • Banned

    Aint that nice…............ :(



  • @Supermule:

    Aint that nice…............ :(

    Yeah, but I guess from the point of view of the Snort VRT, this is a "carrot" to entice folks to buy subscriptions instead of using the free registered user rules.  To stay current with the rules, they drive you to the subscription model.  Can't really fault them for that.

    There is not much that can be done on the pfSense side except to fall back to the older 2.9.4.0 binary.  There are down sides to that as well.


  • Banned

    Its understandable, but very frustrating from the enduser perspective…. :(



  • I found the bug that is causing the whitelist parsing failure in 2.9.4.1 and the error messages saying "…non IP() parameter found...".

    A misplaced call to clear memory in the Spoink plug-in is the culprit.  The buffer is being initialized AFTER being filled with data instead of BEFORE being filled with data to parse.  The function in the Spoink plug-in then calls a Snort API to parse the IP address data.  Because it inadvertently zeroes out the buffer prior to the Snort API call, then Snort returns a parsing error because there is nothing to parse.  The end result is the whitelist does not get populated, and thus WAN IPs get blocked.

    Here is the errant code snippet:

    	while((ret = s2c_parse_line(cad, wfile)) != 0) {
    	              memset(cad, 0, WLMAX);
    		if (ret == 1) {
    			ipw = malloc(sizeof(struct ipwlist));
    			if (ipw == NULL) {
    				ErrorMessage("Could not allocate memory");
    				continue;
    			}
    			if (sfip_pton(cad, &ipw->waddr) != SFIP_SUCCESS) {
    				ErrorMessage("Non ip(%s) parameter passed with white list, skipping...", cad);
    				free(ipw);
    				continue;
    			} // else
    				//printf("IP(%s) parsed succesfuly", cad);
    
    

    Notice the memset() call immediately after the while() statement.  That is zeroing out the buffer containing the IP address to be parsed.  The memset() function call should be BEFORE the while() statement instead of AFTER.

    Bill


  • Banned

    Have you sent this to Ermal?



  • @Supermule:

    Have you sent this to Ermal?

    Yes.  I have not received a reply yet, but he and I are in vastly different time zones and he may not have seen the note yet.


  • Banned

    Allright mate!! You doing a hell of a job for the rest of the community!



  • bmeeks…very impressed with your debugging abilities :-)  As a guy who coded a long time back, it's refreshing to see someone who can get under the hood and ID issues so quickly.

    Cheers,
    Dennis.



  • @dwood:

    bmeeks…very impressed with your debugging abilities :-)  As a guy who coded a long time back, it's refreshing to see someone who can get under the hood and ID issues so quickly.

    Cheers,
    Dennis.

    Thanks, but I must admit I stared at that code for like an hour and did not see the bug.  I knew it had to be there somewhere, and finally I noticed the misplaced memset() call.  After that it was like … Doh!! ... why didn't I see that first thing ??  :D

    Bill



  • Please let us know when it will be fixed.
    So we can pull new package.
    Thanks a lot for all efforts



  • @vizavi:

    Please let us know when it will be fixed.
    So we can pull new package.
    Thanks a lot for all efforts

    I have submitted a Pull Request via GitHub that contains a fix for the whitelist parsing issue.  The pfSense developers have to accept my patch into the packages repository and then compile it into the new binary for Snort.  Changes to the binary are a bit more involved to publish than changes to the GUI code.

    Bill



  • Any news on this ?


  • Banned

    waiting as well for this.



  • @vizavi:

    Any news on this ?

    My Pull Request for the Spoink patch was accepted, but so far it has not been incorporated into a new build of the binary as far as I can tell.  I don't know what the process is nor the timeline for the binary side.  On the GUI side, once a Pull Request is accepted by the Core Team it is immediately available for download.  I know the binaries have to be built, but I don't know if that is automated (I think it is) or a human has to intervene.

    Bill



  • Just uninstall , then install package.
    It looks like is NOT rebuilt yet , I see my WAN blocked . :(
    (Snort 2.9.4.1 Pkg 2.5.4 , Emergingthreats rules only )
    Thanks


  • Banned

    I reported that to Bmeeks some time ago since I saw my WAN blocked as well. It must be the implementation of Snort into PFsense that is causing this behaviour…



  • I found my WAN blocked yesterday.  I removed the block, but it came right back.  Restarted the service and it has been running fine since.  Never saw this before upgrading to Snort 2.9.4.1.

    If it matters, I do have the paid VRT rules.  (Well worth $2.50/month.  I think it's a good value and money worth spent)



  • @priller:

    I found my WAN blocked yesterday.  I removed the block, but it came right back.  Restarted the service and it has been running fine since.  Never saw this before upgrading to Snort 2.9.4.1.

    If it matters, I do have the paid VRT rules.  (Well worth $2.50/month.  I think it's a good value and money worth spent)

    Until my latest bug fix is incorporated into the binary build of Snort on pfSense, you will see your WAN IP (and any other normally whitelisted IPs) get blocked.  The blocking of offenders in Snort on pfSense is done with an optional output plugin.

    Snort, natively, has no "blocking" capability.  The Snort team leaves that to others.  There are two popular methods in use:  Snortsam and Spoink.  The pfSense folks chose Spoink.  This works as an optional output plugin compiled into the Snort binary.  The Snort source code is patched during the pfSense package build process to incorporate the Spoink output plugin.  This plugin receives each Alert from Snort as it is on the way to the log files.  It compares the IP addresses in the Alert (SRC, DST or BOTH according to how you configure blocking) to the list of Whitelist IPs.  If the offending IP is NOT in the whitelist, then an API call is made into the pfSense packet filter code to insert a blocking rule for that IP.  The IP whitelist is just a text file in the same directory as the Snort configuration files.  That file is created by the GUI code and then read at Snort startup by the Spoink plugin patched into Snort.

    The bug that got introduced in 2.9.4.1 is in the Spoink plugin patch.  During startup, when it reads the Whitelist file and stores the addresses in there into the in-memory table of whitelist IP addresses, it zeroes out the data it reads from the file just prior to parsing it!  So it sees an "empty" whitelist file and thus blocks ALL alerting IP addresses.  The intent of the zero-out call was to initialize the buffer with zeros prior to reading in the whitelist, but the memory clearing call was typed in the wrong spot such that it clears the buffer immediately after it was just filled with the file's data.  I submitted a fix for this bug, but it has not made its way into the compiled binary package yet.  Until it gets fixed, this bug will keep causing people issues with their WAN IP and other normally whitelisted IPs getting blocked.

    Bill


  • Banned

    ERMAL WE NEED YOU URGENTLY!!



  • Bill, that explanation was maybe my favourite post ever here.  While I make no claims on code prowess, I really appreciate the under-hood explanation of what's going on.  I used to try the variety of work-arounds that are normally offered up after debugging a package.  It's a lot more time efficient however to watch posts like yours, and enter back into debugging/testing contribution phase once it looks like things "should" work.  Again thanks to all for their efforts.

    Cheers,
    Dennis.



  • @bmeeks:

    Until my latest bug fix is incorporated into the binary build of Snort on pfSense,…......

    Is it possible for us to apply this fix ourselves? If so I am sure we would all be very grateful if you could describe the solution for us.

    Kind regards



  • I wasn't aware this needed a manual package build, I just kicked one off on both the 8.1 (2.0.x) package builders.



  • package build finished and is uploaded. Entirely untested, please try it out and report back.



  • @cmb:

    package build finished and is uploaded. Entirely untested, please try it out and report back.

    Thanks for everyone's hard work on this.  :)

    I just tested out the latest build and it seems to have fixed the wan blocking problem.

    Thanks!

    -th3r3isnospoon



  • @cmb:

    I wasn't aware this needed a manual package build, I just kicked one off on both the 8.1 (2.0.x) package builders.

    Thanks!  When I submitted the Pull Request, I was also unaware that a manual build would be required.  Next time I will raise the flag for the manual rebuild of the binary.

    Is there a reason the Snort package is different from the other packages with regards to the manual build?

    Bill



  • Just uninstall , then install package.
    It looks like is working , I see same IPs blocked ,but WAN is OK so far.
    (Snort 2.9.4.1 Pkg 2.5.4 , Emergingthreats rules only )
    Thanks



  • @bmeeks:

    Thanks!  When I submitted the Pull Request, I was also unaware that a manual build would be required.  Next time I will raise the flag for the manual rebuild of the binary.

    Thanks, I'd appreciate that.

    Otherwise we end up with chicken littles who somehow extrapolate the package not getting built as "the project is dying".  ::)

    @bmeeks:

    Is there a reason the Snort package is different from the other packages with regards to the manual build?

    The 2.0.x packages aren't auto-built at all (AFAIK), I believe that only happens with PBIs. JimP is more authoritative on that subject and he's on vacation at the moment.



  • Thank you so much for the fix. Snort is up and working well



  • It appears to be working for me as well, many thanks this is much appreciated.



  • Whatever I try, I cannot get it to work.

    Mar 26 09:53:26	php: /snort/snort_interfaces.php: Toggle(snort starting) for WAN(WAN)...
    Mar 26 09:53:26	php: /snort/snort_interfaces.php: Seems preprocessor/decoder rules are missing, enabling autogeneration of them
    Mar 26 09:53:27	php: /snort/snort_interfaces.php: Checking for and disabling any rules dependent upon disabled preprocessors for WAN...
    Mar 26 09:53:35	snort[68519]: Non ip() parameter passed with white list, skipping...
    Mar 26 09:53:35	snort[68519]: Non ip() parameter passed with white list, skipping...
    Mar 26 09:53:35	snort[68519]: Non ip() parameter passed with white list, skipping...
    Mar 26 09:53:35	snort[68519]: Non ip() parameter passed with white list, skipping...
    Mar 26 09:53:35	snort[68519]: Non ip() parameter passed with white list, skipping...
    Mar 26 09:53:35	snort[68519]: Non ip() parameter passed with white list, skipping...
    Mar 26 09:53:35	snort[68519]: Non ip() parameter passed with white list, skipping...
    Mar 26 09:53:42	php: /snort/snort_interfaces.php: Interface Rule START for WAN(em0)...
    Mar 26 09:53:42	kernel: em0: promiscuous mode enabled
    Mar 26 09:58:25	kernel: pid 68798 (snort), uid 0: exited on signal 11
    Mar 26 09:58:25	kernel: em0: promiscuous mode disabled
    

    I am on the latest 2.1 snapshot, removed everything related to snort and started from scratch.
    I have an alias Whitelist with some IP's in it, so I do not understand  the "Non ip() parameter passed" error.
    And then the "signal 11 exit". Where should I look, because there is nog logging too?



  • @gogol:

    Whatever I try, I cannot get it to work.

    Mar 26 09:53:26	php: /snort/snort_interfaces.php: Toggle(snort starting) for WAN(WAN)...
    Mar 26 09:53:26	php: /snort/snort_interfaces.php: Seems preprocessor/decoder rules are missing, enabling autogeneration of them
    Mar 26 09:53:27	php: /snort/snort_interfaces.php: Checking for and disabling any rules dependent upon disabled preprocessors for WAN...
    Mar 26 09:53:35	snort[68519]: Non ip() parameter passed with white list, skipping...
    Mar 26 09:53:35	snort[68519]: Non ip() parameter passed with white list, skipping...
    Mar 26 09:53:35	snort[68519]: Non ip() parameter passed with white list, skipping...
    Mar 26 09:53:35	snort[68519]: Non ip() parameter passed with white list, skipping...
    Mar 26 09:53:35	snort[68519]: Non ip() parameter passed with white list, skipping...
    Mar 26 09:53:35	snort[68519]: Non ip() parameter passed with white list, skipping...
    Mar 26 09:53:35	snort[68519]: Non ip() parameter passed with white list, skipping...
    Mar 26 09:53:42	php: /snort/snort_interfaces.php: Interface Rule START for WAN(em0)...
    Mar 26 09:53:42	kernel: em0: promiscuous mode enabled
    Mar 26 09:58:25	kernel: pid 68798 (snort), uid 0: exited on signal 11
    Mar 26 09:58:25	kernel: em0: promiscuous mode disabled
    

    I am on the latest 2.1 snapshot, removed everything related to snort and started from scratch.
    I have an alias Whitelist with some IP's in it, so I do not understand  the "Non ip() parameter passed" error.
    And then the "signal 11 exit". Where should I look, because there is nog logging too?

    Oops!  I only submitted the patch to the 2.0.x tree.  I believe the 2.1-BETA tree is a different Git repository.  I have been working solely in the 2.0.x tree so far as Snort goes.  I'm still new at this and not 100% familiar with the pfSense processes for user code submissions.  Let me see if I can get a fork of the 2.1-BETA repository and submit the same patches into that code branch for the pfSense guys to look at.

    Bill



  • That would be greatly appreciated =).

    Could you tell us what and where to edit in the mean time?


Log in to reply