Snort VRT rule issues in Snort & Suricata – "Server returned error code 422."



  • Hi All

    I've got a pfSense 2.1.4 and I've been playing with Snort and Suricata over the last 2 days. When I set everything up last night, I got a free registered user oink code from Snort to trial snort with. At the time, it worked great, but unfortunately now I am having trouble getting the VRT rules to download in both Snort and Suricata. I've done everythign including re-installing the packages and rolling back my pfSense VM to a point where neither Snort nor Suricata were installed, and yet I can't get the VRT rules down.

    However, the VRT rules are downloading (using the same snort oink code) at another pfSense that I have at another site.

    The log says:

    	Downloading Snort VRT rules md5 file snortrules-snapshot-edge.tar.gz.md5...
    	Snort VRT rules md5 download failed.
    	Server returned error code 422.
    	Snort VRT rules will not be updated.
    

    I have already read the open thread in this forum in which the poster has the same issue but in my instance it didn't seem to help.

    Anyone have any ideas on what this could be?



  • @breakaway:

    The log says:

    	Downloading Snort VRT rules md5 file snortrules-snapshot-edge.tar.gz.md5...
    	Snort VRT rules md5 download failed.
    	Server returned error code 422.
    	Snort VRT rules will not be updated.
    

    I have already read the open thread in this forum in which the poster has the same issue but in my instance it didn't seem to help.

    Anyone have any ideas on what this could be?

    This particular log message must be from a Suricata box because it is trying to download the old snortrules-snapshot-edge.tar.gz files.  The current 1.4.6 version of the Suricata package on pfSense tries to download this file, but unfortunately the Snort VRT discontinued posting this file in early July.  The Snort package will not have this problem because it downloads the current file for the specific Snort version.  Suricata, however, does not.  It was attempting to always use the "most current edge rules snapshot", but as I said, the VRT discontinued that file in early July.

    There is a new Suricata update posted for review by the pfSense developers. It will take them a while to review because it contains lots of code updates.  One of the fixes in the update deals with this issue.

    In the meantime, you can either drop use of the Snort VRT rules in Suricata until the new version is out, or you can manually edit a file on the box to work around the problem.  Only edit the file if you are experienced in such tasks within pfSense.  With that warning, here are the steps:

    1. Open the file /usr/local/pkg/suricata/suricata.inc using Diagnostics…Edit File from the pfSense menu.

    2.  Near the top of the file, locate this line –

    // Rule set download filenames and prefixes
    
    define('VRT_DNLD_FILENAME', 'snortrules-snapshot-edge.tar.gz');
    
    

    and change it to read as follows –

    
    define('VRT_DNLD_FILENAME', 'snortrules-snapshot-2962.tar.gz');
    
    

    3. Save the changes.

    That should fix it on Suricata.

    Bill



  • I can confirm that the above solutions works (see screenshot attached;). Will the Snortrules snapshot number e.g. "snortrules-snapshot-2962.tar.gz" need to be changed accordingly with the most recent number in order to get the right update?

    Cheers!

    ![Screen Shot 2014-08-31 at 9.04.41 PM.png](/public/imported_attachments/1/Screen Shot 2014-08-31 at 9.04.41 PM.png)
    ![Screen Shot 2014-08-31 at 9.04.41 PM.png_thumb](/public/imported_attachments/1/Screen Shot 2014-08-31 at 9.04.41 PM.png_thumb)



  • @n3tninj4:

    I can confirm that the above solutions works (see screenshot attached;). Will the Snortrules snapshot number e.g. "snortrules-snapshot-2962.tar.gz" need to be changed accordingly with the most recent number in order to get the right update?

    Cheers!

    Yes.  In the new Suricata package that will be out soon, there is a new text box on the GLOBAL SETTINGS tab that you must type the Snort VRT rules snapshot file name into.  In the current version of Suricata on pfSense, the VRT snapshot file name is hard-coded.  So if you want the most recent rules, you need to periodically check the Snort VRT web site and edit the file referenced in my post above accordingly.

    Hopefully the VRT will change their mind and go back to posting the old snapshot-edge file on their rules download site.  That was actually a type of symbolic link to the most current version of the VRT rules.

    Bill



  • Hey Bill,

    Thanks again for your awesome solution and answer to my question. Would you mind posting what should be changed specifically and how to change it here when the new update comes and/or possibly post a link to the VRT page that has the info? I am certainly capable of doing it myself but I thought it might be useful for others who don't know how or need a little more info to get things working properly.

    Thanks again,
    n30



  • @n3tninj4:

    Hey Bill,

    Thanks again for your awesome solution and answer to my question. Would you mind posting what should be changed specifically and how to change it here when the new update comes and/or possibly post a link to the VRT page that has the info? I am certainly capable of doing it myself but I thought it might be useful for others who don't know how or need a little more info to get things working properly.

    Thanks again,
    n30

    Yes, I will need to prepare quite an extensive set of RELEASE NOTES for this new version when it is posted for download.  Attached to this post is a picture of how the new page will look for providing the Snort VRT rules snapshot filename.

    Bill




  • I apologize for bumping this old thread, Bill  :-[

    But  :-\

    [quote]
    I can confirm that the above solutions works (see screenshot attached;). Will the Snortrules snapshot number e.g. "snortrules-snapshot-2962.tar.gz" need to be changed accordingly with the most recent number in order to get the right update?

    This rule number changes with every update, right? So every time a new updated rule set is released, the number increases +1.

    Isn't this horribly non-automation, if you manually have to go and:
    A. First look up the new rule number on the site;
    B. Log in to pfSense to update the rule number in Suricata;

    ?

    I mean, this looks more like automation 1960 than automation 2015. While it seems they sold us computers by saying they would automate things, so we don't need to do it manually and can spend our time more useful (time, together with health: the two most important assets we humans have, says the economist - me).

    I have Suricata on WAN, and Snort on WAN2 (testing Suricata on WAN). Snort does not require a filename, Suricata does. Is this the Suricata project making a mess of it, or is it the Snort project making it hard on Suricata to get the rules?

    Bye Bill  ;D


  • Banned

    @Mr.:

    This rule number changes with every update, right? So every time a new updated rule set is released, the number increases +1.

    That snapshot certainly is NOT released every day. There are 3 downloadable snapshots ATM, which match most recent supported Snort versions. Cannot see how you imagine to automate it, it's really not +1 per day.



  • @Mr.:

    I apologize for bumping this old thread, Bill  :-[

    But  :-\

    This rule number changes with every update, right? So every time a new updated rule set is released, the number increases +1.

    Isn't this horribly non-automation, if you manually have to go and:
    A. First look up the new rule number on the site;
    B. Log in to pfSense to update the rule number in Suricata;

    ?

    I mean, this looks more like automation 1960 than automation 2015. While it seems they sold us computers by saying they would [b]automate things, so we don't need to do it manually and can spend our time more useful (time, together with health: the two most important assets we humans have, says the economist - me).

    I have Suricata on WAN, and Snort on WAN2 (testing Suricata on WAN). Snort does not require a filename, Suricata does. Is this the Suricata project making a mess of it, or is it the Snort project making it hard on Suricata to get the rules?

    Bye Bill  ;D

    The Snort VRT rules update to match the current Snort binary version.  However, they do keep some older versions around as well.  They have a defined support cycle for each version.  You can find the details on the snort.org web site.  The current binary version is 2.9.7.3, so the current Snort VRT tarball filename is snortrules-snapshot-2973.tar.gz.

    The Snort package uses a trick with the Snort binary to find the current release version.  When you run this command:

    
    snort -V
    
    

    the Snort binary will print a line of version information and exit.  So I use that trick within the Snort package GUI code to grab the current binary version and use that to automatically construct the rules tarball filename.

    Suricata is a different code base and obviously has its own independent versioning system.  So even though it will also print version information to the command line like Snort will, the version number means nothing in terms of Snort VRT rules.  For example, the current Suricata version is 2.0.8.

    So that is why Suricata as a more manual process for Snort VRT rules.  Truth be known, I suspect the Suricata people would really rather folks use the Emerging Threats (ET) rules anyway.  ET provides monetary support to the Suricata project.  So far as I know, the Sourcefire/Cisco/Snort folks do not provide financial support to Suricata.  Snort and Suricata are in a sort of friendly competition you could say… :D.

    Bill



  • Thank you to both of you, I didn't know these tarballs were for different versions; being an economist I am educated to +1 ( ;D ), so I noobly assumed the version number would simply increase +1 with every new update.

    Which brings me to: which snapshot should I use for the latest Suricata? My Suricata is 2.1.5, should I always simply use the latest Snort snapshot for the latest Suricata?

    Truth be known, I suspect the Suricata people would really rather folks use the Emerging Threats (ET) rules anyway.  ET provides monetary support to the Suricata project.  So far as I know, the Sourcefire/Cisco/Snort folks do not provide financial support to Suricata.  Snort and Suricata are in a sort of friendly competition you could say… .

    Bill

    JFL in the past also recommended to use Suricata rules. What I'm doing is currently simply testing Suricata on PPPoE (WAN1) and running Snort on cable/DHCP (WAN2). The reason for that being Snort has more rules than Suricata has (a remark from you in the past, Bill, about Suricata not parsing +/- 700 Snort rules).

    I once had a (rather boring) email conversation with the ET-people about their pricing (which is ridiculous for home users and SMB's).

    I tried to explain in simple terms why we invented a so called demand curve in economics. They seemed not to have had introduction to economics 101 nor appeared interested in that. I do recall my last email being something like this: "let me attempt in a different way to explain what I mean: would you rather sell to 10 customers at 1500 / year, or to 100.000 customers at 1 / year, given your marginal costs in this digital line of products is next to zero?".

    I never got an answer back.

    Yes, the world hates us because "we caused the financial crisis" (no, we did not, we warned for this to happen ever since 1972; crooked banksters and corrupted politicians (by definition not real economists) caused this mess), but we do have some useful tools in our tool bag. There's a reason so many startups fail in their first 5 years.

    ( ;D ;D ;D )

    Bye  :P



  • Snort free rules are not downloading since June 16th. Anyone else?

    
    	Downloading Snort VRT rules md5 file snortrules-snapshot-2970.tar.gz.md5...
    	Snort VRT rules md5 download failed.
    	Server returned error code 422.
    	Server error message was: 
    	Snort VRT rules will not be updated.
    
    

  • Banned

    @G.D.:

    Snort free rules are not downloading since June 16th.

    I just downloaded them 60 minutes ago.  MD5: 55718e94de95408ec54566dcb993c67c. You are downloading nonexistent snapshot.



  • @doktornotor:

    @G.D.:

    Snort free rules are not downloading since June 16th.

    I just downloaded them 60 minutes ago.  MD5: 55718e94de95408ec54566dcb993c67c. You are downloading nonexistent snapshot.

    Thanks. What do I need to tweak to fix this?
    pfSense 2.1.5-RELEASE (amd64)
    Snort 2.9.7.0 pkg v3.2.3


  • Banned

    @G.D.:

    Thanks. What do I need to tweak to fix this?
    pfSense 2.1.5-RELEASE (amd64)
    Snort 2.9.7.0 pkg v3.2.3

    The current package version is 3.2.5 on 2.2.x and 2.9.7.2 pkg v3.2.4 on 2.1.x



  • Upgraded to 2.9.7.2 and it seems to have fixed the issue.

    
    Starting rules update...  Time: 2015-07-01 12:42:38
    	Downloading Snort VRT rules md5 file snortrules-snapshot-2972.tar.gz.md5...
    	Checking Snort VRT rules md5 file...
    	There is a new set of Snort VRT rules posted.
    	Downloading file 'snortrules-snapshot-2972.tar.gz'...
    	Done downloading rules file.
    	Downloading Emerging Threats Open rules md5 file emerging.rules.tar.gz.md5...
    	Checking Emerging Threats Open rules md5 file...
    	There is a new set of Emerging Threats Open rules posted.
    	Downloading file 'emerging.rules.tar.gz'...
    	Done downloading rules file.
    	Extracting and installing Snort VRT rules...
    	Using Snort VRT precompiled SO rules for FreeBSD-8-1 ...
    	Installation of Snort VRT rules completed.
    	Extracting and installing Emerging Threats Open rules...
    	Installation of Emerging Threats Open rules completed.
    	Copying new config and map files...
    	Updating rules configuration for: WAN ...
    	Updating rules configuration for: LAN ...
    The Rules update has finished.  Time: 2015-07-01 12:46:15
    
    

    So, what happened, they retired the 2.9.7.0 version? I hope 2.9.7.2 stays working, as this seems to be the last version for pfSense 2.1.5…

    Thanks!



  • @G.D.:

    Upgraded to 2.9.7.2 and it seems to have fixed the issue.

    So, what happened, they retired the 2.9.7.0 version? I hope 2.9.7.2 stays working, as this seems to be the last version for pfSense 2.1.5…

    Thanks!

    Yes, the Snort Team has a life cycle program for each version of Snort, and the Snort rules packages are tied to specific versions of the Snort binary.  So 2.9.7.0 has gone EOL along with its rules tarball.  The current Snort version is 2.9.7.3.

    Due to other life cycle issues with FreeBSD 8.3 (which is the code base for pfSense 2.1 and earlier), new packages no longer compile properly for pfSense 2.1.x.  So that's why Snort is frozen at 2.9.7.2 on pfSense 2.1.  You need to bite the bullet and upgrade to pfSense 2.2.x, otherwise Snort will eventually stop working on 2.1.x pfSense (because you won't be able to get new rules updates).

    Bill



  • How do you do a manual upgrade of the snort package?  I running pfs 2.1.5 and can't afford to upgrade beyond 2.1.5 because anything beyond 2.1.5 break squid proxy with traffic shapping limiter.

    Please advise and thank you in advance.



  • Yes, the Snort VRT will periodically deprecate older rules packages.  Each version of Snort (and the associated rules tarball) have a life cycle of support.  At EOL (End of Life), they quit posting rules updates for the older versions of Snort.

    You will need to move up to pfSense 2.2.x to keep using the Snort package.  I expect them to drop 2.9.7.2 rules support in the not too distant future.  You can visit the Snort web site and they post the EOL dates for each version someplace there.  Might have to search a bit to find it as it's not always easy to locate.

    Bill



  • Still running  2.1.5-RELEASE (i386)
    On Sep 09 I upgraded to Snort 2.9.7.2 pkg v2.9.7.2 pkg v3.2.5, VRT Rules never downloaded

    Sep 13 04:17:01 	php: snort_check_for_rule_updates.php: [Snort] Server returned error code 422...
    Sep 13 04:17:01 	php: snort_check_for_rule_updates.php: [Snort] Snort VRT rules md5 download failed...
    


  • @RonpfS:

    Still running  2.1.5-RELEASE (i386)
    On Sep 09 I upgraded to Snort 2.9.7.2 pkg v2.9.7.2 pkg v3.2.5, VRT Rules never downloaded

    Sep 13 04:17:01 	php: snort_check_for_rule_updates.php: [Snort] Server returned error code 422...
    Sep 13 04:17:01 	php: snort_check_for_rule_updates.php: [Snort] Snort VRT rules md5 download failed...
    

    You must upgrade both pfSense and then the Snort package.  The Snort VRT has discontinued support of the older rules.  Each version of Snort has a life cycle, and at the end of the life cycle for a particular version they stop providing rules packages for that version.

    Bill


  • Banned

    Created a PR to get this removed from the 2.1.x packages feed, since the package is useless now.

    https://github.com/pfsense/pfsense-packages/pull/1065