Snort free Registered rules MDS fail
-
Looks like the problem is that Snort has released a new snortrules snap shot and you are trying to download the latest file. @bmeeks will have to create an update to the current Snort and Suricata packages in order to use the latest and greatest snort rules snapshot.
The latest file appears to be snortrules-snapshot-29200.tar.gz. I just check mine and it is still working with no errors but I'm using snortrules-snapshot- 29190.tar.gz.
@bmeeks will check it out and let us know what the issue is. I may be wrong, won't be the first time, but this is what it appears to me to be your issue???
-
@jdeloach said in Snort free Registered rules MDS fail:
Looks like the problem is that Snort has released a new snortrules snap shot and you are trying to download the latest file. @bmeeks will have to create an update to the current Snort and Suricata packages in order to use the latest and greatest snort rules snapshot.
The latest file appears to be snortrules-snapshot-29200.tar.gz. I just check mine and it is still working with no errors but I'm using snortrules-snapshot- 29190.tar.gz.
@bmeeks will check it out and let us know what the issue is. I may be wrong, won't be the first time, but this is what it appears to me to be your issue???
The current Snort version is 2.9.20 for pfSense 2.6.0 CE and pfSense Plus 22.05. What is the version of the Snort package you guys are running? It should be 4.1.6, which uses the 2.9.20 binary. Snort automatically determines the correct version to download because the binary is tethered to the rules version. You cannot use a rules archive version different from the binary version with Snort.
Now Suricata is different. With Suricata, you as the admin explicitly specify what Snort Subscriber rules version you want. With Suricata, the Snort rules version does not matter so long as it is from the 2.9.x branch of Snort. However, the Snort team does periodically remove older rule versions. So, with Suricata when using Snort rules, you must manually adjust the rules filename to track the versions still posted by the Snort team.
-
@bmeeks
Yes, I am running 4.1.6 version. Update this morning failed.
-
It works fine for me in a virtual machine I am testing. Here is the latest log from a forced rules update:
Starting rules update... Time: 2022-09-26 10:43:05 Downloading Snort Subscriber rules md5 file snortrules-snapshot-29200.tar.gz.md5... Checking Snort Subscriber rules md5 file... There is a new set of Snort Subscriber rules posted. Downloading file 'snortrules-snapshot-29200.tar.gz'... Done downloading rules file. Downloading Emerging Threats Open rules md5 file emerging.rules.tar.gz.md5... Checking Emerging Threats Open rules md5 file... There is a new set of Emerging Threats Open rules posted. Downloading file 'emerging.rules.tar.gz'... Done downloading rules file. Downloading Feodo Tracker Botnet C2 IP rules file... Done downloading rules file. Feodo Tracker Botnet C2 IP rules are up to date. Extracting and installing Snort Subscriber Ruleset... Using Snort Subscriber precompiled SO rules for FreeBSD-13 ... Installation of Snort Subscriber rules completed. Extracting and installing Emerging Threats Open rules... Installation of Emerging Threats Open rules completed. Copying new config and map files... Updating rules configuration for: WAN ... Restarting Snort on WAN to activate the new set of rules... Snort has restarted on WAN with your new set of rules. Updating rules configuration for: LAN ... Restarting Snort on LAN to activate the new set of rules... Snort has restarted on LAN with your new set of rules. The Rules update has finished. Time: 2022-09-26 10:43:32
The 422 HTTP Response Code in your log message means there may be something wrong with your Oinkmaster Code. Totally outside of pfSense, try downloading the rules archive directly first using a browser.
I will mention this just in case. Inside pfSense be sure you have properly input your Oinkmaster Code on the GLOBAL SETTINGS page. Be sure there are no leading or trailing spaces in the code, and type in the alpha-numeric code only and DO NOT include anything related to a URL! So for example, if your Oinkmaster Code was 12345abcdef, then those are the only characters you would enter in the blank. Do not put anything else but just the raw code.
-
@bmeeks
This oink code was working fine then it stopped working. I can get new code, but I do not think that will fix problem. Were do I get those rules and how, exactly, I would run them from CLI? -
@andrzejls said in Snort free Registered rules MDS fail:
@bmeeks
This oink code was working fine then it stopped working. I can get new code, but I do not think that will fix problem. Were do I get those rules and how, exactly, I would run them from CLI?Did you have a paid or free subscription? If paid, perhaps your subscription has expired as it only lasts for one year then must be renewed. Maybe the credit card or other payment system failed during the renewal.
Can you sign in and access the rules files here: https://snort.org/downloads#rules?
If that does not work for you in a browser, then your subscription is no longer valid. Having a valid and functional Oinkmaster code is the only way to obtain the Snort Subscriber Rules. It is not feasible to update the rules from the CLI in pfSense. The Snort package is not designed for that. But that is never necessary because if you have a functioning Oinkmaster code the GUI works just fine.
Although not likely, perhaps your IP has been banned or blacklisted by the Snort site.
-
@bmeeks
OK, I got new oink code. and run "update rules" with same result, should I use "force update"?. I have free account. I followed link that you provided and had no problem getting to that site. I can only get "Community" not "Registered" nor "Subscription". -
@andrzejls said in Snort free Registered rules MDS fail:
@bmeeks
OK, I got new oink code. and run "update rules" with same result, should I use "force update"?. I have free account. I followed link that you provided and had no problem getting to that site. I can only get "Community" not "Registered" nor "Subscription".That makes no sense. What do you mean by this statement: I can only get "Community" not "Registered" nor "Subscription"?
There is a sign-in button or icon at the site. Can you sign-in with a valid email address and password at the site? If so, then either the Registered or Subscription Rules will become available. The "Community" rules are available free to anyone including anonymous users. But the other two require a valid registration with the Snort team in order to access and download them. If you cannot sign-in and access either the "Registered" or "Subscription" rules at the web site, then your Oinkmaster code is not going to work. Where did you get that code? If you got somebody's from the Internet, then it is highly likely to have been cancelled.
-
@bmeeks
Yes, you are absolutely right. I did not realized, that I needed, nor seen "sign in" button under "Registered" column. Once I signed in, gotten new oink code and run "update rules" all is working fine.
I thought that just login into Snort.org I would be recognized and I would not need to do additional sign in into Registered rules in order for oink code to work. It was misunderstanding on my part. As I said, I am new to Snort and going through this process I learned a lot. Thank you for your help and understanding my "nub" status in Snort. -
@andrzejls said in Snort free Registered rules MDS fail:
@bmeeks
Yes, you are absolutely right. I did not realized, that I needed, nor seen "sign in" button under "Registered" column. Once I signed in, gotten new oink code and run "update rules" all is working fine.
I thought that just login into Snort.org I would be recognized and I would not need to do additional sign in into Registered rules in order for oink code to work. It was misunderstanding on my part. As I said, I am new to Snort and going through this process I learned a lot. Thank you for your help and understanding my "nub" status in Snort.You do not have to be continually signed-in for the Oink code to work, but perhaps in your case with a new code at least one successful sign-in was required in order to fully activate the new code.
Glad you have it working now.
-
@bmeeks
Once again, thanks for your help! -
@bmeeks
Here we go again. Oct.1st. and Snort fail to update rules. I did not changed/modify anything yet I am back to where I was few days ago. This is getting kind of enjoying and/or stupid. Is this is normal with Snort or I do not understand something?.
-
While I realize it may not be of great help to you, this really looks like a problem specific to your setup. If the rules servers were down or malfunctioning I would expect to see lots of folks posting here about it. There are around 25,000 or so Snort and Suricata users on pfSense around the world according to some stats I was given by the Snort Subscriber rules team two years ago.
I can think of really only two things it could be:
- A problem with your registered user status and/or the oinkcode; or
- An issue with your WAN IP address such as it is maybe getting blocked by the Snort rules server (sort of unlikely, to be honest).
But since it worked for a bit and then stopped, maybe look into whether or nor your public WAN IP is on a list resulting in it being blocked. What frequency do you have the "check for updates" set to? Once a day is plenty. The Snort rules are only updated on Tuesday and Thursday each week if I recall. They are not updated frequently. The Emerging Threats rules do get updated daily. If you check too often, that might result in a restriction against your oinkcode.
In what country are you located? Perhaps that is an issue ???
And I will mention this just to be thorough -- not suggesting this is your case.
Is the Oinkmaster code you have truly yours registered by you to an email address controlled only by you? Is it shared by anyone else? If the same code hits the Snort servers from multiple IP addresses, I could see how they might restrict it since the codes are not intended to be shared.
-
@bmeeks could it just be the files are being corrupted on download, I would think if he was blocked in some way, or his code was blocke/restricted why would it say the bad md5 checksum? Wouldn't it just say failed? How is there anything to get a md5 off of if nothing downloaded?
-
@bmeeks
Thanks for your timely response.
You are right stating that ~25K Snort users do not have issue that I have and, most likely, it is a problem resulting in some setup of my install/config of Snort.
I do not know as to why there would be a problem with my Snort user status. I use my real email address that I use every day, not a fake one. I do not share my email with anyone.
I set up updates to 1 (one) day intervals at 4AM. I am physically located in North Carolina USA and my ISP is Spectrum so my public ip should not and is not blocked. I use NordVPN , occasionally, on 1 (one) laptop connected/hardwired to the pfsense router (static ip). My nordVPN on that laptop is setup to use NORDLYNX technology and my LAN 192.xxx.xxx.xxx/24 is Whitelisted in nordVPN settings. pfSense router is not setup with VPN.
Should I regenerate my ionk code in snort.org? Should I run "Update Rules" or "Force Update"? -
@andrzejls said in Snort free Registered rules MDS fail:
@bmeeks
Thanks for your timely response.
You are right stating that ~25K Snort users do not have issue that I have and, most likely, it is a problem resulting in some setup of my install/config of Snort.
I do not know as to why there would be a problem with my Snort user status. I use my real email address that I use every day, not a fake one. I do not share my email with anyone.
I set up updates to 1 (one) day intervals at 4AM. I am physically located in North Carolina USA and my ISP is Spectrum so my public ip should not and is not blocked. I use NordVPN , occasionally, on 1 (one) laptop connected/hardwired to the pfsense router (static ip). My nordVPN on that laptop is setup to use NORDLYNX technology and my LAN 192.xxx.xxx.xxx/24 is Whitelisted in nordVPN settings. pfSense router is not setup with VPN.
Should I regenerate my ionk code in snort.org? Should I run "Update Rules" or "Force Update"?The VPN might be an issue if traffic happened to go out that route. Just guessing, though.
The difference in Update Rules and Force Update is this:
- Update Rules downloads the MD5 checksum files for each rule archive and compares the content of that file (one line of text representing the md5sum of the much larger gzip archive) to the MD5 checksum file stored locally. The locally stored file is saved from the last time the rules changed. So, if the locally stored MD5 file matches what is posted on the Snort rules website that means the gzip rules archive file has not changed, and there is nothing to actually update.
- Force Update begins by deleting the locally stored MD5 file thus guaranteeing the "new file" test will fail and thus download the full gzip rules archive file.
In your case, the last time you posted your full log, you are getting an HTTP Response Code 422 when attempting to download the Snort files. It downloads the MD5 file first, so that is the first error you see in the log. But it does not matter which option you use - Update Rules or Force Update - it will still fail the same way if your box cannot successfully download the files.
Here is the official definition of HTTP Response Code 422: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/422. Unfortunately, there is no way of knowing precisely what that means in the case of the Snort rules download. Could be an Oinkcode problem, or might be something else the server on their end does not like.
-
@johnpoz said in Snort free Registered rules MDS fail:
@bmeeks could it just be the files are being corrupted on download, I would think if he was blocked in some way, or his code was blocke/restricted why would it say the bad md5 checksum? Wouldn't it just say failed? How is there anything to get a md5 off of if nothing downloaded?
He is getting a 422 HTTP RESPONSE error. That is a somewhat generic error from the Snort rules server. Could be any number of things.
The bad checksum error is happening because the
curl
download is likely generating an empty checksum file that fails the test. The Snort rules server does not send back any specific error messages other than the generic HTTP RESPONSE codes.The most common cause of the bad checksum error is using a RAM disk without enough space to hold the entire file, but the OP says he is not using a RAM disk.
-
@bmeeks
Just to confirm, I am not using RAM disk. -
@bmeeks
I can and I did download "snortrules-snapshot-29200.tar.gz" file from snort.org, so this is not a problem, I think. -
@bmeeks said in Snort free Registered rules MDS fail:
Could be an Oinkcode problem
Just to make sure that I am not making errors or not following Snort proper procedure, what is the proper procedure to obtain "oink" code. If I got more than once "oink" code, should I use the most recent one or it does not matter?.