ET Open Ruleset not downloading
-
The Emerging Threats Open ruleset stopped working for me. The update simply fails. Is anyone else having this problem? Is there a way to update the rules manually? (I was able to download the file directly from the website). How about a way to check/update the url that is being utilized by Snort?
Thanks!
-
@talaverde said in ET Open Ruleset not downloading:
The Emerging Threats Open ruleset stopped working for me. The update simply fails. Is anyone else having this problem? Is there a way to update the rules manually? (I was able to download the file directly from the website). How about a way to check/update the url that is being utilized by Snort?
Thanks!
It's working fine on my firewall. Had an actual update yesterday (April 10th) and a successful check (but no new update posted) today (April 11). The download of the MD5 checksum file was successful.
This is the base URL currently used by Snort for the ET-Open rules: http://rules.emergingthreats.net/. I probably need to update that to use the https version, but for now the http URL is redirecting successfully on my end. I would expect that if this was a widespread issue there would be tons of posts here about it. Have you installed or activated some other type of security package on your firewall such as squid, squidguard or pfBlockerNG?
-
Thanks for the reply. I had actually just rebuilt pfSense from scratch. ET Open was failing on my original firewall and the (clean) new one.
I just re-checked it and it's working now. A big never mind. I should have waited another 24 hours before getting too concerned.
-
@talaverde said in ET Open Ruleset not downloading:
Thanks for the reply. I had actually just rebuilt pfSense from scratch. ET Open was failing on my original firewall and the (clean) new one.
I just re-checked it and it's working now. A big never mind. I should have waited another 24 hours before getting too concerned.
What did the update log say about the failure? You can see it on the UPDATES tab. My guess is something was amiss with DNS name resolution on the firewall itself. The Snort process needs DNS to be working in order to "find" the rules URLS for updates. Another possibility is that the external connectivity was not actually working.
-
Here is the log from when it was failing. I was getting that 302 error on both the new pfSense server and the retired pfSense server, but just on ET Open, not Snort rules. As mentioned, 24 hours later, the issue went away...
Starting rules update... Time: 2019-04-12 00:23:46
Downloading Snort Subscriber rules md5 file snortrules-snapshot-29120.tar.gz.md5...
Checking Snort Subscriber rules md5 file...
There is a new set of Snort Subscriber rules posted.
Downloading file 'snortrules-snapshot-29120.tar.gz'...
Done downloading rules file.
Downloading Emerging Threats Open rules md5 file emerging.rules.tar.gz.md5...
Emerging Threats Open rules md5 download failed.
Server returned error code 302.
Server error message was: 302 Found
Emerging Threats Open rules will not be updated.
Extracting and installing Snort Subscriber Ruleset...
Using Snort Subscriber precompiled SO rules for FreeBSD-11 ...
Installation of Snort Subscriber rules completed.
Copying new config and map files...
Warning: No interfaces configured for Snort were found...
The Rules update has finished. Time: 2019-04-12 00:25:35Starting rules update... Time: 2019-04-12 00:32:57
Downloading Snort Subscriber rules md5 file snortrules-snapshot-29120.tar.gz.md5...
Checking Snort Subscriber rules md5 file...
There is a new set of Snort Subscriber rules posted.
Downloading file 'snortrules-snapshot-29120.tar.gz'...
Done downloading rules file.
Downloading Emerging Threats Open rules md5 file emerging.rules.tar.gz.md5...
Emerging Threats Open rules md5 download failed.
Server returned error code 302.
Server error message was: 302 Found
Emerging Threats Open rules will not be updated.
Extracting and installing Snort Subscriber Ruleset...
Using Snort Subscriber precompiled SO rules for FreeBSD-11 ...
Installation of Snort Subscriber rules completed.
Copying new config and map files...
Updating rules configuration for: LAN ...
The Rules update has finished. Time: 2019-04-12 00:34:44 -
HTTP error code 302 means "resource temporarily moved". That is a server-side problem and not a client-side problem. So it was not something within Snort itself, and likely not something on your firewall either unless you are running Squid or some similar caching server.
-
Thanks for the info. No caching. I looked into the URLs and noticed a change, or thought I noticed a change. This is why I originally asked if there was a way to update/change the URL in the Snort settings. When it started downloading within 24 hours I realized I was either wrong about the URL or Snort updated the URL itself. If 302 means 'temporarily' moved, then maybe they did just that; move the link temporarily. Next time, I'll give a 302 error a little more time before getting concerned. (If I was paying for ET Pro, I might want to look deeper into it) I appreciate the insight.
-
The Snort package will never update an internal URL until a new version of the Snort package is installed. Those URLs are hard-coded in the PHP files. Only a package update performed manually by the user will update the PHP files.
There are several possibilities for your problem. If you are using a DNS provider other than the built-in Unbound resolver in pfSense, then that DNS provider might have cached a bad (or later changed) IP address. If you have another package installed such as Squid or Squidguard, that can interfere potentially if the actual URL updates but Squid is retaining an older cached value. A package like pfBlocker might interfere if one of the ET-Open IP addresses winds up on some blocklist temporarily.
I've never experienced any hiccup of my ET-Open rule updates and my firewall checks twice each day (once every 12 hours). I have only three packages running on my firewall: (1) Snort, (2) nut (for UPS monitoring) and (3) OpenVPN-Client.
-
pfBlockerNG is a possiblity
-
@talaverde said in ET Open Ruleset not downloading:
pfBlockerNG is a possiblity
A package such as pfBlockerNG is a very useful tool, but it can be misused or misapplied sometimes leading to frustration. It works essentially as a long list of IP addresses to be blocked. Those lists can be configured from many sources. Not all of the sources are "current", and even those that are can frequently contain errors in the form of a legitimate web site IP address or netblock being lumped into a "bad actor" list.
So when you have a security tool such as an IDS/IPS layered with another security tool such as pfBlockerNG, you have to immediately consider any "failing to connect" issues on your network as being caused by one of those packages. So in your case, if I saw failing ET-Open downloads, my first instinct would be to check my pfBlockerNG blocks to see if the address had gotten inadvertently blocked. The rules vendors use various CDNs (content distribution networks) to host their rules file for worldwide download. Sometimes a pfBlockerNG list might get overly aggressive and block one of those CDNs (or a segment of a CDN) because a bad actor IP lives in the same netblock. This has happened to folks in the past with AWS addresses.
In the same vein, if I had connectivity issues on a client with a web site or other service, I would check both the IDS/IPS alerts to see if the address showed up there as well as the pfBlockerNG alerts to see if something there tagged it. I would do that before I considered anything else on the client itself. Neither of these tools (Snort/Suricata nor pfBlockerNG) is a "click it on and forget it" type of package. They require constant baby sitting by a knowledgeable admin.
So in the future, when you have any kind of connectivity issues outside of something obvious like a hardware failure, look first at your IDS/IPS and pfBlockerNG tools as the source of the connectivity issue. Only after eliminating both packages as the cause of the "block" should you look at potential client issues such as software bugs or something.
