Snort Subscriber rule in suricata
-
Hi
Have been trying to update my Snort Subscriber in SURICATA and it keeps giving me this error:
Snort rules file download failed. Bad MD5 checksum.
Downloaded Snort rules file MD5: f3d8310cea48c64a9b3b061e74df4b92
Expected Snort rules file MD5: 49f38b32e02f607e4851d8e573c9ee2b
Snort rules file download failed. Snort rules will not be updated.So far I tried the ff:
- Turned of my RAM disk
- Disbale pfblocker
- Just renewed my subscription
- Reissued OINK Code
I have 16GB of memmory and 256GB of SSD
Anything else I could try?
-
The number one cause of the "Bad MD5 checksum error" is running out of disk space when expanding the gzip archive of rules. You say you have disabled the RAM disk. How much free space is available in the
/tmpdirectory/partition? You should have at least 256 MB free and preferably 512 MB or more. The rules update process cleans up behind itself (even after failure), so an "out of disk space" issue will have cleared up by the time you view the rules update log. Look in your pfSense system log to see if any disk space errors were recorded during the update.Another problem that has tripped up users in the past is Squid or SquidGuard causing issues with the download of the gzip archive. I don't know if you have either of those packages installed or not.
The error you see means the file that got downloaded on your system is incomplete or corrupt because its MD5 checksum does not match the value stored on the Snort rules website. The code reads the MD5 posted on the rules download website URL, then calculates the MD5 checksum of the gzip archive it downloaded, and then compares the calculated value for the local file to the official value posted on the rules download URL. If they do not match, the code assumes the local file copy is corrupt and abandons updating those rules.
-
Hi bmeeks!
Thanks for the reply here is my disk size. Reverted back to RAM disk as I do not know how to resize things in SSD. I do not use Squid or Squid Guard
I still got same error messgae
-
It is something wrong on your system specifically. I just ran a rules update in Suricata on my test virtual machine (running pfSense 2.7.0 CE) and it updated fine including the Snort VRT rules. Here is the log:
Starting rules update... Time: 2023-10-25 18:26:06 Downloading Emerging Threats Open rules md5 file... Checking Emerging Threats Open rules md5 file... There is a new set of Emerging Threats Open rules posted. Downloading file 'emerging.rules.tar.gz'... Done downloading rules file. Downloading Snort VRT rules md5 file... Checking Snort VRT rules md5 file... There is a new set of Snort rules posted. Downloading file 'snortrules-snapshot-29200.tar.gz'... Done downloading rules file. Downloading Snort GPLv2 Community Rules md5 file... Checking Snort GPLv2 Community Rules md5 file... There is a new set of Snort GPLv2 Community Rules posted. Downloading file 'community-rules.tar.gz'... Done downloading rules file. Downloading Feodo Tracker Botnet C2 IP rules file... Done downloading rules file. Extracting and installing Feodo Tracker Botnet C2 IP rules... Feodo Tracker Botnet C2 IP rules were updated. Downloading ABUSE.ch SSL Blacklist rules file... Done downloading rules file. Extracting and installing ABUSE.ch SSL Blacklist rules... ABUSE.ch SSL Blacklist rules were updated. Extracting and installing Emerging Threats Open rules... Installation of Emerging Threats Open rules completed. Extracting and installing Snort rules... Installation of Snort rules completed. Extracting and installing Snort GPLv2 Community Rules... Installation of Snort GPLv2 Community Rules completed. Copying new config and map files... Updating rules configuration for: WAN ... Live-Reload of updated rules is enabled... Live-Reload of updated rules requested for WAN. Updating rules configuration for: LAN ... Live-Reload of updated rules is enabled... Live-Reload of updated rules requested for LAN. The Rules update has finished. Time: 2023-10-25 18:26:22As I mentioned, something is causing the file download on your end to pull down a corrupt file. The number one cause of this is insufficient space on the disk at the time the update job is run. Did you check the pfSense system log to see if any messages were logged during the update period that might be relevant?
The Snort VRT rules are stored on Amazon Web Services infrastructure. It would be remotely possible (but really pretty unlikely) that the specific AWS server farm your IP connection is routed to has a corrupted copy of the Snort rules gzip archive.
-
@MagikMark said in Snort Subscriber rule in suricata:
Hi bmeeks!
Thanks for the reply here is my disk size. Reverted back to RAM disk as I do not know how to resize things in SSD. I do not use Squid or Squid Guard
I still got same error messgae
The disk size you see before or after a rules update is not relevant. As I mentioned, the update job cleans all its files when it terminates, so any out-of-disk space issue would disappear (because the job deletes the corrupt files and all others it downloaded and expanded).
You would need to view the disk space during the actual period of time the rules update job is executing. Any disk space issues would be logged to the pfSense system log under STATUS > SYSTEM LOGS.
If your Oinkcode was invalid, you would be seeing a different HTTP error code in the update log because your firewall would be denied connection to the resource completely. An invalid MD5 checksum means the downloaded file was corrupt. Number one cause of that is insufficient disk space during the rules update job. Its remotely possible there is an issue with the AWS server farm you are routed to, but that would be really unusual. And if that's the case, you can't do anything about that except wait for the remote end to be repaired.
-
Thanks a lot bmeeks for the assistance.
Right now I disabled Suricata and installed Snot package. I copied the configuration from Suricata to Snort and surprisingly everything is running smoothly. Rules update are running as expected.
How do I make a clean install of Suricata? I want to delete everything related to it and reconfigure it from scratch
-
@MagikMark said in Snort Subscriber rule in suricata:
Right now I disabled Suricata and installed Snot package.
Eww!! Installing the Snot package is going to make your firewall look kind of gross
. Might want to stock up on tissues.Just having a little fun with your typo -- suspect you mean the Snort package.
-
@MagikMark said in Snort Subscriber rule in suricata:
How do I make a clean install of Suricata? I want to delete everything related to it and reconfigure it from scratch
Reinstall the Suricata package, then go to the GLOBAL SETTINGS tab and uncheck the option to retain settings when deinstalling. Save that change.
Next, remove the package again using the SYSTEM > PACKAGE MANAGER menu. During the package removal all the Suricata configuration information will be wiped from the firewall's
config.xmlfile.The next time you install the package, it will start with an empty configuration.
By the way, what Snort rules filename did you have enabled in Suricata? The download code is pretty much identical in the two packages with the only difference being Snort automatically determines which Snort VRT file version to download by querying the version of the installed Snort binary. Suricata obviously cannot do that, so it depends on the admin specifying the proper filename on the GLOBAL SETTINGS tab. But if you specified an incorrect filename, I would expect some type of HTTP "resource not found" error (like maybe a 404 error), but not a checksum error.
-
I use this URL:
https://www.snort.org/downloads/subscriber/snortrules-snapshot-29200.tar.gz
Then I use this filename:
snortrules-snapshot-29200.tar.gz
Is it correct?
-
Hi Bmeeks!
I resintalled Suricata and disabled custom url and removed "Enctytion: Bypass" in the custom setting. Evrything is now working fine. Dunno which one of the two is the culprit.
I have Ryzen 3200G, 16GB of memmory and 256GB of SSD, what other setting I could tweak for maximum performance, so far I have done the ff:
- Max Pending Packet : 10240 (only 20% memmory usage)
- Detect Engine Profile: High
- Signature Group Header MPM Content: Full
- Run Mode: Workers
- IPS Mode: Inline
I am able to saturate my 700Mbps line without VPN and around 620Mbps with VPN
Are there other settings that will help us maximixe Suricata and pfSense under Systl, Loader conf & Local?
Thanks Again
-
@MagikMark said in Snort Subscriber rule in suricata:
I use this URL:
https://www.snort.org/downloads/subscriber/snortrules-snapshot-29200.tar.gz
Then I use this filename:
snortrules-snapshot-29200.tar.gz
Is it correct?
Yes, that is the correct filename. But for the Snort rules you should not specify the URL. The package has an internal hard-coded URL for those rules since they are included in the default choices. All you need to provide for the Snort VRT rules is your Oinkcode in the proper location.
You did not say earlier that you had specified a custom URL. What did you put in there? That option is only for including rule sets that are not already listed in the GUI. Plus you have to properly handle the MD5 settings in the Custom URL section.
-
@MagikMark said in Snort Subscriber rule in suricata:
Are there other settings that will help us maximixe Suricata and pfSense under Systl, Loader conf & Local?
Depending on the specific NIC you have, there may be some useful tweaks in this Sticky Post located near the top of the IDS/IPS sub-forum: https://forum.netgate.com/topic/138613/configuring-pfsense-netmap-for-suricata-inline-ips-mode-on-em-igb-interfaces.
-
@MagikMark said in Snort Subscriber rule in suricata:
disabled custom url and removed "Enctytion: Bypass" in the custom setting.
It would have been extremely helpful to me if you had shared this setting in your original post. That is most definitely not the way to enable the Snort VRT rules. No wonder you had problems.
I assumed in my suggestions that you were following the standard and accepted method of enabling Snort VRT rules. You simply check the box to enable them and provide your Oinkcode and the Snort VRT filename (without the URL) in the box provided on the GLOBAL SETTINGS tab.
-
@bmeeks Do you recommend those tweaks in the Sticky Post also for igc?
-
@pfsjap said in Snort Subscriber rule in suricata:
@bmeeks Do you recommend those tweaks in the Sticky Post also for igc?
I don't know. I did not create that post - another user contributed the information there. Different NICs of course can have different customizable settings. You will need to research the particular flavor of
igcchip your NIC has to see what might apply. There are families of NIC controller chips that all can use the same generic FreeBSD driver, and each variant in the family might have its own unique settings that another vendor'sigcNIC does not share.
