Snort Updating problems !!!

SnoSalmon

I had issues initially with the manual update, telling me when I tried to copy the rules after extraction.
The third go I had at it seemed to work, didn't think I did anything different, however the instructions on the first page do work as I have the SNORT ruleset available.
And a restart was definitely required.

Screenies would be cool Rune, I've got mine figured out (the manual update process, not SNORT as it's not working for me yet), but for those who struggle it will be a great help seeing the commands typed out in a screenshot.
(kinda funny as it's all text based so it's a screenshot of text of what you already mentioned)

jimbabwe

Having same problem. Searched and found instructions for manually updating the rules file.
Wrote shell script and set cron job to run script; looks ok so far.
The instructions for manual update (like extracting from the .tar.gz file) came from the forum.

Here's the shell script:

#!/bin/sh
cd /var/tmp
fetch -l http://www.snort.org/pub-bin/oinkmaster.cgi/[youroinkmastercodehere]/snortrules-snapshot-2860.tar.gz
tar -zxvf snortrules-snapshot-2860.tar.gz -C /usr/local/etc/snort/ rules/
/usr/local/etc/rc.d/snort.sh reload

NOTES:

replace the brackets [] and 'youroinkmastercodehere' with your oinkmastercode from snort.org
make sure the snortrules-snapshot-2860.tar.gz matches your pfsense snort package (Services –> Snort) shows version at top of screen
save the shell script somewhere on your server and name it snortrules.sh (like /home/scripts/snortrules.sh)
chmod 755 snortrules.sh to make executable
set a cronjob to run the script (Ex: 12:05 daily 5 0 * * * /home/scripts/snortrules.sh)

This is a quick-and-dirty script; feel free to modify it. I don't know if you need to remove the old rules before the new ones are extracted to the rules directory; right now, I'm just overwriting them. You could have the script copy the rules to a backup directory name with the backup date, in case the new rules break anything. You could also have the cronjob or script dump to a log file to inspect results afterward.

rnowotny

The information below is only tested for Snort 2.8.6 pkg v. 1.27 on Pfsense 1.2.3-RELEASE

The script below is downloading the rules to /usr/local/etc/snort/rules , but the updates also needed to be copied
into the rules subdirectory for each interface.
This is, because snort is keeping a ruleset for each interface snort is running on.
If You look at /usr/local/etc/snort You should find some directories like :
snort_59369_le1 or something like that. Inside that directory, there is another rules directory -
and those rules are actually used for that particular interface.

So - please find the new Shellscript below …

Here's the shell script:

#!/bin/sh
cd /var/tmp

get the update

fetch -l http://www.snort.org/pub-bin/oinkmaster.cgi/[youroinkmastercodehere]/snortrules-snapshot-2860.tar.gz

unpack the update

tar -zxvf snortrules-snapshot-2860.tar.gz -C /usr/local/etc/snort/ rules/

copy the rules to each interface snort is using

cp /usr/local/etc/snort/rules/. /usr/local/etc/snort/[yourinterface1directory]/rules/
#cp /usr/local/etc/snort/rules/. /usr/local/etc/snort/[yourinterface2directory]/rules/
#cp /usr/local/etc/snort/rules/. /usr/local/etc/snort/[yourinterface3directory]/rules/

repeat that for each interface You are using

reload snort to use the new rules

/usr/local/etc/rc.d/snort.sh reload

END

NOTES:

replace the brackets [] and 'youroinkmastercodehere' with your oinkmastercode from snort.org
replace the [yourinterface<n>directory] with the directories for each of Your interfaces using snort.
You may find it like that :
cd /usr/local/etc/snort
ls -l
there should be one directory for each interface snort is used with, the name is something like snort_59369_le1
make sure the snortrules-snapshot-2860.tar.gz matches your pfsense snort package (Services –> Snort) shows version at top of screen
save the shell script somewhere on your server and name it snortrules.sh (like /home/scripts/snortrules.sh)
chmod 755 snortrules.sh to make executable
set a cronjob to run the script (Ex: 12:05 daily 5 0 * * * /home/scripts/snortrules.sh)
You may set the crontab with crontab -e what will invoke vi
for those who are not familiar with the vi editor, You may edit crontab like that :
ee /etc/crontab
You may find some information how to make crontab entries on the web.
please be noted that all Your changes to the old Rule Files (Enabling/Disabling a single SID) are lost !

best regards from Vienna, Austria
Ing. Robert Nowotny
Rotek GmbH</n>

rnowotny

Dear James,
I understand completely - but why dont just release a quick patch -
You may create a shellscript on the fly and use it to download the updates out of Your php scripts.
I guess that would be very easy and dont leave Your less experienced users out in the rain with that update mess.
and after time passes by You can release Your new, shiny version …

yours sincerely
and greetings from Vienna / Austria
Ing. Robert Nowotny
Rotek GmbH

@jamesdean:

I wish it was as easy as pointing to a url.

url = http://www.snort.org/pub-bin/oinkmaster.cgi/<oinkcode here="">/ <filename>The file you get from that url you posted redirects to a https server.

Users on the snort.org mail-lists are having trouble with that redirect.
Suggested fix is to install a perl mod that understands https.
I am trying to avoid using Oinkmaster perl script.

I'm trying to do this in pure php script.

While I am hear might as well rewrite the whole "update tab" to include snort GUI updates to.
I been wanting to do this for a long time, I guess this is a good thing for us.

James</filename></oinkcode>

rnowotny

Dear James,
beeing a newbie in pfsense and snort (but not a newbie in scripting) i am a bit disappointed about the update procedures on snort rules.
I have not seen Your updatescript in action, because it is broken since some weeks now, but I think it simply overwrites the old rules, right ?
So, changes that the user have made (enable / disable some rules by SID) are lost. Right ?

This is not very practical - guess I want to use (and update) the emerging.policy-rules, but i want to use Skype -
in that case I would need to disable the SID´s for Skype after every Update …

I guess it would be easy to check if one particular rule is ON/OFF in the current rulefile, before updating to the new rule file, and transfer that state to the new rulefile. After that You might check the new rulefile for new / deleted / changed rules and put that to the update logfile.

Maybe You can consider a better method in new versions, You may get some ideas from the pulledpork project http://code.google.com/p/pulledpork/ what offers more functionality then the oinkmaster original code.

on the wishlist would be :

the Rules (by IDS) Enabled/Disabled state should be preserved on updates
Automatic rule downloads using Oinkcode (of course)
MD5 verification prior to downloading new rulesets (do not download if nothing has changed … )
Full handling of Shared Object (SO) rules
Generation of so_rule stub files
Update Log File should indicate : New Rules / Deleted Rules / Changed Rules

If I find some time I might write some shellscripts to realize SOME of that functionality and share it.

Yours sincerely
Ing. Robert Nowotny
Rotek GmbH
Vienna / Austria

g4m3c4ck

@rnowotny:

Dear James,

Dear rnowotny,

Being the newbie you are. If you already knew the time James and many others have already spent on the pfsense project as well as the complexity of doing so you would not have worded or made the statements in the way you did. pfSense is a great and very powerful project that you use for FREE. If you want to contribute to the project please do so but in the proper way. Basically, I am asking you to put up or shut up.

Sincerly,

Avid pfSense/Snort user and appeciator

simby

James, do you have any news?

EZ

I'm just a newb to this form but not to pf. I have to say Thanks to everyone for all the work. That snort package Is wild. I dug through some of the scripts. All I could say Is AMAZING man. Of course I had a new chunk of hardware I just put in and snort will not update. Ill post the specs on the bottom. After looking through the update code I felt like my brain was stuck in a blinder. But I wrote an add in box for direct url placement. It failed to pick up the package. I'm sure the issue has to be the url that's not allowing the download. The same conclusion you have already come to.

Pretty much I threw a fit for an hour and had a cocktail and it popped into my head that a year or so ago I was using a copy of gallery2 on one of my servers. It had a piece of code that dealt with and took care of url redirect with a cache page. Tomorrow Ill look for that code and see If I could find It and post if you want.

Also It might be possible to post a second server that allows the update page to go fetch the url from and then download the package. Even a Proxy for the download but It would take resources. It might mess with your untar code but could be a final solution. Good luck James. Everyone Thanks for all the work. Its very appreciated.

Snort's on manual for now.

Supermicro X6DPL-4G2 board.
dule LV xeon 2ghz processors
sda/ hdd
2 gig's ram
quad supermicro gig Net card add on.

rnowotny

Dear g4m3c4ck,
I dont wanted to be rude at all, just giving some input what might make sense.
So if my post seems to be unpolite, I apologize.
You may have noticed that I posted a tiny shellscript some posts before,
to make a manual update of snortrules via cronjob. Not a big deal - but some rookies might use it.
I might come up with a script that preserve the ON/OFF state for each IDS across updates,
because I need it myself ;-)

Yours sincerely
Robert

@g4m3c4ck:

Dear rnowotny,
Being the newbie you are. If you already knew the time James and many others have already spent on the pfsense project as well as the complexity of doing so you would not have worded or made the statements in the way you did. pfSense is a great and very powerful project that you use for FREE. If you want to contribute to the project please do so but in the proper way. Basically, I am asking you to put up or shut up.

Sincerly,

Avid pfSense/Snort user and appeciator

EZ

Alright. I dug through my code folder and found this price. Its using fstockopen opt. to pull a redirected url. Its not what I was looking for but might do the job. Ill test it tonight and see if I can pull down the monster url from snort. If I have any luck Ill post my finished pages for inspection.
Take it EZ.

url_redirect.txt

darklogic

rnowotny

I understand what you are saying on a lot of your post. I agree with most statements on the rules and there are better ways to make sure rules are enabled and disabled after updates. These are all the same issues that others are dealing with. James Dean picked up on the SNORT project when no one else did, he also has contributed endless hours to the programming and online fourms. SNORT is getting better and better. I realize you may mean no harm, but your wording is kind of blunt!!! Different people online are going to take your words differently from others. Statements like posting scripts online so some rookies may use it is not what I would call appropriate commits. The fact that anyone on these forums and using pfsense says to me that hey, no one here is really a rookie.

Take Care,

Matt

djnicofun

hello,

I use Pfsense V1.2.3-RC1 , a have install package : Snort : 2.8.6 pkg v. 1.27

i have find a bug , to update the rules the filename of the rules has change since : 10 junes 2010

the file name is now :
snortrules-snapshot-2860.tar.gz

Example for snort 2.8.6.0:
url = http://www.snort.org/pub-bin/oinkmaster.cgi/XXXXXXXXXXXXXX/snortrules-snapshot-2860.tar.gz

Important Note from SNORT website:
We are changing the way we publish rules. In June 2010 we stopped offering rules in the "snortrules-snapshot-CURRENT" format. Instead, rules are released for specific versions of Snort. You will be responsible for downloading the correct rules release for your version of Snort. The new versioning mechanism will require a four digit version in the file name.

Please James can you update the package ?

Best reagrds

djnicofun

Where is the file : oinkmaster.conf on pfsense vers: 1.2.21 ??

rnowotny

Please check this post, it is a quick and dirty workaround until the new version is avail :

http://forum.pfsense.org/index.php/topic,26382.msg139375.html#msg139375

@djnicofun:

hello,

I use Pfsense V1.2.3-RC1 , a have install package : Snort : 2.8.6 pkg v. 1.27

i have find a bug , to update the rules the filename of the rules has change since : 10 junes 2010

the file name is now :
snortrules-snapshot-2860.tar.gz

Example for snort 2.8.6.0:
url = http://www.snort.org/pub-bin/oinkmaster.cgi/XXXXXXXXXXXXXX/snortrules-snapshot-2860.tar.gz

Important Note from SNORT website:
We are changing the way we publish rules. In June 2010 we stopped offering rules in the "snortrules-snapshot-CURRENT" format. Instead, rules are released for specific versions of Snort. You will be responsible for downloading the correct rules release for your version of Snort. The new versioning mechanism will require a four digit version in the file name.

Please James can you update the package ?

Best reagrds

djnicofun

hi,

Ok thanck you, but i have already read this post, i would like know if an official update of this package will be done or not ?

best regards.

jimp

Well they're tricky guys there at Sourcefire. There were a couple things wrong with the rule downloads:

1. The URL changed.
2. They now redirect you to an Amazon s3 URL to get the actual rules
3. The Amazon url is HTTPS.

So I fixed the URL, changed a redirect option, and I had to disable cURL's SSL validation, but now the rules download for me.

The new package version is up now, give it a try and see if it works.

c0urier

Giving it a try now. Will post back soon!

Update:
Seems to be working just fine - Rules updated and Snort running. Thanks jimp!!!

netmethods

I tried it on all 3 of my pfSense boxes and is working fine. Thanks again Jim!

-Jason

DigitalJer

Looking good here!

Many thanks…

I uninstalled the old package and installed the new one and updated it no problem did a port scan test and all is working (fingers crossed) . ;D