Snort 2.9.2.3 pkg v. 2.5.0 Issues

dwood · Jul 19, 2012, 3:34 PM

Fesoj..thanks for posting that information. I've been slowly collecting bits and pieces of the github process into my grey matter..your post helps. It would be helpful to me and no doubt others if you could post a "Github and SNORT for Dummies". This would certainly help out with more detailed feedback in the troubleshooting process for Ermal. I would be interested in that script :-)

I believe another issue here is that reinstalling SNORT over an existing version is behaving differently (depending on what got sync'd) than first uninstalling -> deleting leftover bits -> and reinstalling. Over the last year of messing with SNORT the clean install has become my standard install to avoid the issue of troubleshooting leftover files/code/configuration.

Fesoj · Jul 19, 2012, 3:38 PM

Here it is:

#!/bin/sh
DLURL=http://files.pfsense.com/packages/8/All
SNORT=snort-2.9.2.3-i386.pbi.sha256
if [ -e $SNORT ]; then
  mv -f $SNORT $SNORT.bak
fi
echo "Getting $DLURL/$SNORT"
wget "$DLURL/$SNORT"
echo 
echo

if [ -e $SNORT ] && [ -e $SNORT.bak ]; then
  delta=`diff -q $SNORT $SNORT.bak`
  if [ -n "$delta" ]; then
    echo "+------------------+"
    echo "| Update available |"
    echo "+------------------+"
  else
    echo "+---------------------+"
    echo "| No update available |"
    echo "+---------------------+"
  fi
fi

The script does not update anything and it needn't be on the pfSense box. It just reports about an update between subsequent calls. Strictly speaking it is valid only for pfSense 2.1, but the *.tbz files typically get updated about the same time (give or take an hour, or so).

dwood · Jul 19, 2012, 6:02 PM

Forgive my newbishness..but how/where would you execute the script? Assume little/no knowledge of BSD..sadly I'm a windows guy. :-) If I understand correctly, it does not actually look at SNORT file version on the router?

Fesoj · Jul 19, 2012, 7:11 PM

I'm a windows guy.

Could be even worse.

The setup is as follows:
(1) there's your PC running Windows, some flavor of Linux, whatever, and
(2) there's your pfSense based router
and you do your regular work on your PC (and not on the router). In my case it is Windows 7 or precise Ubuntu.

Save the script in some file and give it a name, e.g. check-snort-update.sh. If you are using a Linux machine put it into ~/bin or /usr/local/bin. On Windows you have several options. Either you use cygwin (http://www.cygwin.com/), or you install your favorite Linux distro using VirtualBox (https://www.virtualbox.org/), then do the same things, as you would do on a Linux machine. You might need to issue a chmod a+x check-snort-update.sh if you want to make the script executable (or always call it with sh check-snort-update.sh).

See the attachment for a typical run on my Windows machine (using cygwin).

breusshe · Jul 19, 2012, 7:01 PM

@breusshe:

@mschiek01:

Ermal this is still not fixed.

If snort is running and downloads a new rule update then tries to apply it, the same will occur. I have the latest version update. Of course if I uninstall snort and reinstall it will start again and run fine but it is still not fixed as the next rule update will cause the same error.

Jul 18 12:19:33 snort[31831]: FATAL ERROR: /usr/local/etc/snort/snort_27588_em1/preproc_rules/sensitive-data.rules(1) Unknown ClassType: sdf
Jul 18 12:19:33 snort[31831]: FATAL ERROR: /usr/local/etc/snort/snort_27588_em1/preproc_rules/sensitive-data.rules(1) Unknown ClassType: sdf
Jul 18 12:19:33 snort[31831]: Initializing rule chains…
Jul 18 12:19:33 snort[31831]: Initializing rule chains…
Jul 18 12:19:33 snort[31831]: +++++++++++++++++++++++++++++++++++++++++++++++++++
Jul 18 12:19:33 snort[31831]: +++++++++++++++++++++++++++++++++++++++++++++++++++

I'm still getting this too. I delete the md5 files and redownload the snort rules, snort starts fine after that. But, next time new rules are downloaded, I'll get this error again.

Actually need to correct this post. I had to delete the snort package altogether and reinstall before downloading the rules would allow snort to start. I got the error about the .so file and FreeDataType error (or something like that. It's in a previous post on this thread). I see an update was done yesterday, I'll post back if this problem continues.

Fesoj · Jul 19, 2012, 7:07 PM

If I understand correctly, it does not actually look at SNORT file version on the router?

That's correct. It just checks, whether there was a change in the md5 value of the snort package. I use it the following way: if there was an announcement about an update, that does not really affect my setup, I just call the script, e.g. every morning. If the script says "Update", I'll just do the binary package update.

If there was an update that touches my setup, I'll have a look at the git repository and then I'll update either by hand or from the repository, but that works only for the php scripts. The package update follows once the script says "Update" to avoid useless downloads (and sometimes the installation hassle that follows).

Essentially, you do the package update only after the script says so.

dwood · Jul 19, 2012, 8:59 PM

Perfecto Fesoj. Exactly what I was looking for. One great thing about sorting through this stuff is learning a lot more about the UNIX/Linus/BSD world. Let's face it..anything really important runs in that OS milieu anyway :-)

CS · Jul 20, 2012, 9:35 AM

Hi guys,
some feedback from my side too:

pfsense 2.0.1-RELEASE
Snort 2.9.2.3 pkg v. 2.5.0

I faced the same issues you mentioned before so there is no need to write them again.
Till yesterday the snort was working and I had only the issue with the blocking of whitelisted IPs so I had blocking OFF.
This morning with the latest auto update I got the known "sdf" FATAL ERROR. So I turned auto-update OFF, reinstalled the snort package, downloaded the rules again and now I get FATAL ERROR "freeRuleData" for a ruleset(bad-traffic.so). I disabled this one as well as all SO rules, but I got still this error. I reinstalled the package (having unselected these rulesets) but still the same error…

So these issues are still unresolved for me:
1. Auto-update issues / FATAL ERRORS
2. Blocking of Whitelisted IPs

Cheers,
/CS

Fesoj · Jul 20, 2012, 2:29 PM

In snort.inc the lines 1308 and 1309 contain bogus sed commands, such that the initial installation cannot complete without errors. You cannot patch this by hand, because this code gets called only during the initial installation as part of the function snort_postinstall(). So this part of snort.inc should better be correct.

Some of the recent posts seem to be related to the sed typos, but I don't know whether it is responsible for all of them. Maybe I was lucky because my setup doesn't need the sensitive data preproc.

ermal,
please update the sed commands (e.g. mwexec("/usr/bin/sed -I.bak -f {$g['tmp_path']}/sedcmd …) and trigger an update of the binary package (or I'll come up with a real dirty patch).

marcelloc · Jul 20, 2012, 2:43 PM

@Fesoj:

In snort.inc the lines 1308 and 1309 contain bogus sed commands…

Again a pull request will be much easier to get applied. Teste this sed, find a fix and pull it at https://github.com/bsdperimeter/pfsense-packages.

This way ermal can check the code and apply it.

att,
Marcello Coutinho

dwood · Jul 20, 2012, 2:54 PM

CS I have been able to reinstall SNORT every time doing this on AMD64, 2.0.1:

1. Uninstall snort. I find that it is helpful to untoggle Global Settings -> "save settings" and start with a new config file when troubleshooting.
2. From Diagnostics -> Command Prompt , copy this command in and excecute it: find /* | grep -i snort | xargs rm -rv
3. Now reinstall and update your rules.

Ermal, I have two installs working of the latest 2.5.0, however this morning both failed to start automatically after an update. I was able to start interfaces manually on one installation. This time no error in the logs.

On the 2nd installation, when manually starting snort, the GUI shows it's not started. However when I start from the shell, one interace starts, but the GUI does not show it started.

Fesoj · Jul 20, 2012, 2:59 PM

Marcello,

it looks as if "pull requests" would make it necessary that I need a GitHub account (I have none) and forked the snort project prior to do anything else. Since I've never used GitHub and git never at work, this might take time before I can contribute…

... unless there is a customized tutorial for snort projects somewhere here.

marcelloc · Jul 20, 2012, 3:06 PM

@Fesoj:

it looks as if "pull requests" would make it necessary that I need a GitHub account (I have none) and forked the snort project prior to do anything else.

You can start using only github web site to do this with few steps.

When you have finished fixes and tests, just:

Create you account(it's free :)) at github
browse snort repo to find the file you want to change
click fork and edit this file
apply your patches and follow pull request options

att,
Marcello Coutinho

breusshe · Jul 20, 2012, 3:35 PM

@CS:

Hi guys,
some feedback from my side too:

pfsense 2.0.1-RELEASE
Snort 2.9.2.3 pkg v. 2.5.0

I faced the same issues you mentioned before so there is no need to write them again.
Till yesterday the snort was working and I had only the issue with the blocking of whitelisted IPs so I had blocking OFF.
This morning with the latest auto update I got the known "sdf" FATAL ERROR. So I turned auto-update OFF, reinstalled the snort package, downloaded the rules again and now I get FATAL ERROR "freeRuleData" for a ruleset(bad-traffic.so). I disabled this one as well as all SO rules, but I got still this error. I reinstalled the package (having unselected these rulesets) but still the same error…

So these issues are still unresolved for me:
1. Auto-update issues / FATAL ERRORS
2. Blocking of Whitelisted IPs

Cheers,
/CS

Me too. Snort isn't working again this morning. The above is my experience getting Snort to work again; however, an uninstall and reinstall of Snort stops the freeRuleData error.

Fesoj · Jul 20, 2012, 3:52 PM

breusshe,

currently the initial installation seems to be broken. Remove the package, then install again (no need to clear residual files). You should see several sed related error messages in the system log, but only immediately after the installation. Later you won't see these messages again, but depending on your settings there might be problems.

CS · Jul 20, 2012, 4:03 PM

@breusshe :

I confirm that the solution mentioned by dwood works fine on i386 too:

1. Uninstall snort. I find that it is helpful to untoggle Global Settings -> "save settings" and start with a new config file when troubleshooting.
2. From Diagnostics -> Command Prompt , copy this command in and excecute it: find /* | grep -i snort | xargs rm -rv
3. Now reinstall and update your rules.

Snort is up and running again without any errors.

Fesoj · Jul 20, 2012, 4:24 PM

Marcello,

apply your patches and follow pull request options

that was easy.

Fesoj · Jul 20, 2012, 8:07 PM

The following applies only for a fresh installation of snort. You must remove the package before applying the following recipe. The procedure is also recommended only for courageous fellows.

(1) Install the snort package

(2) Go to Status: System logs: System

(3) If you find one or more of the following error messages:

Jul 20 18:57:33 php: /pkg_mgr_install.php: Beginning package installation for snort.
Jul 20 18:57:33 check_reload_status: Syncing firewall
Jul 20 18:58:22 php: /pkg_mgr_install.php: The command '/usr/bin/sed -I '' -e -f /tmp/sedcmd /usr/local/etc/snort/snort_26134_em0/preproc_rules/preprocessor.rules' returned exit code '1', the output was 'sed: 1: "-f ": invalid command code -'
Jul 20 18:58:22 php: /pkg_mgr_install.php: The command '/usr/bin/sed -I '' -e -f /tmp/sedcmd /usr/local/etc/snort/snort_26134_em0/preproc_rules/decoder.rules' returned exit code '1', the output was 'sed: 1: "-f ": invalid command code -'
Jul 20 18:58:22 php: /pkg_mgr_install.php: The command '/usr/bin/sed -I '' -e -f /tmp/sedcmd /usr/local/etc/snort/snort_26134_em0/preproc_rules/preprocessor.rules' returned exit code '1', the output was 'sed: 1: "-f ": invalid command code -'
Jul 20 18:58:22 php: /pkg_mgr_install.php: The command '/usr/bin/sed -I '' -e -f /tmp/sedcmd /usr/local/etc/snort/snort_26134_em0/preproc_rules/decoder.rules' returned exit code '1', the output was 'sed: 1: "-f ": invalid command code -'
Jul 20 18:58:22 check_reload_status: Syncing firewall
Jul 20 18:58:23 check_reload_status: Reloading filter
Jul 20 18:58:23 check_reload_status: Syncing firewall

proceed with (4), otherwise stop.

(4) In file /usr/local/pkg/snort/snort.inc find the lines at the end with the sed commands


@file_put_contents("{$g['tmp_path']}/sedcmd", $sedcmd);
mwexec("/usr/bin/sed -I '' -e -f {$g['tmp_path']}/sedcmd {$snortcfgdir}/preproc_rules/preprocessor.rules");
mwexec("/usr/bin/sed -I '' -e -f {$g['tmp_path']}/sedcmd {$snortcfgdir}/preproc_rules/decoder.rules");
@unlink("{$g['tmp_path']}/sedcmd");

and update as follows


@file_put_contents("{$g['tmp_path']}/sedcmd", $sedcmd);
mwexec("/usr/bin/sed -I.bak -f {$g['tmp_path']}/sedcmd {$snortcfgdir}/preproc_rules/preprocessor.rules");
mwexec("/usr/bin/sed -I.bak -f {$g['tmp_path']}/sedcmd {$snortcfgdir}/preproc_rules/decoder.rules");
@unlink("{$g['tmp_path']}/sedcmd");

See also: https://github.com/bsdperimeter/pfsense-packages/pull/286/files

(5) Within the webConfigurator go to Diagnostics: Command Prompt and enter the 2 php statements into the PHP execute field:

require_once('/usr/local/pkg/snort/snort.inc');
snort_postinstall();

This operates silently unless there is an error.

(6) Check the result with

find /usr/local/etc/snort -name '*.bak'

You should see at least 2 backup files.

Good luck.

UPDATE: the patch does not solve the
FATAL ERROR: /usr/local/etc/snort/snort_9486_em1/preproc_rules/preprocessor.rules(202) Unknown ClassType: sdf
problem (though that should be easy to fix, rules were commented out a couple of days ago, but I don't know what the real problem was).

Fesoj · Jul 20, 2012, 5:49 PM

Marcello,

do you have a quick explanation how the GUI part of a package gets installed? Obviously, there are no PHP files inside snort-2.9.2.3.tbz. Maybe they come from pkg-config-0.25_1.tbz or somewhere else.

Thanx for your time.

marcelloc · Jul 20, 2012, 6:24 PM

@Fesoj:

Do you have a quick explanation how the GUI part of a package gets installed?

The package install script copy xml, inc and php files and then install freebsd package.

att,
Marcello Coutinho