Snort fatal error on start

[T5000]

The only solution for me is to uncheck Settings will not be removed during package deinstallation and reinstall the package. The error is gone but i have to reconfigure everything …that sucks, but snort is working again.

[bmeeks]

@T5000:

The only solution for me is to uncheck Settings will not be removed during package deinstallation and reinstall the package. The error is gone but i have to reconfigure everything …that sucks, but snort is working again.

That indicates something got corrupted in the Snort section of the config.xml file.  After you get your setup put back in place, pull off a backup of your firewall configuration and save it using the options under Diagnostics…Backup/Restore.  Do the same each time you make any major changes to the firewall config (and when you make any changes to the Snort setup).  This way, if you get a corrupted configuration, you can easily restore a "current" backup.

Bill

[T5000]

@bmeeks:

@T5000:

The only solution for me is to uncheck Settings will not be removed during package deinstallation and reinstall the package. The error is gone but i have to reconfigure everything …that sucks, but snort is working again.

That indicates something got corrupted in the Snort section of the config.xml file.  After you get your setup put back in place, pull off a backup of your firewall configuration and save it using the options under Diagnostics…Backup/Restore.  Do the same each time you make any major changes to the firewall config (and when you make any changes to the Snort setup).  This way, if you get a corrupted configuration, you can easily restore a "current" backup.

Bill

Yeah it seems so. I noticed that there are less Categories now in Snort 3.1.2. For example ET Games, FTP etc. are now missing in 3.1.2 ?!. Maybe that corrupted the backup because they were there before the update.

[bmeeks]

@T5000:

@bmeeks:

@T5000:

The only solution for me is to uncheck Settings will not be removed during package deinstallation and reinstall the package. The error is gone but i have to reconfigure everything …that sucks, but snort is working again.

That indicates something got corrupted in the Snort section of the config.xml file.  After you get your setup put back in place, pull off a backup of your firewall configuration and save it using the options under Diagnostics…Backup/Restore.  Do the same each time you make any major changes to the firewall config (and when you make any changes to the Snort setup).  This way, if you get a corrupted configuration, you can easily restore a "current" backup.

Bill

Yeah it seems so. I noticed that there are less Categories now in Snort 3.1.2. For example ET Games, FTP etc. are now missing in 3.1.2 ?!. Maybe that corrupted the backup because they were there before the update.

I would not think so.  The Snort VRT sometimes deprecates file categories and removes them from the latest rules tarballs. That might be what happened to the rules you mentioned.  The VRT has been on a mission this year to clean up some older rules, and if you looked in recent months you would have seen that several of the category files were actually empty (contained no defined rules).

Bill

[daplumber]

FIXED!

It looks like the unicode.map file was bad, and not just the CR LF issue, it was missing many entries compared to the one posted here. (Just removing the CR LF problem didn't fix it) I copied the one from here (thanks Supermule!) and I also had to change the snort.conf iis unicode entry to 20127 from 1252 which didn't make sense as I'm in the US anyway.

I have no clue as to the root cause, but a package re-install didn't fix it, along with a restore from pre-2.1.5-RELEASE backup.

[bmeeks]

@daplumber:

FIXED!

It looks like the unicode.map file was bad, and not just the CR LF issue, it was missing many entries compared to the one posted here. (Just removing the CR LF problem didn't fix it) I copied the one from here (thanks Supermule!) and I also had to change the snort.conf iix entry to 20127 from 1252 which didn't make sense as I'm in the US anyway.

I have no clue as to the root cause, but a package re-install didn't fix it, along with a restore from pre-2.1.5-RELEASE backup.

Glad you're fixed, but remember the entries in snort.conf for each interface are completely overwritten with each SAVE command or when stopping/starting the interfaces from the GUI.  If you want to make more permanent changes, you have to edit (carefully !!!) the template file located here:

/usr/local/pkg/snort/snort_conf_template.inc

Don't touch anything in this file inside braces {}.  Let me repeat – DO NOT TOUCH ANYTHING inside braces {}.  If you do, the install can be really badly broken.

Other lines in there can be carefully edited to produce permanent changes.  That's why I broke out the way the conf file is generated from being pure PHP code to this hybrid based on a template with placeholders for critical values passed via strings from the PHP code.  That's what the things in braces "{}" are:  those critical placeholders for configuration parameters coming from the GUI.

Bill

[daplumber]

Turns out /usr/pbi/snort-amd64/etc/snort/snort_61603_re1/snort.conf had already reverted to:

# HTTP Inspect #
preprocessor http_inspect: global \
        iis_unicode_map unicode.map 1252 \

I couldn't find that section in /usr/local/pkg/snort/snort_conf_template.inc so I'm not sure where it's being pulled in from.

I still don't understand why it's using 1252 and apparently the unicode file replacement was the issue rather than the code page # used?

Is a puzzlement.  ???

[BBcan177]

@daplumber:

Turns out /usr/pbi/snort-amd64/etc/snort/snort_61603_re1/snort.conf had already reverted to:

# HTTP Inspect #
preprocessor http_inspect: global \
        iis_unicode_map unicode.map 1252 \

I couldn't find that section in /usr/local/pkg/snort/snort_conf_template.inc so I'm not sure where it's being pulled in from.

I still don't understand why it's using 1252 and apparently the unicode file replacement was the issue rather than the code page # used?

Is a puzzlement.  ???

grep -n "iis_unicode_map" /usr/local/pkg/snort/*

snort_generate_conf.php:1229:$http_inspect_global .= "\\n\tiis_unicode_map unicode.map 1252 \\n";

Its on line 1229 of that file

[daplumber]

Ah, it's obvious after you explain it, thanks!  8)

In any case it looks like the the sole issue was the corrupted unicode.map file. I have no idea how that could have happened. I presume someone's checked the one in the package?

[bmeeks]

@daplumber:

Ah, it's obvious after you explain it, thanks!  8)

In any case it looks like the the sole issue was the corrupted unicode.map file. I have no idea how that could have happened. I presume someone's checked the one in the package?

My mistake on where the edit was.  I was working from obviously faulty memory instead of actually looking it up.  I forgot that the HTTP preprocessor configuration is built into a single large string variable that is then written to the conf file.  Thanks BBcan177 for setting the OP on the right path.

I've done a number of fresh installs and upgrades on 2.1.x and 2.2-ALPHA pfSense virtual machines (along with my own production firewall) and never encountered this problem.

Bill

[daplumber]

The hardware is a Netgate APU2 http://store.netgate.com/NetgateAPU2.aspx, the same as the VK-T40E in the pfSense store with 2GB RAM and an 8GB SD card. I know that's considered a little light for snort, but I've never seen any issues before. Could a resource exhaustion issue messed with install without leaving any log entries I've spotted?

[bmeeks]

@daplumber:

The hardware is a Netgate APU2 http://store.netgate.com/NetgateAPU2.aspx, the same as the VK-T40E in the pfSense store with 2GB RAM and an 8GB SD card. I know that's considered a little light for snort, but I've never seen any issues before. Could a resource exhaustion issue messed with install without leaving any log entries I've spotted?

No, I doubt that low memory would corrupt the file.  If you have a Netgate support contract, contact them and tell them about this issue with the corrupted file.  There was a similar problem a few weeks back caused by some kind of failure between two servers at ESF/Netgate.  It might be messed up again, or it could have just been a temporary glitch.

Bill

[stenio]

@bmeeks:

Check for the file here:

/usr/pbi/snort-amd64/etc/snort

It should be there.  If you see it, remount your filesystem in read/write mode and copy the unicode.map file to the subdirectory snort_61603_re1

I have not had a chance yet to upgrade any of my test VMs to 2.1.5, so I have not tried a Snort update with the latest pfSense security fix.

Bill

Hi,

I have the file but it is empty. What can I do?

[2.1.5-RELEASE][admin@firewall]/root(1): find / -type f -name unicode.map | xargs ls -l
-rw-r–r--  1 root  wheel  0 Sep 12 00:09 /usr/pbi/snort-i386/etc/snort/snort_23326_rl0/unicode.map
-rw-r--r--  1 root  wheel  0 Sep 12 00:08 /usr/pbi/snort-i386/etc/snort/snort_43270_rl1/unicode.map
-r--r--r--  1 root  wheel  0 Sep 12 00:07 /usr/pbi/snort-i386/etc/snort/unicode.map

Thanks,
Stenio

[bmeeks]

@stenio:

@bmeeks:

Check for the file here:

/usr/pbi/snort-amd64/etc/snort

It should be there.  If you see it, remount your filesystem in read/write mode and copy the unicode.map file to the subdirectory snort_61603_re1

I have not had a chance yet to upgrade any of my test VMs to 2.1.5, so I have not tried a Snort update with the latest pfSense security fix.

Bill

Hi,

I have the file but it is empty. What can I do?

[2.1.5-RELEASE][admin@firewall]/root(1): find / -type f -name unicode.map | xargs ls -l
-rw-r–r--  1 root  wheel  0 Sep 12 00:09 /usr/pbi/snort-i386/etc/snort/snort_23326_rl0/unicode.map
-rw-r--r--  1 root  wheel  0 Sep 12 00:08 /usr/pbi/snort-i386/etc/snort/snort_43270_rl1/unicode.map
-r--r--r--  1 root  wheel  0 Sep 12 00:07 /usr/pbi/snort-i386/etc/snort/unicode.map

Thanks,
Stenio

Go to this post my Supermule and copy and paste the contents of his unicode.map file that he generously posted into yours.

https://forum.pfsense.org/index.php?topic=81067.msg442689#msg442689

If removing and reinstalling the package is not bringing you an updated file (one that is not zero-length), then the above copy-paste is the next best option.

Bill

[stenio]

@bmeeks:

Go to this post my Supermule and copy and paste the contents of his unicode.map file that he generously posted into yours.

https://forum.pfsense.org/index.php?topic=81067.msg442689#msg442689

If removing and reinstalling the package is not bringing you an updated file (one that is not zero-length), then the above copy-paste is the next best option.

Bill

Thank you Bill, it works.

[bmeeks]

@stenio:

@bmeeks:

Go to this post my Supermule and copy and paste the contents of his unicode.map file that he generously posted into yours.

https://forum.pfsense.org/index.php?topic=81067.msg442689#msg442689

If removing and reinstalling the package is not bringing you an updated file (one that is not zero-length), then the above copy-paste is the next best option.

Bill

Thank you Bill, it works.

Glad it worked for you.  BTW, I meant to type "by" instead of "my" in the reply up above… :-[

[T5000]

It happens every time now the rules get updated. After every update the unicode.map file is blank again. Found no solution so snort is nearly useless now.

[daplumber]

I'm now having the same problem. For grins and giggles trying:

Backup
De-Install snort
Backup
reinstall pfsense

And then we'll see if a reinstall of snort fixes the issue.

[bmeeks]

@T5000:

It happens every time now the rules get updated. After every update the unicode.map file is blank again. Found no solution so snort is nearly useless now.

I have not noticed the problem on my box, but I use both the Snort VRT rules and the Emerging Threats Open rules.

You want to make sure a good master copy of that file resides in the main Snort directory here:

/usr/pbi/snort-{arch}/etc/snort

where {arch} is either i386 or amd-64.

Also, just to help troubleshoot, enable the ET-OPEN rules on the GLOBAL SETTINGS tab and do a manual update (click the UPDATE button on the UPDATES tab).  You don't have to actually enable any of the ET-OPEN rules, but I just want to see if they will provide a good copy of the missing unicode.map file.

Bill

[stenio]

Hi,

Suddenly every time I start snort I get the following error:  :'(

FATAL ERROR: Failed to load /usr/pbi/snort-i386/lib/snort/dynamicrules/protocol-dns.so: /usr/pbi/snort-i386/lib/snort/dynamicrules/protocol-dns.so: invalid file format

I think that the problem is related to the upgrade from 2.1.4 to 2.1.5.

Do you have any ideas?

Thanks,
Stenio