Snort 2.9.5.6 pkg v3.0.4 Update – Release notes and change log
-
Well, not sure what you might've turned up in your research. I decided that I wanted it working so sacrificed the old database and rebuilt it from scratch using Snorby's update command. Originally I was using ET and Community rules, now just using VRT registered rules. I'll edit / post again if the problem resumes.
-
Well, not sure what you might've turned up in your research. I decided that I wanted it working so sacrificed the old database and rebuilt it from scratch using Snorby's update command. Originally I was using ET and Community rules, now just using VRT registered rules. I'll edit / post again if the problem resumes.
Nothing turned up yet, but admittedly I've had only a few minutes to research this. Been busy with the Suricata BETA package the last few days. I run Barnyard2 on three interfaces on my home firewall, but admittedly the firewall sees low traffic and not a variety of alerts. I run a lot of the ET block lists (ET RBN, ET CINS, etc.) on the WAN side and get a decent number of alerts there. I have not yet seen a Barnyard2 problem. I will try and devote some time to research this a bit more over the next couple of days. In the meantime, if you get any more data, please share it here.
Thanks,
Bill -
Looks like my barnyard2 has started crashing also:
Feb 27 08:31:30 barnyard2[91121]: Barnyard2 exiting Feb 27 08:31:30 barnyard2[91121]: FATAL ERROR: [dbProcessSignatureInformation()]: Failed, stoping processing Feb 27 08:31:30 barnyard2[91121]: [dbProcessSignatureInformation()] Line[1556], call to dbSignatureInformationUpdate failed for : [gid :122] [sid: 6] [upd_rev: 1] [upd class: 3] [upd pri 2] Feb 27 08:31:30 barnyard2[91121]: ERROR database: Returned signature_id [478] is not equal to updated signature_id [700] in [dbSignatureInformationUpdate()]I'm using ET Free and Snort paid rules.
-
Looks like my barnyard2 has started crashing also:
Feb 27 08:31:30 barnyard2[91121]: Barnyard2 exiting Feb 27 08:31:30 barnyard2[91121]: FATAL ERROR: [dbProcessSignatureInformation()]: Failed, stoping processing Feb 27 08:31:30 barnyard2[91121]: [dbProcessSignatureInformation()] Line[1556], call to dbSignatureInformationUpdate failed for : [gid :122] [sid: 6] [upd_rev: 1] [upd class: 3] [upd pri 2] Feb 27 08:31:30 barnyard2[91121]: ERROR database: Returned signature_id [478] is not equal to updated signature_id [700] in [dbSignatureInformationUpdate()]I'm using ET Free and Snort paid rules.
Thanks for the report. I will dig into this tomorrow (Friday). I suspect it has something to do with the new preprocessor and decoder rules getting included in the sid-msg.map. That's the most likely cause since that is all that really changed other than bumping Barnyard up to 2.13 from 2.12. I'm not seeing it yet in my installation, but you are at least the second person reporting the same issue on the forum.
What backend system are you writing to with Barnyard? Is it Snorby or something else? I use Snorby on Ubuntu.
Bill
-
I have Snorby running on Debian.
-
-
Well, not sure what you might've turned up in your research. I decided that I wanted it working so sacrificed the old database and rebuilt it from scratch using Snorby's update command. Originally I was using ET and Community rules, now just using VRT registered rules. I'll edit / post again if the problem resumes.
I found information on this in a Google Groups discussion with one of the Barnyard2 folks. The root cause seems to be not using the new optional sid-msg.map v2 format. Use of that format is supposed to be "optional", but apparently it can sometimes lead to issues if the old v1 format is used. Here is the link to the Google Groups thread:
https://groups.google.com/forum/#!topic/barnyard2-users/IIoyClc7XTc
Here is another good post with the same information, but maybe easier to read.
http://sourceforge.net/p/snort/mailman/message/31925851/
There is a solution posted in each link for a workaround. Unfortunately it involves a fair amount of SQL. There is a script posted in each thread, though. In the next version of the Snort package, I will migrate the sid-msg.map file to the new v2 format and hopefully that will prevent this in the future.
Sorry you had the issue,
Bill
-
Hi,
I was able to get barnyard2 running again with the sql queries provided in the 2 links. :)
-
Hi,
I was able to get barnyard2 running again with the sql queries provided in the 2 links. :)
Great! As I said up above, in the next Snort update I will migrate to the new v2 format of the sid-msg.map file. This is a "look up" file Barnyard uses to obtain supporting descriptive information about the alert signatures. Beginning with this version of Barnyard2, a new format with some added columns of information is supported.
Bill
-
Bill,
Since I have upgraded to this snort version, my firewall has experienced two kernel panics (pulled from log) and rebooted itself. Prior to going from previous version number to this one ( I do keep updated as much as I can) the firewall has never once panicked on me or rebooted. Nothing else has been touched. No other packages updated or core components updated. I am on the most recent build of pfsense. Would this package cause this for some strange reason or are the two not related and its just more of of a coincidence?
The first time was the next day after i updated earlier in the week. Then it did it again at 3:45 am this morning..
I can list out full packages and hardware if you need it.. I am just scratching my head at this one..Also when i idid the upgrade, i did a full remove, kept settings though and then installed most recent package and the settings were restored.. I am going to do a full remove again and re-install …
I dont see anything else in the log that i can tell that would tell me what caused this. I did upload both crash dumps to the pfsense. I deleted the first one but kept the second one on my hard drive. Not sure if it would be any help if it is related to the updated snort...
Also upon the reboot i get this error / alert. Never seen it before.
There were error(s) loading the rules: pfctl: DIOCADDRULE: Device busy - The line in question reads [0]:
Just thought I would ask and see if it could be this package now or something un-related..
-
Bill,
Since I have upgraded to this snort version, my firewall has experienced two kernel panics (pulled from log) and rebooted itself. Prior to going from previous version number to this one ( I do keep updated as much as I can) the firewall has never once panicked on me or rebooted. Nothing else has been touched. No other packages updated or core components updated. I am on the most recent build of pfsense. Would this package cause this for some strange reason or are the two not related and its just more of of a coincidence?
The first time was the next day after i updated earlier in the week. Then it did it again at 3:45 am this morning..
I can list out full packages and hardware if you need it.. I am just scratching my head at this one..Also when i idid the upgrade, i did a full remove, kept settings though and then installed most recent package and the settings were restored.. I am going to do a full remove again and re-install …
I dont see anything else in the log that i can tell that would tell me what caused this. I did upload both crash dumps to the pfsense. I deleted the first one but kept the second one on my hard drive. Not sure if it would be any help if it is related to the updated snort...
Also upon the reboot i get this error / alert. Never seen it before.
There were error(s) loading the rules: pfctl: DIOCADDRULE: Device busy - The line in question reads [0]:
Just thought I would ask and see if it could be this package now or something un-related..
Is this error written out to the console, or did you pull it from the system log (or both)?
I think the error message you posted would be happening before Snort is even loaded on a reboot. I'm not saying Snort couldn't be at fault, but I don't think it is in this case. There have been no other user reports of this particular problem here on the Forum. If someone else reading this thread has experienced a similar issue, please speak up.
Are you running something in addition such as pfBlocker or URL aliases that might be downloading new "rules" for insertion into the pf firewall? Could be that a corrupt "rule" has been downloaded by one of those (if you are using them). If you are game, you can do these two things in this order, one at the time, for troubleshooting purposes:
1. Disable "block offenders" for Snort. That removes all Snort writes to the pf firewall tables.
2. Disable Snort entirely by unchecking the "enable" checkboxes for each interface (or just stop it manually).
If you still get a kernel panic (or that error message), then we know Snort is not the cause. If you do not get any further panics, then that would indicate Snort might be involved. In that case, post back here.
Bill
-
Thanks bill. I do use pfblocker.
Here are the other packages I have just for reference. I did a clean removal of snort and then installed it. If I get another Panic then I will do as you mention to see if I can isolate it. I am on the 386 version too… Havent migrated over to 64 bit yet.. maybe I should lol.. When I removed snort I did reboot to make sure nothing was running and still got that pfctl: DIOCADDRULE: Device busy so I know that part is not snort.. Just got to isolate whats caused the panics.. Just strange this system has been up with no issues for approx 2 years and before that first panic 183 days straight... I will keep you posted on what i find out, if anything. Thanks for your time!The panic error message was from the sys log. I do not have a monitor hooked up atm for console readout.. I am sure it would have been gone from the reboot anyways.
AutoConfigBackup 1.21
Backup 0.1.5
Dashboard Widget: Snort 0.3.7
File Manager 0.1.3
LCDproc-dev lcdproc-0.5.6 pkg v. 0.9.7
mailreport
nmap nmap-6.25_1 pkg v1.2
nut 2.6.4 pkg 2.0
OpenVPN Client Export 1.2.4
pfBlocker 1.0.2
phpSysInfo 2.5.4
RRD Summary 1.1
Service Watchdog 1.5
System Patches 1.0 -
Thanks bill. I do use pfblocker.
Here are the other packages I have just for reference. I did a clean removal of snort and then installed it. If I get another Panic then I will do as you mention to see if I can isolate it. I am on the 386 version too… Havent migrated over to 64 bit yet.. maybe I should lol.. When I removed snort I did reboot to make sure nothing was running and still got that pfctl: DIOCADDRULE: Device busy so I know that part is not snort.. Just got to isolate whats caused the panics.. Just strange this system has been up with no issues for approx 2 years and before that first panic 183 days straight... I will keep you posted on what i find out, if anything. Thanks for your time!The panic error message was from the sys log. I do not have a monitor hooked up atm for console readout.. I am sure it would have been gone from the reboot anyways.
AutoConfigBackup 1.21
Backup 0.1.5
Dashboard Widget: Snort 0.3.7
File Manager 0.1.3
LCDproc-dev lcdproc-0.5.6 pkg v. 0.9.7
mailreport
nmap nmap-6.25_1 pkg v1.2
nut 2.6.4 pkg 2.0
OpenVPN Client Export 1.2.4
pfBlocker 1.0.2
phpSysInfo 2.5.4
RRD Summary 1.1
Service Watchdog 1.5
System Patches 1.0pfctl is the FreeBSD binary that allows manipulation of the firewall tables directly. It can be used to insert rules, remove rules, get status of rules, etc. From first glance at that error message, it appears to result from pfctl trying to load something invalid. It's hard to see, but is it saying the rule was "*:"?
A package like pfBlocker routinely downloads updated lists of IP addresses and then generates new firewall rules designed to block the IPs. It then uses pfctl to insert the new or updated rules into the firewall. These tables are persisted and reloaded on a reboot. Could be pfBlocker that is choking on one of its downloaded rules files, and as a consequence it might be in turn choking the pfctl utility.
The above is just a theory, though… :)
Bill
-
The pasting of the error didnt come out right. I added spaces to see if it wouldnt try to convert it to an emoticon
There were error(s) loading the rules: pfctl: DIOCADDRULE: Device busy - The line in question reads [ 0 ] :
Your theory is sound. It pulls rules daily from iblocklist.com so its possibly that one of the lists messed something up. Shouldnt i get that though when it reloads the rules. lets say I make FW rule changes etc and that provokes a reload or when I update my manual list in pfblocker it always reloads them and I never get the error. I noticed it with the recent reboots only..
Anyhow.. I was not intending to derail the topic.. I appreciate the insight.
-
The last few builds of snort I've installed have had some real issues with CPU usage on my machines. Has anyone else seen snort burn 100% of a CPU core, per instance, even when idle?
This always happens on startup of snort for the first 30 seconds or so but after that it should settle down. Right now snort has been disabled on all 3 of my boxes (1 at home, 2 at work) it runs my CPUs at 100% all the time and I end up with poor throughput as a result.
-
The last few builds of snort I've installed have had some real issues with CPU usage on my machines. Has anyone else seen snort burn 100% of a CPU core, per instance, even when idle?
This always happens on startup of snort for the first 30 seconds or so but after that it should settle down. Right now snort has been disabled on all 3 of my boxes (1 at home, 2 at work) it runs my CPUs at 100% all the time and I end up with poor throughput as a result.
Using 100% of a CPU is definitely not right. Check and make sure you have only one instance of Snort per interface it's enabled on using this command –
ps -ax | grep snortAssuming you do not have Barnyard2 enabled, you should see exactly one Snort process per interface. Each will have a UUID along with the physical interface name in the command-line arguments. If you have Barnyard2 enabled, you will also see one Barnyard2 process per interface.
If the command above shows the correct number of interfaces, then I would start disabling some rules to see if maybe one is consuming CPU time. You can also enable the Preprocessor Stats on the Preprocessors tab. This will give you statistics for all the preprocessors and may help identify a problem area.
Bill
-
The last few builds of snort I've installed have had some real issues with CPU usage on my machines. Has anyone else seen snort burn 100% of a CPU core, per instance, even when idle?
This always happens on startup of snort for the first 30 seconds or so but after that it should settle down. Right now snort has been disabled on all 3 of my boxes (1 at home, 2 at work) it runs my CPUs at 100% all the time and I end up with poor throughput as a result.
Using 100% of a CPU is definitely not right. Check and make sure you have only one instance of Snort per interface it's enabled on using this command –
ps -ax | grep snortAssuming you do not have Barnyard2 enabled, you should see exactly one Snort process per interface. Each will have a UUID along with the physical interface name in the command-line arguments. If you have Barnyard2 enabled, you will also see one Barnyard2 process per interface.
If the command above shows the correct number of interfaces, then I would start disabling some rules to see if maybe one is consuming CPU time. You can also enable the Preprocessor Stats on the Preprocessors tab. This will give you statistics for all the preprocessors and may help identify a problem area.
Bill
Barnyard2 is off and I do have one process per interface. I'll try enabling stats and see if that tells me what it's doing.
EDIT: Question. Where exactly would I find the stats it's collecting?
-
I have been noticing Memory increasing at times also. CPU usage has usually been fairly low thou.
Looks like I had two dead snort process's on my box. Those are both on my WAN interface.
ps -ax | grep snort
30859 ?? SNs 7:06.60 /usr/pbi/snort-amd64/bin/snort -R 44200 -D -q -l /var/log/snort/snort_bce044200 –pid-path /var/run --nolock-pidfile -G 44200 -c /usr/pbi/s
34575 ?? SNs 6:37.28 /usr/pbi/snort-amd64/bin/snort -R 9739 -D -q -l /var/log/snort/snort_em09739 --pid-path /var/run --nolock-pidfile -G 9739 -c /usr/pbi/snort
47151 ?? Ss 27:02.58 /usr/pbi/snort-amd64/bin/snort -R 44200 -D -q -l /var/log/snort/snort_bce044200 –pid-path /var/run --nolock-pidfile -G 44200 -c /usr/pbi/s
63296 ?? Ss 26:48.10 /usr/pbi/snort-amd64/bin/snort -R 44200 -D -q -l /var/log/snort/snort_bce044200 --pid-path /var/run --nolock-pidfile -G 44200 -c /usr/pbi/s(After shutting down Snort from the Interface GUI.)
ps -ax | grep snort
47151 ?? Ss 27:04.42 /usr/pbi/snort-amd64/bin/snort -R 44200 -D -q -l /var/log/snort/snort_bce044200 –pid-path /var/run --nolock-pidfile -G 44200 -c /usr/pbi/s
63296 ?? Ss 26:49.93 /usr/pbi/snort-amd64/bin/snort -R 44200 -D -q -l /var/log/snort/snort_bce044200 --pid-path /var/run --nolock-pidfile -G 44200 -c /usr/pbi/spkill snort
This killed the two dead processes.
I restarted Snort on the Interface GUI and all seems ok now. Memory down 20-30%
ps -ax | grep snort
69121 ?? Ss 0:08.67 /usr/pbi/snort-amd64/bin/snort -R 44200 -D -q -l /var/log/snort/snort_bce044200 –pid-path /var/run --nolock-pidfile -G 44200 -c /usr/pbi/s
91224 ?? Ss 0:00.63 /usr/pbi/snort-amd64/bin/snort -R 9739 -D -q -l /var/log/snort/snort_em09739 --pid-path /var/run --nolock-pidfile -G 9739 -c /usr/pbi/snort -
Is this on 32/64bit versions or only on 64bit?
I havent had issues at all running 32 bit.
-
Is this on 32/64bit versions or only on 64bit?
I havent had issues at all running 32 bit.
My systems are all 64-bit.
