If you are having Barnyard2 troubles with the last Snort update – try this
-
bill's patch has been added to snort already… barnyard2 needs to be updated but that is out of scope from the pfsense project.. The creators of barnyard2 would have to fix their code... When I run into the problem, I purge a few tables; start barnyard2 and everything is good again.
-
Would be interested to know if there's any update on this. I've followed all the instructions here and yet I'm still having the problem. Thought it was working after I updated snort and pfsense but the problem returned after an hour.
Go to the BARNYARD tab and down in the MySQL DB section on all your Snort interfaces (except one) and check the box to disable updates to the signature reference table. Leave this box unchecked on only a single Barnyard2 interface (typically the one that registers the most alerts).
The problem is with Barnyard2 itself and concurrent updates to the same DB table from multiple processes. The Barnyard2 guys, in my opinion, broke their code in the version 2.1.3 update a while back. The Snort and Suricata packages on pfSense use this updated version since the FreeBSD ports tree was also updated.
Bill
-
The option is already checked on both interfaces. Is it significant that I'm only getting the duplicate entry error for the 'event' table, and not 'sig_reference?'
-
The option is already checked on both interfaces. Is it significant that I'm only getting the duplicate entry error for the 'event' table, and not 'sig_reference?'
Do you by chance have the same rules running on more than one interface, and are the Snort instances each alerting on the same traffic? In other words, duplicate rules on WAN and LAN and each Snort instance alerts on the same traffic.
The Barnyard2 guys made a few "adjustments" to the MySQL DB code in their last update, and those adjustments seem to have caused a race condition with data inserts and some tables.
I was able to fix this by running the scripts linked earlier in this thread to clean up the tables, and then running only one BY2 interface with the checkbox not checked.
Bill
-
Nope, still happening.
Jun 30 11:00:17 barnyard2[33829]: database: Closing connection to database "snorby"
Jun 30 11:00:17 barnyard2[33829]: Barnyard2 exiting
Jun 30 11:00:17 barnyard2[33829]: FATAL ERROR: database mysql_error: Duplicate entry '5-20' for key 'PRIMARY' SQL=[INSERT INTO event (sid,cid,signature,timestamp) VALUES (5, 20, 18861, '2014-06-30 11:00:15');]
Jun 30 11:00:16 snort[57013]: [1:2003315:3] ET P2P Edonkey Search Reply [Classification: Potential Corporate Privacy Violation] [Priority: 1] {UDP} 77.80.253.67:27001 -> 192.168.1.102:27960
Jun 30 11:00:16 snort[57013]: [1:2003315:3] ET P2P Edonkey Search Reply [Classification: Potential Corporate Privacy Violation] [Priority: 1] {UDP} 77.80.253.67:27001 -> 192.168.1.102:27960This rule is ONLY present on LAN. Ironically it's a false positive detecting a Quake Live connection as Edonkey :P
-
The plot thickens… my snorby instance is still receiving alerts from the LAN interface despite the barnyard2 process for LAN being marked as stopped in the GUI.
-
Im seeing this rear its head again (im using Suricata), even thought I have the box checked for both sensors that are reporting…
-
Im seeing this rear its head again (im using Suricata), even thought I have the box checked for both sensors that are reporting…
Oops.. It's the event table this time as stated above.. just started happening with the last update..
-
gscasny, are you using this on Security Onion?
http://blog.securityonion.net/2014/06/new-barnyard2-nsm-rule-update-and.html
"You may have noticed previously that when barnyard2 started up, it would consume a large amount of CPU (on both the sensor and the server) for a while (more than a minute in some cases) while it updated Snorby's reference table. Multiply this by several barnyard instances per interface and several interfaces per physical sensor and you now have multiple instances fighting each other for scarce CPU resources.
To alleviate this, the barnyard2 folks introduced a new option called disable_signature_reference_table that allows you to disable the reference table update on all sensors, leaving just one barnyard2 instance on the server itself to update Snorby's reference table, avoiding the duplication of effort. I packaged the latest version of barnyard2 (version 2.1.13 Build 333) which contains this option and also updated the NSM scripts to add the new option to all barnyard2.conf files on all sensors. rule-update has been modified such that right after the master downloads new rules from the Internet, it will use barnyard2 to update Snorby's reference table. Finally, since we're now forcing the server to use barnyard2 to update Snorby's reference table, I updated the securityonion-server metapackage to require securityonion-barnyard2 as a dependency."
-
Im running snorby on Security Onion, but Suricata/Barnyard2 on pfsense. I see that behavior on pfsense with Barnyard2 (the intense CPU and long startup times)
-
And the issue I see now is with the event table, not the sig_references table. I have 2 sensors (LAN/WAN) and am not running the same rules on each:
barnyard2[58758]: FATAL ERROR: database mysql_error: Duplicate entry '2-802153' for key 'PRIMARY' SQL=[INSERT INTO event (sid,cid,signature,timestamp) VALUES (2, 802153, 62634, '2014-06-30 15:47:27');]
-
gscasny - I've just solved mine, so here's what was happening in case it's the same issue for you.
Turns out I had some kind of ghost barnyard2 process running in the background. The one that was visible in the PFSense GUI kept failing because this invisible intruder was reading from the same unified2 file and writing the entries to the snorby instance first, so when the visible by2 process tried to do it, the entry was already there. I shut down pfsense, deleted the entry that had caused the fatal error, and started PFSense again, and it's been running for about 2 hours so far without dying. If it fails overnight, I'll put another post in here to say so. -
You could try to
pgrep barnyard or ps aux | grep barnyard
and see how many processes are running. And then "kill (pid) (duplicate one)
-
gscasny - I've just solved mine, so here's what was happening in case it's the same issue for you.
Turns out I had some kind of ghost barnyard2 process running in the background. The one that was visible in the PFSense GUI kept failing because this invisible intruder was reading from the same unified2 file and writing the entries to the snorby instance first, so when the visible by2 process tried to do it, the entry was already there. I shut down pfsense, deleted the entry that had caused the fatal error, and started PFSense again, and it's been running for about 2 hours so far without dying. If it fails overnight, I'll put another post in here to say so.Right on the money.. that was it.. I feel stupid :) I should have checked that.. Thanks so much for putting my brain back on track!! I appreciate it! That's why my snorby logs kept populating when BY2 "died".. it had a ghost :)
Greg
-
You could try to
pgrep barnyard or ps aux | grep barnyard
and see how many processes are running. And then "kill (pid) (duplicate one)
Note to anyone trying this in future - if you're doing it locally, this will be fine, but in a remote ssh, you should probably use "ps auxw" - just using "aux" truncates the output to the screen width and if that means the word "barnyard" wouldn't be displayed, "ps aux" will get nothing.
-
You could try to
pgrep barnyard or ps aux | grep barnyard
and see how many processes are running. And then "kill (pid) (duplicate one)
Note to anyone trying this in future - if you're doing it locally, this will be fine, but in a remote ssh, you should probably use "ps auxw" - just using "aux" truncates the output to the screen width and if that means the word "barnyard" wouldn't be displayed, "ps aux" will get nothing.
ps auxww | grep barnyard2
This will wrap the output of ps to as many columns as needed for your display, piping to 'grep barnyard2' will only show the barnyard2 processes (including your grep command) so you can see which exact one is older and shouldn't be there.
-
guys, sorry for BUMP this post, but i feel mine problem belong here. I have exact same issue.
Feb 25 12:33:52 pfsense barnyard2[70828]: FATAL ERROR: database mysql_error: Duplicate entry '7690-1' for key 'PRIMARY' SQL=[INSERT INTO sig_reference (ref_id,sig_id,ref_seq) VALUES ('15818','7690','1');] Feb 25 12:33:52 pfsense barnyard2[70828]: Barnyard2 exiting Feb 25 12:33:52 pfsense barnyard2[70828]: database: Closing connection to database "snort" Feb 25 12:33:52 pfsense barnyard2[70828]: =============================================================================== Feb 25 12:33:52 pfsense barnyard2[70828]: Record Totals: Feb 25 12:33:52 pfsense barnyard2[70828]: Records: 0 Feb 25 12:33:52 pfsense barnyard2[70828]: Events: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: Packets: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: Unknown: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: Suppressed: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: =============================================================================== Feb 25 12:33:52 pfsense barnyard2[70828]: Packet breakdown by protocol (includes rebuilt packets): Feb 25 12:33:52 pfsense barnyard2[70828]: ETH: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: ETHdisc: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: VLAN: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: IPV6: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: IP6 EXT: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: IP6opts: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: IP6disc: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: IP4: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: IP4disc: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: TCP 6: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: UDP 6: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: ICMP6: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: ICMP-IP: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: TCP: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: UDP: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: ICMP: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: TCPdisc: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: UDPdisc: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: ICMPdis: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: FRAG: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: FRAG 6: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: ARP: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: EAPOL: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: ETHLOOP: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: IPX: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: IPv4/IPv4: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: IPv4/IPv6: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: IPv6/IPv4: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: IPv6/IPv6: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: GRE: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: GRE ETH: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: GRE VLAN: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: GRE IPv4: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: GRE IPv6: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: GRE IP6 E: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: GRE PPTP: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: GRE ARP: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: GRE IPX: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: GRE LOOP: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: MPLS: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: OTHER: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: DISCARD: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: InvChkSum: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: S5 G 1: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: S5 G 2: 0 (0.000%) Feb 25 12:33:52 pfsense barnyard2[70828]: Total: 0 Feb 25 12:33:52 pfsense barnyard2[70828]: =============================================================================== Feb 25 12:34:26 pfsense check_reload_status: updating dyndns WANGW Feb 25 12:34:26 pfsense check_reload_status: Restarting ipsec tunnels Feb 25 12:34:26 pfsense check_reload_status: Restarting OpenVPN tunnels/interfaces Feb 25 12:34:26 pfsense check_reload_status: Reloading filter2.1.5-RELEASE (amd64)
built on Mon Aug 25 07:44:45 EDT 2014
FreeBSD 8.3-RELEASE-p16command prompt says one PID, right?
$ ps aux | grep barnyard root 28066 0.0 0.0 9068 1384 ?? S 1:56PM 0:00.00 grep barnyardIm using snort Security 2.9.7.0 pkg v3.2.3
So, will recreating database fix this problem? Or is there any other fix?
Snort is runing, but barnyard2 does not! Mine database server is on local 192.168.x.x IP, and many other things works great with it. But when i disable Barnyard2 - restart Snort - enable Barnyard2 - i reads database, but after 5-15 minutes process is red, and it fails from database with same error. Proces for Barnyard2 eats 100% CPU for 2-3 minutes too. When CPU goes down i check processes and see Barnyard2 is down :(
Many thnx in advance.
Cheers !
-
You will have to wipe the sig_reference table in the database. That means delete all the entries in it.
Login to your MySQL database and issue this command:
DELETE FROM sig_reference;Here is what I did to pretty much permanently fix this problem for me. The issue is within Barnyard2 itself and the way it updates a particular table of references and signature mappings. First, on the BARNYARD tab of each Snort interface check the box to disable signature reference table updates.
I then made a cron task on my MySQL box that runs daily. It runs PulledPork on my MySQL box. Next, it runs the DELETE command above to clear out the sig_reference table. Then I run Barnyard2 locally on the MySQL box to let it update the table again using the files updated by PulledPork.
Doing all of this requires installing PulledPork and Barnyard2 on your MySQL server directly.
Bill
-
Bill,
thank you for your help. It seems that works. As for Cron and other things.. it must wait to build up mine brain muscles in that area.
Many thnx, you just help one happy n00b :)
-
It will work for a while, but you most likely will have to periodically repeat clearing out that table. Barnyard2 is just messed up in this area IMHO since the 2.1.13 update last year. Each time it starts up, it reads the sid-msg.map file and tries to update this table. The problem is it does a relatively poor job of coordinating with other possible Barnyard2 processes that may be updating the table at the same time. It also runs into an issue whenever subtle updates to the references associated with a rule SID get rearranged.
I found the solution I described in a post by one of the Security Onion guys while doing a vast Google search for answers. Search for "barnyard2 sig_reference delete" and that should get you some relevant results to start combing through.
Bill
