Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    If you are having Barnyard2 troubles with the last Snort update – try this

    IDS/IPS
    9
    37
    14.8k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • G
      gscasny
      last edited by

      @fearnothing:

      @BBcan177:

      You could try to

      pgrep barnyard  or  ps aux | grep barnyard

      and see how many processes are running. And then "kill (pid) (duplicate one)

      Note to anyone trying this in future - if you're doing it locally, this will be fine, but in a remote ssh, you should probably use "ps auxw" - just using "aux" truncates the output to the screen width and if that means the word "barnyard" wouldn't be displayed, "ps aux" will get nothing.

      ps auxww | grep barnyard2

      This will wrap the output of ps to as many columns as needed for your display, piping to 'grep barnyard2' will only show the barnyard2 processes (including your grep command) so you can see which exact one is older and shouldn't be there.

      1 Reply Last reply Reply Quote 0
      • H
        ha11oga11o
        last edited by

        guys, sorry for BUMP this post, but i feel mine problem belong here. I have exact same issue.

        Feb 25 12:33:52 pfsense barnyard2[70828]: FATAL ERROR: database mysql_error: Duplicate entry '7690-1' for key 'PRIMARY' 	SQL=[INSERT INTO sig_reference (ref_id,sig_id,ref_seq) VALUES ('15818','7690','1');]
Feb 25 12:33:52 pfsense barnyard2[70828]: Barnyard2 exiting
Feb 25 12:33:52 pfsense barnyard2[70828]: database: Closing connection to database "snort"
Feb 25 12:33:52 pfsense barnyard2[70828]: ===============================================================================
Feb 25 12:33:52 pfsense barnyard2[70828]: Record Totals:
Feb 25 12:33:52 pfsense barnyard2[70828]:    Records:           0
Feb 25 12:33:52 pfsense barnyard2[70828]:    Events:           0 (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]:    Packets:           0 (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]:    Unknown:           0 (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]:    Suppressed:           0 (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]: ===============================================================================
Feb 25 12:33:52 pfsense barnyard2[70828]: Packet breakdown by protocol (includes rebuilt packets):
Feb 25 12:33:52 pfsense barnyard2[70828]:       ETH: 0          (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]:   ETHdisc: 0          (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]:      VLAN: 0          (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]:      IPV6: 0          (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]:   IP6 EXT: 0          (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]:   IP6opts: 0          (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]:   IP6disc: 0          (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]:       IP4: 0          (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]:   IP4disc: 0          (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]:     TCP 6: 0          (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]:     UDP 6: 0          (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]:     ICMP6: 0          (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]:   ICMP-IP: 0          (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]:       TCP: 0          (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]:       UDP: 0          (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]:      ICMP: 0          (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]:   TCPdisc: 0          (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]:   UDPdisc: 0          (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]:   ICMPdis: 0          (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]:      FRAG: 0          (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]:    FRAG 6: 0          (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]:       ARP: 0          (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]:     EAPOL: 0          (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]:   ETHLOOP: 0          (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]:       IPX: 0          (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]: IPv4/IPv4: 0          (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]: IPv4/IPv6: 0          (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]: IPv6/IPv4: 0          (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]: IPv6/IPv6: 0          (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]:       GRE: 0          (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]:   GRE ETH: 0          (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]:  GRE VLAN: 0          (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]:  GRE IPv4: 0          (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]:  GRE IPv6: 0          (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]: GRE IP6 E: 0          (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]:  GRE PPTP: 0          (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]:   GRE ARP: 0          (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]:   GRE IPX: 0          (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]:  GRE LOOP: 0          (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]:      MPLS: 0          (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]:     OTHER: 0          (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]:   DISCARD: 0          (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]: InvChkSum: 0          (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]:    S5 G 1: 0          (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]:    S5 G 2: 0          (0.000%)
Feb 25 12:33:52 pfsense barnyard2[70828]:     Total: 0         
Feb 25 12:33:52 pfsense barnyard2[70828]: ===============================================================================
Feb 25 12:34:26 pfsense check_reload_status: updating dyndns WANGW
Feb 25 12:34:26 pfsense check_reload_status: Restarting ipsec tunnels
Feb 25 12:34:26 pfsense check_reload_status: Restarting OpenVPN tunnels/interfaces
Feb 25 12:34:26 pfsense check_reload_status: Reloading filter

        2.1.5-RELEASE (amd64)
        built on Mon Aug 25 07:44:45 EDT 2014
        FreeBSD 8.3-RELEASE-p16

        command prompt says one PID, right?

        $ ps aux | grep barnyard
root   28066  0.0  0.0  9068  1384  ??  S     1:56PM   0:00.00 grep barnyard

        Im using snort Security 2.9.7.0 pkg v3.2.3

        So, will recreating database fix this problem? Or is there any other fix?

        Snort is runing, but barnyard2 does not! Mine database server is on local 192.168.x.x IP, and many other things works great with it. But when i disable Barnyard2 -  restart Snort - enable Barnyard2 - i reads database, but after 5-15 minutes process is red, and it fails from database with same error. Proces for Barnyard2 eats 100% CPU for 2-3 minutes too. When CPU goes down i check processes and see Barnyard2 is down :(

        Many thnx in advance.

        Cheers !

        1 Reply Last reply Reply Quote 0
        • bmeeksB
          bmeeks
          last edited by

          You will have to wipe the sig_reference table in the database.  That means delete all the entries in it.

          Login to your MySQL database and issue this command:

          
DELETE FROM sig_reference;

          Here is what I did to pretty much permanently fix this problem for me.  The issue is within Barnyard2 itself and the way it updates a particular table of references and signature mappings.  First, on the BARNYARD tab of each Snort interface check the box to disable signature reference table updates.

          I then made a cron task on my MySQL box that runs daily.  It runs PulledPork on my MySQL box.  Next, it runs the DELETE command above to clear out the sig_reference table.  Then I run Barnyard2 locally on the MySQL box to let it update the table again using the files updated by PulledPork.

          Doing all of this requires installing PulledPork and Barnyard2 on your MySQL server directly.

          Bill

          1 Reply Last reply Reply Quote 0
          • H
            ha11oga11o
            last edited by

            Bill,

            thank you for your help. It seems that works. As for Cron and other things.. it must wait to build up mine brain muscles in that area.

            Many thnx, you just help one happy n00b :)

            1 Reply Last reply Reply Quote 0
            • bmeeksB
              bmeeks
              last edited by

              It will work for a while, but you most likely will have to periodically repeat clearing out that table.  Barnyard2 is just messed up in this area IMHO since the 2.1.13 update last year.  Each time it starts up, it reads the sid-msg.map file and tries to update this table.  The problem is it does a relatively poor job of coordinating with other possible Barnyard2 processes that may be updating the table at the same time.  It also runs into an issue whenever subtle updates to the references associated with a rule SID get rearranged.

              I found the solution I described in a post by one of the Security Onion guys while doing a vast Google search for answers.  Search for "barnyard2 sig_reference delete" and that should get you some relevant results to start combing through.

              Bill

              1 Reply Last reply Reply Quote 0
              • H
                ha11oga11o
                last edited by

                Yup, delete db lasted till this morning, need to do same fix again. Is it me or this is common problem? Im curious if its that old why devs not fix it so far?

                Thanks

                1 Reply Last reply Reply Quote 0
                • bmeeksB
                  bmeeks
                  last edited by

                  @ha11oga11o:

                  Yup, delete db lasted till this morning, need to do same fix again. Is it me or this is common problem? Im curious if its that old why devs not fix it so far?

                  Thanks

                  It is a somewhat common problem.  It was introduced with the 2.1.13 update of Barnyard2.  I don't know why it has not been addressed by the Barnyard2 devs.  There seem to be two triggers for the error.  The first one is more than one Barnyard2 instance updating the same MySQL database.  The second one is a change in the order references are listed for an individual SID in the sid-msg.map file.  This happens when the rule authors (usually ET or VRT) add a new reference for a SID and do so by inserting it into the existing list instead of appending it at the end.  This changes the index values that are part of the primary key generated by Barnyard2.  Oh…and this can also happen when an existing reference is edited.

                  Bill

                  1 Reply Last reply Reply Quote 0
                  • H
                    ha11oga11o
                    last edited by

                    Bill,

                    thanks for clarification. At least now i know its not mine mistake, misconfiguration or similar. I have only one instance to that database.

                    Many thnx :)

                    1 Reply Last reply Reply Quote 0
                    • J
                      JFlórez
                      last edited by

                      @bmeeks:

                      Some folks have reported database errors coming from Barnyard2 with the latest Snort package.  One of the things that changed recently with the Snort package is Barnyard2 was updated from 1.2 to 1.3.  This was done to keep pace with upstream and because 1.3 of Barnyard2 offers a warm restart ability via SIGHUP.  Also, the newer Barnyard2 version supports the new v2 sid-msg.map file format.  This updated format includes more information about rule signatures.  The Snort package on pfSense now generates v2 sid-msg.map files for use by Barnyard2.

                      However, there were apparently some important changes to the way Barnyard2 v1.3 interacts with a MySQL database as compared to the older versions.  This can become a problem when your MySQL database contains records put there by a previous Barnyard2 version.  Here are two links to post on the web that can help you fix this problem if you have it.

                      This is the one I recommend you try first –
                      http://sourceforge.net/p/snort/mailman/message/31925851/

                      Here is another longer thread with similar information –
                      https://groups.google.com/forum/#!topic/barnyard2-users/IIoyClc7XTc

                      In the first link I posted, you can copy and paste the script there into a PuTTY session with your DB server and run it all quite easily.  I was having issues just like users have reported here on the Forum, and running the scripts in the first link above fixed them.

                      Bill

                      Thank you Bill.

                      Second link worked for me
                      (https://groups.google.com/forum/#!topic/barnyard2-users/IIoyClc7XTc]https://groups.google.com/forum/#!topic/barnyard2-users/IIoyClc7XTc)

                      Here is the error I was getting:
                      … barnyard2: FATAL ERROR: database mysql_error: Duplicate entry '502-6' for key 'PRIMARY'#012#011SQL=[INSERT INTO sig_reference (ref_id,sig_id,ref_seq) VALUES ('23242','502','6');]

                      Basically what I did was to clear the "sig_reference" table (barnyard2 was not running), and start barnyard2.  That fixed my issue.

                      For those of you who may not know how to go about clearing a table, follow the steps listed below.

                      Stop all instances of barnyard that are feeding data to the database.
                      Go to the server where the database is located and issue the following 4 commands:

                      mysql -p    (after you press enter, you will be prompted for the password for mysql)
                      use snort    (or whatever name you have given to the snort database - "snort" is the default)
                      delete from sig_reference;    (this will clear all data in the table.  Don't worry, it will be re-created later)
                      quit

                      Re-start barnyard2

                      That did it for me. No more FATAL ERRORS when starting barnyard!! Yahoooo!

                      1 Reply Last reply Reply Quote 0
                      • C
                        condealisson
                        last edited by

                        Hi guys, this topic really helped me, but in my case the solution was create two databases, one for each Barnyard instance (i have two monitored interfaces), no more "Duplicate entry" now, thanks!

                        1 Reply Last reply Reply Quote 0
                        • First post
                          Last post