Barnyard2 high CPU usage

romainp · Mar 10, 2015, 4:15 AM

Hi guys.
I have setup snort and barnyard2 on 2 interfaces (wan and DMZ) with some parameters coming from some how-tos found on the forum. The alerts feed a snorby database. Until recently, all was working fine but recently, the barnyard2 processes consume most of the CPU time (30-40% each).

I have tried the following:

Reinstall pfsense from scratch and restore a backup
Run different mysql (I run mariadb but for mist settings it the same as mysql) optimizing scripts to tune some settings (cache, etc).
Clear the SID in the snorby DB

Still, the processes of barnyard2 use a lot of CPU. I only have a vdsl link and very poor traffic coming from the internet to my web server I host behind pfense.

I really don't know how to start troubleshoot this issue..any help will be very appreciated on this :)
Thanks!

bmeeks · Mar 10, 2015, 9:16 PM

@romainp:

Hi guys.
I have setup snort and barnyard2 on 2 interfaces (wan and DMZ) with some parameters coming from some how-tos found on the forum. The alerts feed a snorby database. Until recently, all was working fine but recently, the barnyard2 processes consume most of the CPU time (30-40% each).

I have tried the following:

Reinstall pfsense from scratch and restore a backup

Run different mysql (I run mariadb but for mist settings it the same as mysql) optimizing scripts to tune some settings (cache, etc).

Clear the SID in the snorby DB

Still, the processes of barnyard2 use a lot of CPU. I only have a vdsl link and very poor traffic coming from the internet to my web server I host behind pfense.

I really don't know how to start troubleshoot this issue..any help will be very appreciated on this :)
Thanks!

Is this sustained high usage? By that I mean after say an hour does it back off? Barnyard2 does this weird business as of the last Barnyard update from upstream where it reads the sid-msg.map file and tries to repopulate/update the signature references table in the MySQL database. This happens with every Barnyard2 startup, so after each rules update or anytime you make a change in the config, Barnyard2 is restarted and this process kicks off. On my firewall, it runs the CPU utilization to about 75% for around 30 minutes after Barnyard2 startup. After about 30 minutes things settle down to normal for me.

Bill

romainp · Mar 11, 2015, 2:36 AM

Hi,
And thanks for your help.
In fact the CPU load for the 2 barnyard2 process was steady (35% for each so they consume all the CPU).
I have tried to not enable all the signature definition with no luck.. Tried to install mytop (top for mysql) and can't find any issue with the load on mysql.
So, I have decided to drop my snorby database, recreate a new one and reinstall snorby and… TADA! The load on the CPU rise when the signature definition are updated or when the 2 barnyard2 processes start but after a while (less than 10 mns) all is quieter and seem to stay like this. I will not say victory until several days (and some alerts). I will keep you updated.
Thanks!

bmeeks · Mar 11, 2015, 10:26 PM

I have three Barnyard2 instances writing to a Snorby database. My example of 30 minutes of high CPU utilization is probably a bit high. It does get to 75%, but I have not timed it precisely. I just come back later and check and things have calmed down to normal. I'm not liking how Barnyard2 1.13 talks to databases at all… :(.

Bill

romainp · Mar 12, 2015, 4:18 PM

I have the same feeling as you. I really do not know how barnyard2 perform with several sensors and do some queries/update.
I definitively have to put some monitoring on the mysql/mariadb database to know exactly what's going on a do better things than "drop the database and reinstall snorby" :)

Maybe barnyard2 itself should produce some alerting info when it sees that there is an issue with the database.

Well, I will start to find some good monitoring solution for mysql and keep you updated.

Romain