@Azgarech:

Hello,

I am looking to send the suricata log to snorby. To do so I need to activate barnyard functionnality.
I went to Suricata: Interface LAN - Barnyard2 Settings

I did let the default option checks and add my mysql informations. and enabled Barnyard2.

Then I did restart suricata service. (after restarting only the interface didn't work) .

The logo with the red cross is always here close to barnyard in the interfaces information.
I click on it still don't want to start.

Here are the logs from the system logs:

Apr 17 13:19:37 barnyard2[82555]: Suppressed: 0 (0.000%) Apr 17 13:19:37 barnyard2[82555]: =============================================================================== Apr 17 13:19:37 barnyard2[82555]: Packet breakdown by protocol (includes rebuilt packets): Apr 17 13:19:37 barnyard2[82555]: ETH: 0 (0.000%) Apr 17 13:19:37 barnyard2[82555]: ETHdisc: 0 (0.000%) Apr 17 13:19:37 barnyard2[82555]: VLAN: 0 (0.000%) Apr 17 13:19:37 barnyard2[82555]: IPV6: 0 (0.000%) Apr 17 13:19:37 barnyard2[82555]: IP6 EXT: 0 (0.000%) Apr 17 13:19:37 barnyard2[82555]: IP6opts: 0 (0.000%) Apr 17 13:19:37 barnyard2[82555]: IP6disc: 0 (0.000%) Apr 17 13:19:37 barnyard2[82555]: IP4: 0 (0.000%) Apr 17 13:19:37 barnyard2[82555]: IP4disc: 0 (0.000%) Apr 17 13:19:37 barnyard2[82555]: TCP 6: 0 (0.000%) Apr 17 13:19:37 barnyard2[82555]: UDP 6: 0 (0.000%) Apr 17 13:19:37 barnyard2[82555]: ICMP6: 0 (0.000%) Apr 17 13:19:37 barnyard2[82555]: ICMP-IP: 0 (0.000%) Apr 17 13:19:37 barnyard2[82555]: TCP: 0 (0.000%) Apr 17 13:19:37 barnyard2[82555]: UDP: 0 (0.000%) Apr 17 13:19:37 barnyard2[82555]: ICMP: 0 (0.000%) Apr 17 13:19:37 barnyard2[82555]: TCPdisc: 0 (0.000%) Apr 17 13:19:37 barnyard2[82555]: UDPdisc: 0 (0.000%) Apr 17 13:19:37 barnyard2[82555]: ICMPdis: 0 (0.000%) Apr 17 13:19:37 barnyard2[82555]: FRAG: 0 (0.000%) Apr 17 13:19:37 barnyard2[82555]: FRAG 6: 0 (0.000%) Apr 17 13:19:37 barnyard2[82555]: ARP: 0 (0.000%) Apr 17 13:19:37 barnyard2[82555]: EAPOL: 0 (0.000%) Apr 17 13:19:37 barnyard2[82555]: ETHLOOP: 0 (0.000%) Apr 17 13:19:37 barnyard2[82555]: IPX: 0 (0.000%) Apr 17 13:19:37 barnyard2[82555]: IPv4/IPv4: 0 (0.000%) Apr 17 13:19:37 barnyard2[82555]: IPv4/IPv6: 0 (0.000%) Apr 17 13:19:37 barnyard2[82555]: IPv6/IPv4: 0 (0.000%) Apr 17 13:19:37 barnyard2[82555]: IPv6/IPv6: 0 (0.000%) Apr 17 13:19:37 barnyard2[82555]: GRE: 0 (0.000%) Apr 17 13:19:37 barnyard2[82555]: GRE ETH: 0 (0.000%) Apr 17 13:19:37 barnyard2[82555]: GRE VLAN: 0 (0.000%) Apr 17 13:19:37 barnyard2[82555]: GRE IPv4: 0 (0.000%) Apr 17 13:19:37 barnyard2[82555]: GRE IPv6: 0 (0.000%) Apr 17 13:19:37 barnyard2[82555]: GRE IP6 E: 0 (0.000%) Apr 17 13:19:37 barnyard2[82555]: GRE PPTP: 0 (0.000%) Apr 17 13:19:37 barnyard2[82555]: GRE ARP: 0 (0.000%) Apr 17 13:19:37 barnyard2[82555]: GRE IPX: 0 (0.000%) Apr 17 13:19:37 barnyard2[82555]: GRE LOOP: 0 (0.000%) Apr 17 13:19:37 barnyard2[82555]: MPLS: 0 (0.000%) Apr 17 13:19:37 barnyard2[82555]: OTHER: 0 (0.000%) Apr 17 13:19:37 barnyard2[82555]: DISCARD: 0 (0.000%) Apr 17 13:19:37 barnyard2[82555]: InvChkSum: 0 (0.000%) Apr 17 13:19:37 barnyard2[82555]: S5 G 1: 0 (0.000%) Apr 17 13:19:37 barnyard2[82555]: S5 G 2: 0 (0.000%) Apr 17 13:19:37 barnyard2[82555]: Total: 0 Apr 17 13:19:37 barnyard2[82555]: ===============================================================================

can you help me with it ?

EDIT: Apparently Barnyard2 don't even go to the database login

You may need to enable the viewing of more log entries.  The snippet you posted is Barnyard2 shutting down.  If you display more log entries, you may seen the error thrown by Barnyard2.  My guess is that database login is failing or it is not finding the specified host.  Many users, including me, are using the Barnyard2 feature to feed Snorby and it works.

Bill

Snort is a real stickler for requiring adherence to all the RFCs for web servers.  If a site's server deviates one little bit, the HTTP_INSPECT processor in Snort will pounce… ;D.

Glad you found it.  You can either suppress that alert or disable that rule entirely.  There are a number of those HTTP_INSPECT rules that will false positive.

Bill

Sorry it took a little longer than I anticipated, but I did finally get around to replicating the problem and will have the fix in the next Suricata update.  I'm hoping that won't be too far in the future.  I'm waiting for FreeBSD ports to update to the 2.0.7 release.  If that continues to drag out, then I will just post a separate GUI package update to fix this log management problem.

Bill

Do as @Supermule says, and also be sure you wait on the package installation screen until you see it print a text message that says something like "…package installation completed...".  I don't remember the exact wording, but if you leave the package installation screen to quickly, the last part of the install will not complete and Snort will be missing from the SERVICES menu.

Second possibility is a NanoBSD install with not enough free space on the /tmp partition.  If you are running a Nano install, first manually increase the /tmp partition to 100 MB (the default is 40 MB) before trying to reinstall Snort.
Bill

Hey guys,

Found this while working on some rules;

https://github.com/inliniac/suricata/tree/master/contrib/file_processor

This directory contains what's needed for reading the JSON file /var/log/suricata/files-json.log and processing those entries against plugins.  Included are plugins for checking the MD5 of the observed file on the network against already created reports on anubis.iseclab.org, malwr.com, and threatexpert.com.  If you have a virustotal.com API key (free, though see the terms of use on virustotal.com/documentation/public-api/), you can enable the virustotal.com plugin and configure your API key so you can check the MD5 against over forty AV vendors' results.

F.

@bmeeks:

The CPU utilization problem is more likely caused by the IPsec decryption of that video stream.  Snort can't decrypt that traffic to actually look at it.

Isn't that what I said? LOL

@bmeeks:

Snort puts your WAN interface into promiscuous mode, so it will then see any traffic crossing the interface.  With NAT, I prefer to run Snort on the LAN.  That might help in your case, but it depends on your network and what you are protecting behind the various interfaces.

Ahh, that makes sense. I might try that.

@bmeeks:

When you have this spiking problem, have you tried stopping Snort and seeing what happens to CPU utilization then?

Sure, the "snort" process in `top' that shows 90% CPU utilization goes away. As one might expect.  :P

thanks a lot guys!

The custom rule solution works perfect for me.

@dgall:

Thanks for the answers!!! One last question is there a way to see when you updated if the rules are free or paid subscription ? When I log at the view the MANAGE RULE SET LOG I cant see anything that shows that the rules are paid or not. Its probably there but I do not see it.

No, you can't tell because the file names from the VRT web site are identical.  Your Oinkcode is read by the VRT rules download server and it decides which package of rules to send down to you.  It gets them from one of two directories depending on "paid" or "free" subscription.  There is nothing you need to do on your end other than disabling the Snort GPLv2 Community Rules if you were using those.  They are already bundled into the paid VRT rules.

Bill

Manual installation along with having the GUI interface hooks into pfSense is extremely hard to do.  It requires hand-editing a number of critical files.  However, even if you did that, the new Snort PHP files won't run on 2.0.3 pfSense because they call and use system features that are only available in pfSense 2.1.x and higher.

So the short answer is you can't have the GUI with the current Snort PHP package on pfSense versions prior to 2.1.x.  You can manually download and install the old *.tbz package, but you will need to use Snort exclusively from the CLI (command line) like you would if you installed it on a plain-vanilla FreeBSD 8.1 machine.  You will have to create the snort.conf file by hand, download rules by hand, and start-stop Snort from the command-line.

Bill

Yes works like a charm

They are simply Base64 encoded.  You can use one of several online tools to convert the string from encoded Base64 to plaintext. Here is one site I found using a quick Google search:  http://www.motobit.com/util/base64-decoder-encoder.asp.

The string is Base64 encoded to avoid issues with any XML reserved characters.  You can copy it literally as-is from one config.xml to the other, or if you want to decode it and paste the plaintext into a new Snort GUI window, then use an online Base64 tool like the one I referenced.

Bill

Hello,
I had a closer look on these settings. Great !  :) Very good and impressive job.
Thank you for your answers, Bill.
Bye !

Been busy with Suricata lately, havent played with Snort in some time, but you are right. My fault. As of now you cant negate the appID part. But you can negate src, dst, ports as usual. For an example these rules would trigger;

alert tcp $HOME_NET any -> $EXTERNAL_NET ![80,8080] (msg:"HTTP Port Unauthorized"; appid: http; classtype:policy-violation; sid:12171008; rev:1;) alert tcp $HOME_NET any -> $EXTERNAL_NET !443 (msg:"HTTPS Port Unauthorized"; appid: https; classtype:policy-violation; sid:12171009; rev:1;)

appID is really a work in progress and its not voodoo magic, most of the detection script are just looking for cert, protocol, etc…but I guess thats why they made it Open, it will grow and refine itself pretty fast with the community.

Cheers.

F.

You can always create some traffic of your own to trigger some of the Snort VRT rules as a test.

You can see what rules are actually being enforced if you look in this file /usr/pbi/snort-amd64/etc/snort/snort__{uuid}__{if}/rules/snort.rules where {uuid} is a random number and {if} is the physical interface Snort is running on.

The choices are grayed out when you choose a policy because the chosen policy dictates the rules selected.  If you want to overrule that, you can do so on the SID MGMT tab using the features there.

Bill

@jeffh:

@bmeeks:

The memory of the PHP process for Suricata is being exhausted.  That is currently hard-coded for 256 MB in the file /usr/local/pkg/suricata/suricata.inc.  You can edit that file and try bumping up the value.

Thanks Bill. Do you happen to know if the Snort package has the same limitation? If so is manually bumping the memory of the PHP process for Snort an option too?

Yes, both packages share a lot of the same code.  The parameter is set in the /usr/local/pkg/snort/snort.inc file for Snort.

Bill

OK I connected a tunnel from another pfSense box using 2.2.1-RELEASE and another using version 2.0.1-RELEASE ….I get the same result.

I can ping from other computers on the remote LAN subnet to computers on the local LAN subnet but not from the pfSense boxes themselves.

This should be an IPsec topic not a IDS/IPS topic.  I will start a new thread in the IPsec fourm.

@jeffh:

In addition to what Bill said, what I do is run Snort on both WAN and LAN interfaces.

On the LAN interface I have blocking disabled and quite a few rules enabled so I can get some visibility into what is happening on my network.

On the WAN interface I have blocking enabled and only specific security related rules enabled. All rules that are running on the WAN (in blocking mode) are also running on the LAN (in alert mode).

This allows me to block security threats, while still seeing what NAT'd local devices are having their traffic blocked, as well as alert on other rules that may not be security threats or that may have higher rates of false positives.

This is the exact same thing that I do and it works great.  It does take a bit more memory and processing power, and a lot more if you're doing barnyard.  I ended up turning the barnyard push notifications off because of this…but with this combination, you get the blocking on the WAN and can then trace it to your internal LAN ip address.

I thought the same as I did find a reference to that while searching the forum. I changed the Web protocol to HTTP but that didn't help… I am not sure what it is.. I have 3 W8.1 machines that do the same thing. If I get some time I'll dig a little deeper.

Yes I am very glad and thanks again for your help ...