pfSense Zeek (fka Bro) Package

markoverholser

@thiamata did you select an interface and enable Zeek on the "General" tab of the Zeek package configuration? Usually when we see this behavior, it is because there is no log yet generated, because the process hasn't actually been enabled. You must check the box "Enable Zeek NSM" on the General page, and select one or more interfaces from the "Zeek Interfaces" selection box, or it will not monitor anything.

cplmayo

Wanted to check in and see where development on this was at. I previously installed Zeek using the FreeBSD packages and had it running on my SG-5100. I am a huge proponent of Zeek vs other traffic monitoring solutions. If the pfSense webgui could be built to handle multiple interfaces, plugin installation, and changing logging type to JSON it would go a long way towards making this an easy solution to deploy Zeek. Ideally packages to ship logs that are more robust than syslog would be preferable as well.

When I set it up I manually installed Splunks log forwarder and used systemcmd to launch both the forwarder and zeek at boot. While it worked quite well it wasn't easy to make changes and I quickly overran the Splunk daily limits.

Pondering the idea of trying this again with my XG-7100 and pushing the data to an ELK stack via filebeats but this would also require manual installation.

markoverholser

@cplmayo Shadonet has submitted the package into the pfSense package repository, and I believe the pull has been accepted, but I'm not sure what else needs to be done for it to show up in the pfSense UI. As for the other enhancement suggestions, can you head over to the GitHub development repo for the package and submit your ideas there in the "issues" department? https://github.com/shadonet/pfSense-pkg-zeek

markoverholser

@cplmayo also, as far as getting the logs out, I saw someone once used an external mount, most likely NFS, and had the Zeek package set to drop the logs in the mount. Then, they ran the agents (Splunk forwarder, Logstash, Filebeat, Fluentd, whatever) on the remote system to keep the load down on the firewall. Perhaps that helps?

cplmayo

@markoverholser said in pfSense Zeek (fka Bro) Package:

@cplmayo also, as far as getting the logs out, I saw someone once used an external mount, most likely NFS, and had the Zeek package set to drop the logs in the mount. Then, they ran the agents (Splunk forwarder, Logstash, Filebeat, Fluentd, whatever) on the remote system to keep the load down on the firewall. Perhaps that helps?

Should have thought of that myself... I went added three enhancement requests.

I know next to nothing about PHP and haven't tried to learn but I am willing to provide what ever support I can. Would have to look through the code and how packages work on pfSense but I see Zeek as a huge improvement to pfSense.

viktor_g

@cplmayo You can create a feature request:
https://docs.netgate.com/pfsense/en/latest/development/feature-requests.html

cplmayo

@viktor_g said in pfSense Zeek (fka Bro) Package:

@cplmayo You can create a feature request:
https://docs.netgate.com/pfsense/en/latest/development/feature-requests.html

Never submitted one of those before but I guess theres no time like the present; Request: 11396. Hopefully I didn't come off like a total noob in how I structured it.

occamsrazor

@markoverholser said in pfSense Zeek (fka Bro) Package:

@cplmayo Shadonet has submitted the package into the pfSense package repository, and I believe the pull has been accepted, but I'm not sure what else needs to be done for it to show up in the pfSense UI.

I see it in my available packages list today and have installed it. Can anyone point me to a basic setup guide for someone who's never used Bro/Zeek? What do you need to view its results in a graphical user interface? Just looking to play to see what it can do, nothing fancy.

thiamata

@markoverholser

sorry for the delay, ... .. .

Yes, an interface was selected.

steping down to the shell and open zeekctl and their using the comannd deploy helps the system to come up correctly.

Now it seemes to work fine.

regards Thiamata

thiamata

@thiamata typo "and there" ;-)

markoverholser

@occamsrazor Oh, cool, thanks for pointing that out. I checked a couple of times after the code was merged, but didn't see the option in the user interface, but it's there now! Awesome.

occamsrazor

@markoverholser said in pfSense Zeek (fka Bro) Package:

@occamsrazor Oh, cool, thanks for pointing that out. I checked a couple of times after the code was merged, but didn't see the option in the user interface, but it's there now! Awesome.

Welcome. I still have no idea at all how to use it :-) Do you need some kind of external process running (Grafana etc?) to actually view the results? Or is there some way within pfSense user interface?

markoverholser

@occamsrazor In the pfSense interface, you can "review" the logs, but honestly at the moment it's my opinion that functionality is only good for sanity checking the logs to make sure they're being generated and look roughly like you expect them to (if you know what you expect them to look like, of course).

The best way to review the data is to ship it out to something else. I think for most people that would be whatever SIEM they already have running. If you're starting from scratch, Elasticsearch is freely available, but there are many little gotchas that can trip someone up, so it's not "for the faint of heart." Humio Cloud is dead simple, but requires signing their EULA, and now requires a corporate email address to sign up. You could also consider running the free Splunk, which has a limit to the amount of data you can supply it, but is pretty powerful.

I should probably try to hack together a recipe for getting the Zeek data from pfSense to something (Elastic or Splunk, perhaps). Some of it has already been covered by Eric Ooi's blogs (https://www.ericooi.com/zeekurity-zen-zeries/), but the plumbing would be slightly different since most people would probably not run the log forwarding agent directly on pfSense, and instead would have the logs made available elsewhere via SMB or NFS, and then run the log forwarding agent on some external system consuming the logs remotely and then sending them to the SIEM.

Hope that helps!

JGdgZPQatDDjpA

@thiamata said in pfSense Zeek (fka Bro) Package:

zeekctl

Your info here helped me fix it.
The real question is why do we have to jump through this hoop for this package?

thiamata

After the last zeek-update (4.0.2) I cannot start zeek , ... .. .

trying to use the zeekctl deploy, shows the following result:
--- snipp ---
zeekctl deploy
checking configurations ...
zeek scripts failed.
fatal error in /usr/local/share/zeek/site/local.zeek, line 16: can't find misc/app-stats
--- snipp end ---

also trying to delete and install the application newly shows the same behavior.

any ideas?

regards Thiamata

thiamata

sorry I missed this:

--- snipp ---
more local.zeek
##! Local site policy. Customize as appropriate.
##!
##! This file will not be overwritten when upgrading or reinstalling!

This script logs which scripts were loaded during each run.

@load misc/loaded-scripts

Apply the default tuning scripts for common tuning settings.

@load tuning/defaults

Load the scan detection script.

@load misc/scan

Log some information about web applications being used by users

on your network.

@load misc/app-stats

--- snipp end ---

last entry is line 16, ... .. .

regards Thiamata

PS
A reinstall and a remove and installing again does not help

during installation I got some eorros relating some cfg files (zeekctl.cfg, node.cfg, networks.cfg) in /usr/local/etc
The first two I could identify as zeek related cfgs. So removing these files helps to bypass the these errors. But with networks.cfg I am not sure, if this file is only a zeek related cfg.

Is there an option to completely remove zeek and install from scratch like a (nearly) fresh system, without knowing any information from the instance installed before?

regards Thiamata

markoverholser

@thiamata Can you comment out the line @load misc/app-stats (change it to # @load misc/app-stats by adding the # at the beginning) and try to load Zeek again?

Did you install with pkg install or via the web UI? I think there shouldn't be much state kept between installations but if you are at the command line you could rm -rf /usr/local/share/zeek after uninstalling to remove the remaining elements (if there are any).

lncc63

This post is deleted!

thiamata

Hi

remarking helps zeek to come up again, but I need to run zeekctl deploy again on the shell.

I am still looking for howto implement custom scripts in the correct way.

secondly, what is needed to get this misc/appstat running in the correct way. This question is still open, ... .. .

For it seems that zeek is running for now in the known way, ... .. .

thanx 4 hlp

regards Thiamata

markoverholser

@thiamata I don't think it's necessary to run misc/appstat, I've never used that functionality. So, I think it's safe to just remove that from your local.zeek

As for running other custom scripts, put them somewhere and use an @load directive in your local.zeek file to load them.

For example, if you download and unzip the IcannTLD package (https://github.com/corelight/icannTLD) to a specific directory, you can add a line like @load /opt/icanntld/scripts/ (assuming that's where it ends up) and it will load the script and use it when Zeek loads.