Nprobe on pfSense - experiences?

keyser

Hi All.

I have been using NtopNG for a while on pfSense, but since it's the community edition it's rather useless (no history).
So a few months back I bought a NtopNG Enterprise embedded for my Raspberry pi, and started feeding it with flowdata form pfsense using the softflowd package.
It works just fine, but the information level is quite limited in flow metadata only.

I'm missing the all the advanced DNS and DPI decoding and alerting features NtopNG/Nprobe has!

So I was wondering - does anyone have experience with running Nprobe on pfSense for the capture and analytics, and then have it send the detailed information to a separate NtopNG install via its builtin ZMQ?

Ntop has a guide on how to install/run Nprobe on pfSense, so it probably works fine, but does any one have any experiences?

Unfortunately I'm not a coder, because I would love to create a simple Nprobe pfSense package to allow simple install and configuration of Nprobe.
I hate adding manual repositories and adding non-persistent configuration to pfSense.

NollipfSense

@keyser said in Nprobe on pfSense - experiences?:

Ntop has a guide on how to install/run Nprobe on pfSense, so it probably works fine

Yes, here: https://www.ntop.org/guides/nprobe/how_to_start.html although, I have never configured nor have any experience with it. If you follow their instructions, it should work...

dennypage

@keyser said in Nprobe on pfSense - experiences?:

So I was wondering - does anyone have experience with running Nprobe on pfSense for the capture and analytics, and then have it send the detailed information to a separate NtopNG install via its builtin ZMQ?

FWIW, you could use ntopng on pfSense to send the flows via zmq rather than than nprobe...

keyser

@dennypage Does that still work? As far as I can see that “feature” was deprecated way back when Ntop went NtopNG. It also makes sense it no longer works because then you could use a free NtopNG instead of a licensed nProbe on remote systems.

dennypage

@keyser Yes. See the ntopng command line doc here.

The option

--interface zmq://<IP address>

is used to collects flows, and the option

--export-flows tcp://<IP address>

is used to export flows. As far as I know, these are not deprecated.

FWIW, there was an older form of flow collection that is marked as deprecated

--interface tcp://<IP address>

but I expect it still works as it is still in the help message produced by

--help

The current pfSense package does not allow for configuration of these parameters however.

dennypage

@keyser This might be of interest to you: Redmine #14375.

I have a package built if you would like to test.

keyser

@dennypage Hi Denny

Really great that you are willing to put this effort into providing more options with NtopNG on pfSense.

I already have a licensed NtopNG Enterprise Embedded running on a Raspberry Pi 4 collecting flows from Softflowd and a licensed nProbe Pro embedded I have (Portmirror on switch). I have been testing the difference between flows recorded by SoftflowD on pfSense and Nprobe Pro (portmirrored LAN to pfSense).
The difference is HUGE. NProbe does a lot of DPI analysis + records all DNS queries and fills alll that in as flow metadata to NtopNG. So in the UI you can the client sessions with domainnames instead of IP addresses and a lot of trafficanalysis of the sessions.
So it is much easier to dissect/analyze what happened in the nProbe flows than from SoftflowD.

I record this to a Clickhouse server on the same Pi. Runs great, and gives me 180 days history of all flows back in time.

I have decided to forego running the NtopNG package on pfSense as it cannot be licensed and work fully featured. I realize that one could perhaps avoid the licensing cost of a nProbe (And a port switchmirror) by setting up nTopNG like you suggested, but its a “heavy” package with lots of discwrites for nothing compared to nProbe. So I’ll stick with the nProbe Embedded as the deluxe flow generator, and look forward to testing the built-in pf flow exporter in 24.03 as the poormans flow solution.

But your work is still very much appreciated, and I’m sure it will be very well recieved in the community