@mcury Great stuff much appreciated. I feel like I have to setup a perm VM server for various reasons so this may be the excuse that gets me off my heels. Since my primary driver is to "oversee" where users are visiting so I can apply restrictions, having an easy to digest interface or log that notes URL's might be a necessity but will see if I can get this up and running or take advantage of the 30 minute demo the developer offers to see where if any gaps exist and how I can close them. Thanks again, your input is very welcomed.

PS.. I love the Lawrence videos, great resource.

@andyrh LAN.

But listening on my. LAN it’s able to pick up on other interfaces such as my DMZ

@keyser Bump - no one?

To elaborate, I get these every once in a while:

06/Apr/2022 10:51:12 [LuaEngine.cpp:1206] WARNING Script failure [/usr/local/share/ntopng/scripts/lua/about.lua][/usr/local/share/ntopng/scripts/lua/about.lua:35: attempt to concatenate a nil value (field 'pro.systemid')]

06/Apr/2022 10:46:01 [LuaEngine.cpp:1206] WARNING Script failure [/usr/local/share/ntopng/scripts/lua/rest/v2/get/host/alert/list.lua][.../alert_definitions/other/alert_ip_outsite_dhcp_range.lua:60: attempt to concatenate a nil value (local 'ifid')]

Thanks again everyone. I ended up going with BandwidthD and disabling IPv6 in pfSense. I think this will work for the next month or so to figure out where the data is going each month. I suspect it's the two Apple TV 4K that we have but time will tell. I don't recalling use this much data when we used the Roku devices.

@ma0f97 Has no one an idea?

Ok After some research i did, the best practice is not monitoring the tunnel itself but some device in the end point using a ping software capable of ping multiple hosts and reporting feature or Zabbix Proxy.
Another way is using PfSense notifications with Telegram and e-mail then download a package called Watchdog and activate the notifications option.

@clokwork said in What is the safest way to disable ntopng?:

I like what it does, but I don't need it eating the amount of resources it does. Do I simply stop the process? I wan't to make sure it is stopped gracefully without causing issues.

In NTopNG Setting remove the tick in “enable NTopNG” and click save.

Are there any real disadvantages installing the latest version of ntopng manually according to these instructions?

https://www.ntop.org/guides/ntopng/third_party_integrations/pfsense.html

@lockie said in Logging URLs:

No one? :(

There is no “easy fix” to this anymore. Almost all websessions are HTTPS these days, so unless you do one of the two below, you can no longer get the info you are looking for.:

1: Setup a proxy and force all your clients to use it
2: Setup a transparent man in the middle proxy and make all your clients trust its certificate

I'm using the latest nightly build of ntopng pro and finding it's using a large amount of CPU time constantly, even when the user interface isn't open in a web browser, and even when there's barely any traffic passing through the server.

My configuration file contains:

-G=/var/run/ntopng.pid
--disable-login=1
-F "mysql;localhost;ntopng;flows;username;password"
-w=3050

However the same still occurs even if I remove the mysql line (I suspected it was file IO of writing flows, but apparently now).

Any suggestions to track down what ntopng is doing would be greatly appreciated - the load average on the server I'm running it on is through the roof!

@movielover76 What worked for me was to remove the package and then add it back. This removed collected data but preserved configuration in /usr/local/bandwidthd/etc/...

@toddwebnet Did you ever find one you like? I have the same use-case.

@deanfourie Service Watchdog? It's a plugin that restarts the service if it crashes.

@sgmc is softflowd not starting? or does it start, then stop?

softflowd on my netgate router hasn't run properly or exported flows since May 2021. I expect it's due to this issue:

https://redmine.pfsense.org/issues/10436?tab=history

bump, can someone maybe shed some light on this.

@ovecka said in Traffic monitoring reduces bandwidth to a third:

I've already tried adding more resources to the VM

Have you tried passing through the NICs pfsense uses.
And using another NIC for other functions on Proxmox