@sgmc is softflowd not starting? or does it start, then stop?

softflowd on my netgate router hasn't run properly or exported flows since May 2021. I expect it's due to this issue:

https://redmine.pfsense.org/issues/10436?tab=history

bump, can someone maybe shed some light on this.

@ovecka said in Traffic monitoring reduces bandwidth to a third:

I've already tried adding more resources to the VM

Have you tried passing through the NICs pfsense uses.
And using another NIC for other functions on Proxmox

Just upgraded to 21.05/2.5.2 and am greeted with this nifty window when logging into ntopng:
baf5caa5-4f3a-48a9-9da4-9d1fb5adab2c-image.png
The instructions link is this https://github.com/ntop/ntopng/blob/dev/doc/README.geolocation.md.

Package Manager says it is:
0.8.13_10

And logging into ntopng says it is:
4.2.210601 (0) - Community Edition

EDIT: Solved
Solution:

Go to Diagnostics->ntopng Settings->Utilities->GeoLite2 DB License Key, press Update GeoLite2DB button Restart ntopng service

You may see that the old files (pre-upgrade) are still on disk.

ls -lah /usr/local/share/GeoIP total 25628 drwxr-xr-x 3 root wheel 1.0K May 6 12:00 . drwxr-xr-x 52 root wheel 1.5K Jun 5 05:16 .. -rw-r--r-- 1 806011168 806011168 55B May 4 06:40 COPYRIGHT.txt -rw-r--r-- 1 806011168 806011168 11M May 4 06:39 GeoLite2-Country-Blocks-IPv4.csv -rw-r--r-- 1 806011168 806011168 3.5M May 4 06:40 GeoLite2-Country-Blocks-IPv6.csv -rw-r--r-- 1 806011168 806011168 9.6K May 4 06:40 GeoLite2-Country-Locations-de.csv -rw-r--r-- 1 806011168 806011168 9.7K May 4 06:40 GeoLite2-Country-Locations-en.csv -rw-r--r-- 1 806011168 806011168 9.8K May 4 06:40 GeoLite2-Country-Locations-es.csv -rw-r--r-- 1 806011168 806011168 10K May 4 06:40 GeoLite2-Country-Locations-fr.csv -rw-r--r-- 1 806011168 806011168 15K May 4 06:40 GeoLite2-Country-Locations-ja.csv -rw-r--r-- 1 806011168 806011168 11K May 4 06:40 GeoLite2-Country-Locations-pt-BR.csv -rw-r--r-- 1 806011168 806011168 15K May 4 06:40 GeoLite2-Country-Locations-ru.csv -rw-r--r-- 1 806011168 806011168 11K May 4 06:40 GeoLite2-Country-Locations-zh-CN.csv -rw-r--r-- 1 root wheel 3.9M May 4 06:40 GeoLite2-Country.mmdb -rw-r--r-- 1 root wheel 2.0M May 6 12:00 GeoLite2-Country.tar.gz -rw-r--r-- 1 root wheel 3.8M Mar 2 09:30 GeoLite2-Country.tar.gz.orig -rw-r--r-- 1 806011168 806011168 398B May 4 06:40 LICENSE.txt -rw-r--r-- 1 806011168 806011168 116B May 4 06:40 README.txt drwxr-xr-x 2 root wheel 32K May 6 12:00 cc

Manually running fails ntopng-geoip2update.sh:

ntopng-geoip2update.sh Fetching GeoLite2-City fetch: https://geolite.maxmind.com/download/geoip/database/GeoLite2-City.tar.gz: No address record GeoLite2-City.tar.gz download failed

EDIT: My MaxMind settings are in pfBlockerNG. Wrong, they are in Diagnostics->ntopng Settings->Utilities->GeoLite2 DB License Key.

Maybe an issue with the upgrade? My MaxMind settings are in pfBlockerNG.

-LamaZ

Just upgraded to 21.05. This still works after replying the patch. I just figured out that the file is actually somewhere else and then likely gets copied to the /var folder.

/usr/local/share/ntopng/scripts/plugins/alerts/security/unexpected_ntp/user_scripts/flow/unexpected_ntp.lua

@gertjan Ok, thank you for your reply. Understood.

@johnj

I also ran it on my SG3100, fairly easy to configure, but there were some modifications that had to be made to display the map's correctly. That info was on this forum.

I have stopped running it mostly because my SG3100 cpu was getting a bit stressed running it, Suricata and PfblockerNG.

When I get a better firewall I will probably install it again.

@rai80 said in ntopng on 2.5.0 settings lost after restart service:

Since on 2.5.0 when I restart ntopng service all my preferences/settings are gone. Seems like it does a reset of everything.

Btw. I use ramdisk for /var/db. Don't know if this has affect.

fixed in NTOPNG 0.8.13_10:
https://redmine.pfsense.org/issues/11640

@pwnell got the same, only reset of ntopng to factory defaults helped

@viktor_g said in Ntopng GUI launch double URL issue...:

@nz Please provide more information
Same error in another browser? Same problem after refreshing the page? What's your full WebGUI url?

We actually have 3 PF installs with the exact same issue.
Started in PF v2.50
Hoping the new update to PF v2.5.1 and PFBlocker v3.0.0_16, would help.
Sadly it did not. Whatever the issue is, it just carried over.
And I just tried both Firefox and Chrome. Same result.
Refreshing doesn't help, because the actual URL is the problem.

So issue is this:
-Going to Diagnostics > ntopng
-or "ntop settings" page, click "Access ntopng"

A double URL is being called.
Comes up as:
FULL URL: http://192.168.0.2/://192.168.0.2:3000

If you manually change the URL to
http://192.168.0.2:3000
its all good, ntopng login comes up fine.

Whatever is causing the issue is from
http://192.168.0.2/ntopng_redirect.php file.
Since this is the module being executed.

And just a few hours ago, we removed the ntopng package "Keep settings" turned OFF, to do fresh install.
Rebooted the PF box.
Reinstalled fresh ntopng.
Same issue.

So why does http://192.168.0.2/ntopng_redirect.php, call up:
this page: http://192.168.0.2/://192.168.0.2:3000
instead of this page: http://192.168.0.2:3000

thanks

@kbohlken

I haven't installed Splunk Forwarder on pfSense itself. But, I'll throw out what I did to get pfSense logs into Splunk.

I have two syslog-ng servers setup that I can forward my pfSense logs to via syslog. I then have the Universal Splunk Forwarder setup on the two syslog servers to forward the logs into Splunk. I only use one of the syslog servers at a time, the other one is a backup in case I take the main syslog server down for maintenance. Both syslog-ng servers run on Ubuntu server in virtual machines. I set it up this way so that I don't have to always have my Splunk server running, I just need to have one of the syslog-ng servers running collecting the logs, which uses less system resources on the VM's host system.

I used this guide and modified it for my use case:
https://www.nuharborsecurity.com/splunk-data-onboarding-success-with-syslog-ng-and-splunk-part-2

There's a list here. I've used Bandwidthd before and it was fine for our purpose.