pfBlocker not logging after 2.5.2 pfSense upgrade

[Gertjan]

@maddy_in65
What is your CPU doing ?
See here Diagnostics > System Activity
Or better : console or SSH access, option 8 and enter

top

See here for more info.

[maddy_in65]

@gertjan
Here is "Top" output:

 processes:  4 running, 56 sleeping
CPU: 91.3% user,  0.0% nice,  8.7% system,  0.0% interrupt,  0.0% idle
Mem: 958M Active, 337M Inact, 750M Wired, 208K Buf, 1698M Free
ARC: 278M Total, 161M MFU, 105M MRU, 2654K Anon, 2044K Header, 8178K Other
     160M Compressed, 585M Uncompressed, 3.66:1 Ratio
Swap: 2048M Total, 2048M Free

  PID USERNAME    THR PRI NICE   SIZE    RES STATE    C   TIME    WCPU COMMAND
62378 root         88  20    0   468M   423M nanslp   0   0:37 132.04% ntopng
38952 unbound       1  77    0    91M    79M RUN      1   0:03  32.27% unbound
  347 root          1  52    0   102M    39M piperd   0   1:06  25.19% php-fpm
43559 root          1  20    0    13M  3508K CPU1     1   0:00   0.59% top
62111 root          4  20    0    22M  5356K kqread   1   0:00   0.29% redis-server
 9748 root          1  20    0    19M  6824K select   1   0:03   0.12% ntpd
36481 root          1  20    0    20M  9220K select   0   0:00   0.10% sshd
85972 root          5  52    0    11M  2592K uwait    0   0:00   0.06% dpinger
47972 dhcpd         1  25    0    23M    12M select   1   0:00   0.02% dhcpd
47822 root          1  20    0    18M  7552K kqread   0   0:00   0.01% lighttpd_pfb
 5149 root          3  20    0    18M  7168K select   1   0:06   0.01% pcscd
85427 root          5  52    0    11M  2592K uwait    1   0:00   0.01% dpinger
  346 root          1  52    0   102M    39M accept   0   2:03   0.00% php-fpm
48256 root          1  52    0   102M    39M accept   1   1:06   0.00% php-fpm
44666 root          1  52    0   102M    39M accept   1   0:51   0.00% php-fpm
62020 root          1  23    0   102M    38M accept   0   0:35   0.00% php-fpm
19548 root          1  20    0    11M  2648K select   0   0:07   0.00% syslogd
95969 root          2  20    0   229M   191M bpf      0   0:02   0.00% snort
19460 root          1  20    0   104M    36M nanslp   0   0:01   0.00% php-cgi
 8574 root          1  20    0    30M  9792K kqread   0   0:01   0.00% nginx
49739 root          1  20    0    61M    39M piperd   0   0:00   0.00% php_pfb
  345 root          1  20    0   102M    27M kqread   0   0:00   0.00% php-fpm
31190 root          1  20    0    12M  2956K bpf      1   0:00   0.00% filterlog
 8290 root          1  20    0    29M  9248K kqread   1   0:00   0.00% nginx
 9162 root          1  45    0    11M  2484K nanslp   1   0:00   0.00% cron
24662 root          1  20    0    21M  8448K select   0   0:00   0.00% mpd5
  376 root          1  40   20    11M  2840K kqread   0   0:00   0.00% check_reload_status
49643 root          1  20    0    11M  2212K kqread   0   0:00   0.00% tail_pfb
54837 root          1  20    0    44M    35M bpf      0   0:00   0.00% arpwatch
56501 root          1  20    0    44M    35M bpf      0   0:00   0.00% arpwatch
[2.5.2-RELEASE][admin@

[Gertjan]

@maddy_in65

Well ?
Who is this : ntopng ? I don't know what it is. Is it useful ? It's going at light speed (132 % CPU usage !?!). For me, that alarming. What if that "ntopng" is doing a lot of DNS requests ? That would explain something.
You stopped it ?

[maddy_in65]

Yes I stopped it. CPU utilization is normal but Memory utilization is still high.

[Gertjan]

Use top again.
It shows the memory usage per process also.
Who is the winner ?

[provels]

Isn't high memory usage the nature of the Beastie anyway (file caching)?
I figure I paid for 100% of the memory and CPU and as long as I'm not running out, that's good ROI.

[maddy_in65]

One thing I noticed post 2.5.2 upgrade is that most clients are loosing internet connection for a while and connects back. I have observed this on few WLAN clients (mobile, tablets), they shows no connectivity . I thought it might be issue with AP (Unifi). I tried rebooting APs but issue still persists. I also check on few LAN clients and they also had similar issue. I have removed pfblocker and the issue didn't appear again. Today I have installed pfblocker again with unbound mode and few feeds. As of now all clients are stable and I am monitoring devices.

Unbound and Snort has major memory utilization.

last pid: 31316;  load averages:  0.81,  0.68,  0.72                                                                up 0+06:28:52  15:51:28
65 processes:  1 running, 64 sleeping
CPU:  1.4% user,  0.0% nice,  0.6% system,  0.2% interrupt, 97.9% idle
Mem: 951M Active, 273M Inact, 531M Wired, 208K Buf, 1988M Free
ARC: 214M Total, 126M MFU, 84M MRU, 257K Anon, 1113K Header, 3189K Other
     114M Compressed, 368M Uncompressed, 3.22:1 Ratio
Swap: 2048M Total, 2048M Free

  PID USERNAME    THR PRI NICE   SIZE    RES STATE    C   TIME    WCPU COMMAND
  347 root          1  21    0   102M    39M accept   1   0:38   1.56% php-fpm
17542 root          2  20    0   230M   193M bpf      1   2:03   0.49% snort
90137 root          1  20    0    13M  3540K CPU1     1   0:00   0.11% top
70928 root          1  20    0    18M  7924K kqread   0   0:01   0.10% lighttpd_pfb
32031 root          1  20    0    28M  8616K kqread   0   0:03   0.08% nginx
 6428 root          3  20    0    23M    11M select   0   0:12   0.05% pcscd
21702 root          3  20    0   238M   197M bpf      1   0:03   0.04% snort
 4582 root          1  20    0   104M    36M nanslp   1   0:02   0.03% php-cgi
 4447 root          1  20    0    11M  2652K select   1   0:03   0.03% syslogd
71496 root          1  20    0    59M    38M piperd   1   0:01   0.02% php
37225 dhcpd         1  20    0    23M    12M select   0   0:00   0.02% dhcpd
23994 root          5  52    0    11M  2592K uwait    1   0:00   0.02% dpinger
36222 root          1  20    0    12M  2960K bpf      1   0:01   0.02% filterlog
78612 zabbix        1  20    0    19M  9056K nanslp   1   0:00   0.02% zabbix_agentd
24559 root          5  52    0    11M  2592K uwait    1   0:00   0.01% dpinger
33035 root          1  20    0    19M  6900K select   0   0:03   0.01% ntpd
28353 root          1  20    0    20M  9208K select   0   0:00   0.01% sshd
71058 root          1  20    0    10M  2148K kqread   1   0:00   0.01% tail_pfb
71320 root          1  20    0    61M    39M piperd   0   0:00   0.00% php_pfb
  345 root          1  20    0   102M    27M kqread   1   0:01   0.00% php-fpm
76619 root          1  20    0    44M    35M bpf      0   0:00   0.00% arpwatch
75582 root          1  20    0    44M    35M bpf      0   0:00   0.00% arpwatch
74827 root          1  20    0    44M    35M bpf      1   0:00   0.00% arpwatch
76215 root          1  20    0    44M    35M bpf      1   0:00   0.00% arpwatch
75224 root          1  20    0    44M    35M bpf      1   0:00   0.00% arpwatch
74047 root          1  20    0    44M    35M bpf      1   0:00   0.00% arpwatch
76918 root          1  20    0    44M    35M bpf      1   0:00   0.00% arpwatch
75978 root          1  20    0    44M    35M bpf      1   0:00   0.00% arpwatch
28242 unbound       2  20    0   383M   367M kqread   0   0:36   0.00% unbound
46839 root          1  52    0   104M    40M accept   1   0:33   0.00% php-fpm
  346 root          1  39    0   101M    39M accept   0   0:32   0.00% php-fpm

[Gertjan]

@maddy_in65 said in pfBlocker not logging after 2.5.2 pfSense upgrade:

28242 unbound 2 20 0 383M 367M kqread 0 0:36 0.00% unbound

I have

55688 unbound       2  20    0   105M    82M kqread   1   2:57   0.00% unbound

The difference is probably tour DNSBL usage (pfBlockerNG).

Still, these values are ok.

Btw : i'm not trying to block "everybody", as many and/or huge DNSBL/IP feeds do impact the system.
I'm just using the minimal :

[marcosm]

Using the latest pfBlockerNG-devel, IP stats won't work unless using auto rules. If using only the "Alias" actions, i.e. custom rules, the package is not able to keep track of the rule matches. I imagine this is the intended behavior.

Note: I've only tested on 21.05.

[RonpfS]

@marcos-ng From the under Action it says :

Note:

When manually creating 'Alias' type firewall rules, Prefix the Firewall rule Description with pfb_ .
This will ensure that that Dashboard widget reports those statistics correctly. 

Do not prefix with (pfB_) as those Rules will be auto-removed by package when 'Auto' rules are defined.

[marcosm]

@ronpfs
I missed that, thanks. I suppose then this should be confirmed by whomever reports that it's not working.

[n3xus_x3]

Hello,
today i updated the latest version of pfsense (2.5.2) , and i have the same problem . In the DNSBL configuration , i changed to localhost , i uninstalled pfblockerNG-devel,reinstalled , I have updated the lists , but i still have the same problem .
Someone solved it?

[Gertjan]

@n3xus_x3 said in pfBlocker not logging after 2.5.2 pfSense upgrade:

Someone solved it?

The initial issue was : pfBlockerNG stopped blocking.
Thus is stops logging.
I tend to say that's quiet normal.

Here are the 'pfBlockerNG' files :

ls -al /var/unbound/var/log/pfblockerng/

You means these files are not growing in size for you ?

Your stats say : it blocks things, as numbers are shown in the "Packet" column.

@n3xus_x3 said in pfBlocker not logging after 2.5.2 pfSense upgrade:

, i changed to localhost

Changed what ? Where ?

'localhost' is the default, and works :

Or I do not understand what you mean by "i changed to localhost".

[n3xus_x3]

Thanks for replay
in pfblockerng/alerts/Reports , i noticed that after pfsense update, the contents are not blocked like before , in the Deny section it seems that everything works

The problem is on DNSBL Block , it seems to me that the lists work randomly , I'm sure they should be a lot more

these are my DNS settings

I don't use Python mode in DNSBL

[Gertjan]

@n3xus_x3

I advise you look up why this so called Python mode was introduced.
There are some reddit posts where the author explains it.

To make a long story short :
To make DNSBL work better, info is needed that unbound won't log in detail.
Or worse, won't make available any more.
But, recently, the latest unbound versions, could interface with 'plugins' written in Python.

So, thre solution was : this python mode.
Unbound has to be used as a resolver of course - not as a forwarder as you do.

[SillieWous]

@gertjan

The initial issue was : pfBlockerNG stopped blocking.

Not sure what you're referring to here. The topic of this thread is 'stopped logging' not stopped blocking.

So, thre solution was : this python mode.

As long as that isn't fully compatible with the rest of pfSense e.g. Python DNSBL mode is not compatable with the DNS Resolver DHCP Registration option (Unbound will Crash)! that advise doesn't hold up. I'd gladly switch to python mode, however I can't.

[vjizzle]

@silliewous Hi! I solved this problem by disabling dnsbl mode in pfblocker and moving that functionality over to a rpi4 with adguard home. Pfblocker with dnsbl was giving me all kinds of crazy behavior and in the end I just need my internet to work while wfh.

[n3xus_x3]

I changed to python mode in DNSBL settings , it works a little better, I mean more content is blocked ,but i think to go back to the previous version of pfsense,I've never had any problems and all the lists I have are working fine

[vjizzle]

@n3xus_x3 Be aware that older versions of pfsense cannot run the latest version of pfblockerng which have a lot of fixes compared to older versions. For the time being pfblockerng looks death and is not actively being worked on, from what I can see at least.

[Gertjan]

@MG85

Here is my regex.
It's more test-of-proof sample for me. I remember finding it somewhere on Reddit.

^(.+[_.-])?adse?rv(er?|ice)?s?[0-9]*[_.-] #Regex RGX1
^(.+[_.-])?telemetry[_.-] #Regex RGX2
^ad([sxv]?[0-9]*|system)[_.-]([^.[:space:]]+\.){1,}|[_.-]ad([sxv]?[0-9]*|system)[_.-] #test RGX3
^adim(age|g)s?[0-9]*[_.-] #Regex RGX4
^adtrack(er|ing)?[0-9]*[_.-] #Regex RGX5
^advert(s|is(ing|ements?))?[0-9]*[_.-] #Regex RGX6
^aff(iliat(es?|ion))?[_.-] #Regex RGX7
^analytics?[_.-] #Regex RGX8
^banners?[_.-] #Regex RGX9
^beacons?[0-9]*[_.-] #Regex RGX10
^count(ers?)?[0-9]*[_.-] #Regex RGX11
^mads\. #Regex RGX12
^pixels?[-.] #Regex RGX13
^stat(s|istics)?[0-9]*[_.-] #Regex RGX14

Keep in mind : the ending
"Space # text string" needs to be unique.