pfBlocker not logging after 2.5.2 pfSense upgrade

cefleet · Jul 9, 2021, 4:56 PM

@ronpfs Looks like the sqlite files are correct

NickD 0 · Jul 9, 2021, 6:13 PM

I'm seeing the same issues with DNSBL. pfSense 2.5.2 upgrade with pfBlocker 3.0.0.16. I just noticed that all blocked HTTP requests are logged fine, however, blocked HTTPS requests are not logged.

cefleet · Jul 9, 2021, 10:59 PM

Looks like mine is logging only HTTP and not HTTPS as well.

NickD 0 · Jul 9, 2021, 11:21 PM

@cefleet looks like unbound was regressed from 1.13.x to 1.12.x in 2.5.2 due to some other issues... likely related? although IDK when 1.13.x was added to the main tree. Maybe a configuration option available in 1.13.x but not in 1.12.x is borking the logging?

https://docs.netgate.com/pfsense/en/latest/releases/2-5-2.html#dns-resolver

https://redmine.pfsense.org/issues/11915

https://redmine.pfsense.org/issues/11316

berthis1958 · Jul 9, 2021, 11:27 PM

@nickd-0 said in pfBlocker not logging after 2.5.2 pfSense upgrade:

@cefleet looks like unbound was regressed from 1.13.x to 1.12.x in 2.5.2 due to some other issues... likely related? although IDK when 1.13.x was added to the main tree. Maybe a configuration option available in 1.13.x but not in 1.12.x is borking the logging?

https://docs.netgate.com/pfsense/en/latest/releases/2-5-2.html#dns-resolver

https://redmine.pfsense.org/issues/11915

https://redmine.pfsense.org/issues/11316

oops .. very interesting. It seems a possible cause.

dotsch · Jul 10, 2021, 6:27 PM

@cefleet said in pfBlocker not logging after 2.5.2 pfSense upgrade:

Looks like mine is logging only HTTP and not HTTPS as well.

I have the same problems. Blocking works afer reload, but don't log anymore.

cefleet · Jul 13, 2021, 2:23 AM

I changed from Unbound mode to Unbound Python mode and that has seemed to have fixed things. I thought I did this the other day and it did not work. In any case, it appears to be working now. Thanks for everyone's input.

keyser · Jul 13, 2021, 5:40 AM

@cefleet Please monitor your disk usage as python mode on 21.05/2.5.2 has a an issue on some systems with slowly consuming all diskspace. The key issue is that no files/logfiles report a size / diskusage that accounts for the space usage - they remain sized like before. So you cannot locate the file/problem that fills the filesystem.

This leads to a situation where the filesystem is full, and you need to stop/start pfBlockerNG completely or reboot pfSense to regain your filesystem space.

cefleet · Jul 13, 2021, 2:11 PM

@keyser Thanks for the heads up. I will keep an eye on the disk usage. So far everything looks good.

dotsch · Jul 14, 2021, 10:43 AM

@cefleet said in pfBlocker not logging after 2.5.2 pfSense upgrade:

I changed from Unbound mode to Unbound Python mode and that has seemed to have fixed things. I thought I did this the other day and it did not work. In any case, it appears to be working now. Thanks for everyone's input.

Thank you very much for the hint. I can confirm, that it is working for me with Unbound Python and enabling Python in pfBlocker DNBL.

NickD 0 · Jul 14, 2021, 10:36 PM

@dotsch Same here, no issues with the python module and logging.

badprocess · Jul 16, 2021, 1:12 PM

Ok so i also enabled Python Unbound mode and actually the logs are working again. I'll monitor in the coming days the disk occupation

SillieWous · Jul 16, 2021, 3:35 PM

Same story for me. Upgraded on the 15th (as clearly visible in the pictures below). Next to not logging of DNSBL there also seems to be a memory leak, unsure if related.

Is it possible to switch to python unbound with DHCP registration? As it still says "Python DNSBL mode is not compatable with the DNS Resolver DHCP Registration option (Unbound will Crash)!" in the information for selecting python unbound.

maddy_in65 · Jul 17, 2021, 4:33 AM

I have faced similar issue, DNSBL stopped blocking after 2.5.2 upgrade. I reinstalled pfsense but it didn't work. Later I changed mode to python unbound and it started working. However post this upgrade, my CPU and memory utilization is very high, earlier I had large list of IP and DNSBL but after this upgrade i cant enable all the list. I am running with only few list b of IP and DNSBL but after this upgrade i cant enable all the list. I am running with only few list but memory is still high. ut memory is still high. Is there any solution for this.

Gertjan · Jul 17, 2021, 7:19 AM

@maddy_in65
What is your CPU doing ?
See here Diagnostics > System Activity
Or better : console or SSH access, option 8 and enter

top

See here for more info.

maddy_in65 · Jul 17, 2021, 8:22 AM

@gertjan
Here is "Top" output:

 processes:  4 running, 56 sleeping
CPU: 91.3% user,  0.0% nice,  8.7% system,  0.0% interrupt,  0.0% idle
Mem: 958M Active, 337M Inact, 750M Wired, 208K Buf, 1698M Free
ARC: 278M Total, 161M MFU, 105M MRU, 2654K Anon, 2044K Header, 8178K Other
     160M Compressed, 585M Uncompressed, 3.66:1 Ratio
Swap: 2048M Total, 2048M Free

  PID USERNAME    THR PRI NICE   SIZE    RES STATE    C   TIME    WCPU COMMAND
62378 root         88  20    0   468M   423M nanslp   0   0:37 132.04% ntopng
38952 unbound       1  77    0    91M    79M RUN      1   0:03  32.27% unbound
  347 root          1  52    0   102M    39M piperd   0   1:06  25.19% php-fpm
43559 root          1  20    0    13M  3508K CPU1     1   0:00   0.59% top
62111 root          4  20    0    22M  5356K kqread   1   0:00   0.29% redis-server
 9748 root          1  20    0    19M  6824K select   1   0:03   0.12% ntpd
36481 root          1  20    0    20M  9220K select   0   0:00   0.10% sshd
85972 root          5  52    0    11M  2592K uwait    0   0:00   0.06% dpinger
47972 dhcpd         1  25    0    23M    12M select   1   0:00   0.02% dhcpd
47822 root          1  20    0    18M  7552K kqread   0   0:00   0.01% lighttpd_pfb
 5149 root          3  20    0    18M  7168K select   1   0:06   0.01% pcscd
85427 root          5  52    0    11M  2592K uwait    1   0:00   0.01% dpinger
  346 root          1  52    0   102M    39M accept   0   2:03   0.00% php-fpm
48256 root          1  52    0   102M    39M accept   1   1:06   0.00% php-fpm
44666 root          1  52    0   102M    39M accept   1   0:51   0.00% php-fpm
62020 root          1  23    0   102M    38M accept   0   0:35   0.00% php-fpm
19548 root          1  20    0    11M  2648K select   0   0:07   0.00% syslogd
95969 root          2  20    0   229M   191M bpf      0   0:02   0.00% snort
19460 root          1  20    0   104M    36M nanslp   0   0:01   0.00% php-cgi
 8574 root          1  20    0    30M  9792K kqread   0   0:01   0.00% nginx
49739 root          1  20    0    61M    39M piperd   0   0:00   0.00% php_pfb
  345 root          1  20    0   102M    27M kqread   0   0:00   0.00% php-fpm
31190 root          1  20    0    12M  2956K bpf      1   0:00   0.00% filterlog
 8290 root          1  20    0    29M  9248K kqread   1   0:00   0.00% nginx
 9162 root          1  45    0    11M  2484K nanslp   1   0:00   0.00% cron
24662 root          1  20    0    21M  8448K select   0   0:00   0.00% mpd5
  376 root          1  40   20    11M  2840K kqread   0   0:00   0.00% check_reload_status
49643 root          1  20    0    11M  2212K kqread   0   0:00   0.00% tail_pfb
54837 root          1  20    0    44M    35M bpf      0   0:00   0.00% arpwatch
56501 root          1  20    0    44M    35M bpf      0   0:00   0.00% arpwatch
[2.5.2-RELEASE][admin@

Gertjan · Jul 17, 2021, 12:59 PM

@maddy_in65

Well ?
Who is this : ntopng ? I don't know what it is. Is it useful ? It's going at light speed (132 % CPU usage !?!). For me, that alarming. What if that "ntopng" is doing a lot of DNS requests ? That would explain something.
You stopped it ?

maddy_in65 · Jul 18, 2021, 6:22 AM

Yes I stopped it. CPU utilization is normal but Memory utilization is still high.

Gertjan · Jul 18, 2021, 10:59 PM

Use top again.
It shows the memory usage per process also.
Who is the winner ?

provels · Jul 19, 2021, 12:32 AM

Isn't high memory usage the nature of the Beastie anyway (file caching)?
I figure I paid for 100% of the memory and CPU and as long as I'm not running out, that's good ROI.