pfBlockerNG-devel cron job stuck @ Loading DNSBL Statistics...

subodh.c

Hello All,

I am on version:
pfsense: 2.4.5-p1
pfBlockerNG-devel: 2.2.5_33

Cron job hangs in the below step.

group


===[  DNSBL Process  ]================================================

 Loading DNSBL Statistics...

I have cleared all the logs still would not go away.
Is there a way to clear the stats

Thanks

subodh.c

The default dashboard in the index page started timing out. Ended up un-installing. I will wait for few more days before I install it again.

Gertjan

Hi,

What's your memory size ?

How many feeds / feeds size ?

At the "Loading DNSBL Statistics..." stage, it's reading the SQIGHT PHP-based database(s).
They are here : /var/db/pfblockerng/ - the dot sqlite files (2, 3 or 4 files).

What about stopping pfBlockerNG, delete these two files, and restart - force an update reload.

subodh.c

@Gertjan said in pfBlockerNG-devel cron job stuck @ Loading DNSBL Statistics...:

Hello @Gertjan ,
Thanks a lot for responding.

What's your memory size ?
4GB RAM

How many feeds / feeds size ?
~29 feeds.

delete the below files

dnsbl.sqlite
dnsbl_levent.sqlite

Did not start. Forced install and it still freezes

Gertjan

@subodh-c said in pfBlockerNG-devel cron job stuck @ Loading DNSBL Statistics...:

29 feeds.

'29' is nothing.
How many (hundreds - tens of) thousands entries - see my image above. My question mark is on the second codon, not the first.
pfBlockerNG-devel accepts everything "up to the memory ceiling" and even more - and then the system starts to swap .... which means : it becomes one big crawl.

What I would do : when issues, start with one or two feeds, and re test.

@subodh-c said in pfBlockerNG-devel cron job stuck @ Loading DNSBL Statistics...:

dnsbl.sqlite
dnsbl_levent.sqlite

When the process started, they were recreated ?

subodh.c

Here's an extract of the force update. It hangs on the last line Saving DNSBL database...
As suggested I have tried this with my complete list and also with just EasyList_Adware 100 entries.
forceupdate.zip
File sizes:

[2.4.5-RELEASE][admin@blackbox.home]/var/db/pfblockerng: ls -la
total 48
drwxr-xr-x  11 root  wheel   512 Jun 17 10:21 .
drwxr-xr-x   8 root  wheel   512 Jun 17 09:46 ..
drwxr-xr-x   2 root  wheel   512 Jun 16 21:46 ET
drwxr-xr-x   2 root  wheel   512 Jun 16 21:46 deny
drwxr-xr-x   2 root  wheel  1024 Jun 17 10:21 dnsbl
-rw-r--r--   1 root  wheel     0 Jun 16 22:00 dnsbl.sqlite.bkp
-rw-r--r--   1 root  wheel     0 Jun 17 10:21 dnsbl_levent.sqlite
-rw-r--r--   1 root  wheel     0 Jun 16 21:47 dnsbl_levent.sqlite.bkp
drwxr-xr-x   2 root  wheel   512 Jun 16 22:03 dnsblalias
drwxr-xr-x   2 root  wheel  1024 Jun 17 10:21 dnsblorig
-rw-r--r--   1 root  wheel     0 Jun 17 10:21 mastercat
-rw-r--r--   1 root  wheel     0 Jun 17 10:21 masterfile
drwxr-xr-x   2 root  wheel   512 Jun 16 21:46 match
drwxr-xr-x   2 root  wheel   512 Jun 16 21:46 native
drwxr-xr-x   2 root  wheel   512 Jun 16 21:46 original
drwxr-xr-x   2 root  wheel   512 Jun 16 21:46 permit
-rw-r--r--   1 root  wheel  2152 Jun 17 10:21 pfbdnsblsuppression.txt

Memory graph:

CPU Graph:

RonpfS

Disable BBC_DGA_Agr URL, Force Reload All

[ BBC_DGA_Agr ]			 Reload . completed ..
  ----------------------------------------------------------------------
  Orig.    Unique     # Dups     # White    # TOP1M    Final                
  ----------------------------------------------------------------------
  955687   64447      1          0          0          64446                
  ----------------------------------------------------------------------

  DNSBL FAIL - Skipped! Use previous data, if found:
/var/db/pfblockerng/dnsbl/BBC_DGA_Agr.bk:64446: error: unknown keyword 'local-'
read /var/unbound/check.conf failed: 1 errors in configuration file

[ CoinBlocker_All ]		 Reload [ 06/17/20 10:22:40 ] . completed ..

To be on the safe side, Disable pfblockerNG, reboot, Enable pfblockerNG.

subodh.c

@RonpfS Moved to pfblockerng and it seems to be working fine.