PfBlockerNG deduplication is out of sync a lot

dbennett

Greetings,

I'm noticing that pfBlockerNG is producing a yellow exclamation point on the dashboard and when I hover over it stating that the 'deduplication is out of sync' and to reload to correct the issue.

This is happening a lot. Why

Look forward to your comments

Dino

BBcan177

Check the pfblockerng.log for details.

You can try to clear out all the previously downloaded feeds:

Uncheck - Enable pfBlockerNG
Uncheck - Keep Settings

Save

Then check both checkboxes
Force Update

LowKnee

@BBcan177
Hi, I know pretty old, but I run into this problem after upgrading iron CE 2.7.2 to Plus 25.07.1-RELEASE (amd64).
I uninstalled the packages before I started the upgrade process.
With CE 2.7.2 with 3.2.0_20 _devel everything was fine.

Since 25.07.1 the IP Depublication stays out of sync.
Whatever I do, with 3.2.7 _devel or with 3.2.7, uninstall/reinstall, disable/enable, force reload all or only IP or Update, reboot, I am not getting it working.
DNSBL is working correctly.

Could you please help me out of this?

Thanks in advance

Marc

jrey

@LowKnee

Just out of curiosity are you referring to the Database Sanity Check reporting that "these two counts should match"

it the count is off by 1 (which I suspect is your case)
there was a fix (manual code change) to change masterfile to mastercat
in pfblolckerng.sh you want to change this

change the line 
from		s1="$(grep -cv ^${ip_placeholder2}$ ${masterfile})"
to		s1="$(grep -cv ^${ip_placeholder2}$ ${mastercat})"

There is also an edge case if the count is greater than one, here is how that goes

if in the deny directory you have say two flies (because of the list / file selection you have and they have repeat addresses

file 1 has say 100 lines
file 2 has say 10 lines (but those 10 lines are also in file 1, file 2 is a subset)

you get two uniquely named deny files and then
when the "count" is calculated on the deny directory it sees 110 entries
when the "count:" is calculated on the "mastercat" file it only contains 100 entries
the count doesn't match
in my case the issue was caused by full list I had selected, also having an available subset lists (I had inadvertently selected one of) this causing two deny files with some of the same (overlapping data)
I unselected the subset and bingo matched again, was a "my bad" selection.

Edit: this applied to 25.07 (and 25.07.1) and pfblockerng 3.2.7 as it is labelled on those versions of pfSense