pfblockerng error: Unknown Not listed!

RonpfS

@l0rdraiden said in pfblockerng error: Unknown Not listed!:

Other minor problem is that when I delete logs in pfsense and pfbockerng packets in the widget after a while old logs appears for pfblockerng

There isn't much reasons to delete the logs
The alerts tab and widget keep the history of alerts. You can clear the widget counter.

RonpfS

@l0rdraiden said in pfblockerng error: Unknown Not listed!:

then services from google, like drive, and google.com doesn’t work because for some reason pfblockerng is blocking them, even this forum wasn’t working while other sites did.
First my configuration is fine and I have not done any change between it was working and then not, so it must be a bug in pfblockerng
Second, same problem in the previous version.
Third it doesn’t matter what you do, or change in the config the only way to make it work again is to delete everything and start from scrach.

Did you inspect pfblockerng.log? If some sites are blocked they should generate Alerts or FW log if logging is enabled.

l0rdraiden

@ronpfs said in pfblockerng error: Unknown Not listed!:

It’s not an error, the Alerts will display Unknown Not listed when the IP is no longer in any IPV4 lists or during a Update as the database is being rebuilt.

It's a error because those IP's are still being blocked even if they are not in the list, so, somehow, somewhere, pfblockerng has a list of ip's that doesn't belong to any list and is active and blocking them.

l0rdraiden

@ronpfs Why I delete the packets in the widget sometimes a few minutes later I have 20k blocks again which is the number of blocks accumulated in the last days, then I click on the number to see the log all those entries are not there, because I have deleted hours ago (for example) the pfsense blog. So it no working as intended.

l0rdraiden

@ronpfs said in pfblockerng error: Unknown Not listed!:

@l0rdraiden said in pfblockerng error: Unknown Not listed!:

then services from google, like drive, and google.com doesn’t work because for some reason pfblockerng is blocking them, even this forum wasn’t working while other sites did.
First my configuration is fine and I have not done any change between it was working and then not, so it must be a bug in pfblockerng
Second, same problem in the previous version.
Third it doesn’t matter what you do, or change in the config the only way to make it work again is to delete everything and start from scrach.

Did you inspect pfblockerng.log? If some sites are blocked they should generate Alerts or FW log if logging is enabled.

Not the file itself but the log in the web interface and I get new entries of pfblockerng (unknown not listed) blocking those sites that should be blocked because they don't belong to any list.
Is not that these are old log entries of blocked IP's that after an update are not part of the new IP's to be blocked, I'm talking that somehow is actively blocking those IPs even when they don't belong to any list, therefore new connections to those IP's shouldn't be blocked.

I hope is clear, probably there is no solution, I have pfblockerng disabled now because when I enable it, google.com, drive, this forum and other places don't work.

RonpfS

@l0rdraiden said in pfblockerng error: Unknown Not listed!:

Not the file itself but the log in the web interface

I am talking about the pfblockerng.log under the tab Logs
The Alerts are generated from ip_block.log

RonpfS

You can spot which file contain IP by doing

grep ^178.236.134 /var/db/pfblockerng/deny/*.txt /var/db/pfblockerng/original/*.orig

/var/db/pfblockerng/deny/PRI1_CINS_army_v4.txt:178.236.134.99
/var/db/pfblockerng/original/PRI1_CINS_army_v4.orig:178.236.134.99
/var/db/pfblockerng/original/PRI2_Alienvault_v4.orig:178.236.134.99 # Malicious Host

from the Diagnostics / Command Prompt

RonpfS

You should probably show your IPV4 table and pfblockerNG.log.

From what I see, VxVault_v4 should be a DNSBL table.

BBcan177

It looks like the Feed "VXVault" has a large CIDR: 208.0.0.0/4 which is causing your issue... Not sure which URL you are using for that? I checked the feed, and don't see that IP listed. The only way to overcome that CIDR blocked entry, is to create a "Permit Outbound" Alias to allow outbound to it.
In regards to the "Not Listed". The new DEVEL version keeps its own copy of the events in "/var/log/pfblockerng/ip_block.log" So if you wanted to start over, you could delete that log file.

RonpfS

@bbcan17 said in pfblockerng error: Unknown Not listed!:

Not sure which URL you are using for that?

Maybe something like : http://vxvault.net/ViriList.php

l0rdraiden

@BBcan17 @RonpfS

Thanks for your help. I am going to post all the evidences I found to clarify this issue.

First and example of the log where for example this forum is being blocked and they all appear as not listed... but somehow those IP's are part of the blocklist

This is the list of url inside that group

http://vxvault.net/ViriList.php?s=0&m=100

This is the ip_block.log

Jun 16 10:53:15,1770010922,igb0,LAN,block,4,6,TCP-S,192.168.1.110,208.123.73.199,55932,443,out,US,pfB_MalwareAndSites_v4,208.0.0.0/4,VxVault_v4,forum.netgate.com,pc,+
Jun 16 10:53:15,1770010922,igb0,LAN,block,4,6,TCP-S,192.168.1.110,208.123.73.199,55932,443,out,US,pfB_MalwareAndSites_v4,208.0.0.0/4,VxVault_v4,forum.netgate.com,pc,-
Jun 16 10:53:15,1770010922,igb0,LAN,block,4,6,TCP-S,192.168.1.110,208.123.73.199,55932,443,out,US,pfB_MalwareAndSites_v4,208.0.0.0/4,VxVault_v4,forum.netgate.com,pc,-

Masterfile

VxVault origin file
[0_1529140752720_VxVault_v4.orig](Uploading 100%)
VxVault_v4.orig -> https://pastebin.com/ntsNk03s

So I don't understand why it's blocking 208.0.0.0/4 and 176.0.0.0/5 if these are not in the original list and still remain after reload the ip lists. I don't know how to get rid of them. Maybe is a problem with the VXVault format. But still that doesn't explain why those IPs don't disappear and why is blocking something that is not related to any list, at least I don't understand it.
I can understand that If I update and an IP disappear in the log the old blocks will appear as "not listed ~~old_list_name~~" but I don't undestand why a non listed IP is blocking stuff, or that a "non listed ~~not listed~~" IP what ever it means (it was not listed and now is either listed) is blocking something. pfblockerng should delete not listed IPs so this doesn't block anything anymore and I think this is what is failing here.
How I can fix it? It's the 3rd or 4th time that I have had this problem with pfblockerng since I started to use it (3 months ago) and the only way to fix it what to start from scracth.

The only thing I can think that I did wrong maybe was this "** AVOID ** Running these "Force" options - when CRON is expected to RUN!" so maybe I run and update while cron was working, I don't know for sure but I can't discard it since when I whitelist something I usually run the Update task. Could this be the root cause of my issue? maybe I'm that stupid and I did the same mistake 3 times xD

On the other hand the widget packet count doesn't work very well

and then if I click in attackv4

Thanks

BBcan177

Run these commands to see where these IPs are listed:

grep "\.0\.0\.0" /var/db/pfblockerng/deny/*
grep "\.0\.0\.0" /var/db/pfblockerng/original/*
grep "\.0\.0\.0" /var/db/aliastables/*

Do you have any entries defined in this Alias "Customlist"?

For the Widget pivot to the Alerts Tab. Run this command to see what entries are in the ip_block.log:

grep "pfB_Attack_v4" /var/log/pfblockerng/ip_block.log

It could be that the ip_block.log is rotating the max. line count and clearing out those entries. You can increase these log line limits in the General Tab.

l0rdraiden

@bbcan17 said in pfblockerng error: Unknown Not listed!:

Run these commands to see where these IPs are listed:
grep "\.0\.0\.0" /var/db/pfblockerng/deny/*
grep "\.0\.0\.0" /var/db/pfblockerng/original/*
grep "\.0\.0\.0" /var/db/aliastables/*
Do you have any entries defined in this Alias "Customlist"?

For the Widget pivot to the Alerts Tab. Run this command to see what entries are in the ip_block.log:
grep "pfB_Attack_v4" /var/log/pfblockerng/ip_block.log
It could be that the ip_block.log is rotating the max. line count and clearing out those entries. You can increase these log line limits in the General Tab.

Shell Output - grep ".0.0.0" /var/db/pfblockerng/deny/*
/var/db/pfblockerng/deny/ET_Block_IP_v4.txt:161.0.0.0/19
/var/db/pfblockerng/deny/ET_Block_IP_v4.txt:223.0.0.0/15

Shell Output - grep ".0.0.0" /var/db/pfblockerng/original/*
/var/db/pfblockerng/original/ET_Block_IP_v4.orig:161.0.0.0/19
/var/db/pfblockerng/original/ET_Block_IP_v4.orig:223.0.0.0/15

Shell Output - grep ".0.0.0" /var/db/aliastables/*
grep: /var/db/aliastables/*: No such file or directory

https://rules.emergingthreats.net/fwrules/emerging-Block-IPs.txt

So it's a problem with this list?

What do you mean with this? Do you have any entries defined in this Alias “Customlist”?
Firewall->Aliases? yes I have defined custom ports that I'm using like this, so pfblockerng only blocks ports inbound that I have open

For the second part

Shell Output - grep "pfB_Attack_v4" /var/log/pfblockerng/ip_block.log
Jun 16 10:38:00,1770010014,igb0,LAN,block,4,6,TCP-S,192.168.1.209,196.196.193.44,48140,45278,out,IE,pfB_Attack_v4,196.196.0.0/14,ET_Block_IP_v4,Unknown,Unknown,+

I have increased the limits to 40k

BBcan177

Those IPs must have been in the feed at some point. But the grep commands are telling you that they are no longer in any feed.

I assume that pfBlockerNG is disabled, as this should not return that error if there are files in that folder:

Shell Output - grep “.0.0.0” /var/db/aliastables/*
grep: /var/db/aliastables/*: No such file or directory

What do you mean with this? Do you have any entries defined in this Alias “Customlist”?

At the bottom of each Alias is "IPv4 Custom_list" where you can manually add IPs to an Alias.

l0rdraiden

@bbcan177 said in pfblockerng error: Unknown Not listed!:

Those IPs must have been in the feed at some point. But the grep commands are telling you that they are no longer in any feed.

I assume that pfBlockerNG is disabled, as this should not return that error if there are files in that folder:

Shell Output - grep “.0.0.0” /var/db/aliastables/*
grep: /var/db/aliastables/*: No such file or directory

What do you mean with this? Do you have any entries defined in this Alias “Customlist”?

At the bottom of each Alias is "IPv4 Custom_list" where you can manually add IPs to an Alias.

Right, I enabled it and run the command again

Shell Output - grep ".0.0.0" /var/db/aliastables/*
/var/db/aliastables/pfB_Attack_v4.txt:161.0.0.0/19
/var/db/aliastables/pfB_Attack_v4.txt:223.0.0.0/15

And custom lists are all empty

it's a missconfiguration in my side or a bug? can I fix it?

BBcan177

@l0rdraiden said in pfblockerng error: Unknown Not listed!:

it’s a missconfiguration in my side or a bug? can I fix it?

Well in its current state, I can't see any Feed that has those IPs? So I don't see anything to fix either way.

If it happens again, run those commands and we can do some more debugging.

Also note that there is a new feature in the IP Alias settings > Advanced Tuneables > Suppression CIDR Limit. Here you can define a max CIDR to utilize, so that a Feed doesn't try to block a large range of IPs. YMMV

RonpfS

@l0rdraiden Why don't you remove the http://vxvault.net/ViriList.php?s=0&m=100 URL as it's not geared for IPV4

jazzl0ver

Hi,

Sorry for bumping this topic up, but can somebody explain why I get Unknown Not listed in this case:

# grep 113.1.135.78 /var/db/pfblockerng/* -r
/var/db/pfblockerng/deny/CINS_army_v4.txt:113.1.135.78
/var/db/pfblockerng/mastercat:113.1.135.78
/var/db/pfblockerng/masterfile:CINS_army_v4 113.1.135.78
/var/db/pfblockerng/original/CINS_army_v4.orig:113.1.135.78

Why if this IP is not listed, it's still getting blocked?

Is there a description of what all of those files/folders under /var/db/pfblockerng/ are intended for?

Thanks in advance!

NollipfSense

@jazzl0ver You might be better off starting a new thread and linking this thread as reference.

jazzl0ver

@NollipfSense not sure it's wise to create different threads for the same topic. It'll be harder to search things if someone face same issue.