TLD white list not working

wolfsden3

The TLD white list is supposed to be used in conjunction with the blacklist according to the "i" for information in that section ( Firewall > pfBlockerNG > DNSBL). That means, at least the way I interpret it to be, that if you have a ".cloud" TLD for example on the black list but have a host name you need to reach like: esa04mx1.3zden.cloud

...that it should ALLOW that esa04mx1.3zden.cloud host name. This however doesn't seem to be the case.

ping esa04mx1.3zden.cloud

Pinging esa04mx1.3zden.cloud [10.10.10.1] with 32 bytes of data:
Reply from 10.10.10.1: bytes=32 time<1ms TTL=64
Reply from 10.10.10.1: bytes=32 time<1ms TTL=64

In fact it's blocking it on the interface. How can I get this feature to work? It doesn't seem to be.

It's been in the white list for ages.

2.4.4-RELEASE-p3 (amd64)
built on Thu May 16 06:01:19 EDT 2019
FreeBSD 11.2-RELEASE-p10

pfBlockerNG-devel		net 	2.2.5_29

wolfsden3

My update / edit is getting flagged incorrectly by Akismet...FFS!

Assembling DNSBL database... completed [ 03/04/20 08:59:58 ]
TLD:

 TLD Whitelist - Missing data | esa04mx1.3zden.cloud | No IP found! |

Why is it saying "No IP found!"!!!???

Gertjan

For me, esa04mx1.3zden.cloud is 104.218.109.10
For you, it's some RFC 1918, 10.10.10.1

There is something you didn't tell us yet ;)

wolfsden3

No - as I said. If you put "cloud" in the TLD black list > then reload your DNSBL. If you flush your DNS cache on any other DNS servers you use then also flush it on your local PC you should get 10.10.10.1.

My setup is:

PC > Local DNS Server (windows) > PFSense DNS > OpenDNS

So once I reload the DNSBL putting a # in front of cloud like: # cloud <-- that comments it out of the list...then I reload DNSBL > flush DNS on the windows DNS server > flush DNS on the local machine.

Then I ping I get:

Pinging esa04mx1.3zden.cloud [104.218.109.10] with 32 bytes of data:
Reply from 104.218.109.10: bytes=32 time=48ms TTL=49
Reply from 104.218.109.10: bytes=32 time=47ms TTL=49

It's only AFTER comment it out and take it out of the list that it works.

It won't work if you have "cloud" in the TLD blacklist + esa04mx1.3zden.cloud in the TLD white list.

I think your setup is different than mine.

Gertjan

@Gertjan said in TLD white list not working:

For you, it's some RFC 1918, 10.10.10.1
There is something you didn't tell us yet ;)

Probably because :

I'm sorry, I wasn't thinking.

wolfsden3

@Gertjan Yes - that's the way it's supposed to work. That's the "block interface" of PFBlockerNG and from my memory...always has been.

That's how it "blocks", it points everything to a non-routed IP address, the PFBlockerNG web site. That's the website that shows you something is blocked and why.

That's WORKING you see but what's NOT working is the TLD White list :-)

A Former User

@wolfsden3 said in TLD white list not working:

@Gertjan Yes - that's the way it's supposed to work. That's the "block interface" of PFBlockerNG and from my memory...always has been.

That's how it "blocks", it points everything to a non-routed IP address, the PFBlockerNG web site. That's the website that shows you something is blocked and why.

That's WORKING you see but what's NOT working is the TLD White list :-)

clear dnsbl and ip counters, update|reload pfblockerng and dnsbl test again? if the proper ip to be queried is already being blocked, it won't get returned to your box to update cache (from what i've experienced in testing at least in regards to whitelisting in DNSBL at least)

community edition 2.4.5RC pfblockerng-devel latest version

wolfsden3

@sparkyMcpenguin What do you mean by "clear dnsbl and ip counters, update|reload pfblockerng and dnsbl test again?"

Do you mean: Firewall > PFBlockerNG > Update > Reload > All, IP or DNSBL...?

...or do you mean something else?

I still don't know why it's telling me when reloading that NONE of the host names on the TLD white list can be found (their IP).

Assembling DNSBL database... completed [ 03/01/20 10:02:17 ]
TLD:
Error: error sending query: Error creating socket

 TLD Whitelist - Missing data | esa04mx1.3zden.cloud | No IP found! |
Error: error sending query: Error creating socket

I think I tried to paste that in and their dumb Akismet spam posting plugin squashed it.

A Former User

eroor sending cause blocked in dnsbl.

firewall > pfblockerng > update all> reload all = correct.

for the 'counters' i mentioned, i usually clear them with the pfblocker widget on the dashboard. its a little button in the column showing 'packets' at least for me

A Former User

@sparkyMcpenguin said in TLD white list not working:

eroor sending cause blocked in dnsbl.

firewall > pfblockerng > update all> reload all = correct.

for the 'counters' i mentioned, i usually clear them with the pfblocker widget on the dashboard. its a little button in the column showing 'packets' at least for me

i removed the picture here so it did not duplicate

i also, in the widget, have 'clear ip and dnsbl counters' set to one day, not never (this gave me so many headaches)

wolfsden3

@sparkyMcpenguin said in TLD white list not working:

'packets' at

OK great! I didn't know that in the widget this was a thing. I had 98 million unbound queries since the last clear LOL! I have a lot of DNSBL's.

Mine had a trash can next to the word "packets" and once I clicked that it wiped the count out. I also clicked the gear and the pop under, under the table (widget) showed up where you can configure it to clear the count daily. I did that now too.

Cool!

I'm reloading both IP and DNSbl lists now. It takes like 10 minutes :P

wolfsden3

You know what...

I think this is an "error" but not an "error" LOL.

Assembling DNSBL database... completed [ 03/04/20 10:45:56 ]
TLD:

 TLD Whitelist - Missing data | esa04mx1.3zden.cloud | No IP found! |

Then the "i" (information bubble says):

Enter each specific Domain and/or Sub-Domains to be Whitelisted. (Used in conjunction with TLD Blacklist only) 
Enter one  Domain per line
Examples:

    example.com
    example.com|x.x.x.x  (Replace x.x.x.x with associated Domain/Sub-Domain IP Address.

The First option above will collect the IP Address on each Cron run, while the second option will define a Static IP Address.

You must Whitelist every Domain or Sub-Domain individually.
No Regex Entries and no leading/trailing 'dot' allowed!
You may use "#" after any Domain/Sub-Domain to add comments. IE: (example.com|x.x.x.x # TLD Whitelist)
This List is stored as 'Base64' format in the config.xml file.

Notice: "The First option above will collect the IP Address on each Cron run, while the second option will define a Static IP Address."

So...I don't think it can FIND the IP address. It says that if I just list a domain like "example.com" with no pipe + IP address that it will go find the IP.

I think this is broken and that BBCan will need to fix it. It's a bug. It's not finding the IP address and therefore still points it at the block interface of 10.10.10.1.

I'll email him a link to this discussion and perhaps he can chime in.

A Former User

@wolfsden3 said in TLD white list not working:

You know what...

I think this is an "error" but not an "error" LOL.
Assembling DNSBL database... completed [ 03/04/20 10:45:56 ]
TLD:

 TLD Whitelist - Missing data | esa04mx1.3zden.cloud | No IP found! |
Then the "i" (information bubble says):
Enter each specific Domain and/or Sub-Domains to be Whitelisted. (Used in conjunction with TLD Blacklist only) 
Enter one  Domain per line
Examples:

    example.com
    example.com|x.x.x.x  (Replace x.x.x.x with associated Domain/Sub-Domain IP Address.

The First option above will collect the IP Address on each Cron run, while the second option will define a Static IP Address.

You must Whitelist every Domain or Sub-Domain individually.
No Regex Entries and no leading/trailing 'dot' allowed!
You may use "#" after any Domain/Sub-Domain to add comments. IE: (example.com|x.x.x.x # TLD Whitelist)
This List is stored as 'Base64' format in the config.xml file.
Notice: "The First option above will collect the IP Address on each Cron run, while the second option will define a Static IP Address."

So...I don't think it can FIND the IP address. It says that if I just list a domain like "example.com" with no pipe + IP address that it will go find the IP.

I think this is broken and that BBCan will need to fix it. It's a bug. It's not finding the IP address and therefore still points it at the block interface of 10.10.10.1.

I'll email him a link to this discussion and perhaps he can chime in.

sure thing. i only mention the steps ive taken to diagnose similar issues. the ordering of rules loaded can sometimes affect this as well.

but yea this puzzles me a little too, if clearing the packet counter after a force update and reload didn't work... maybe the rules order is incorrect? i've had that cause the loaded rules drop from tens of thousands down to a couple hundred just because my ordering was incorrect on the ipv4 ip page in pfblockerng and the lists under dnsbl. pictures below:

this is how i have it now, accidentally had some pass lists below block lists before (oops) because of the order i added feeds in (and didn't change order afterwards)

A Former User

in regards to whitelisting i tend to just use the main whitelist in DNSBL, not the lower down TLD white/blacklisting, and just manually tend to it. my network is a garden ya dig?

A Former User

@sparkyMcpenguin said in TLD white list not working:

You must Whitelist every Domain or Sub-Domain individually.
No Regex Entries and no leading/trailing 'dot' allowed!
You may use "#" after any Domain/Sub-Domain to add comments. IE: (example.com|x.x.x.x # TLD Whitelist)
This List is stored as 'Base64' format in the config.xml file.

"You must Whitelist every Domain or Sub-Domain individually.
No Regex Entries and no leading/trailing 'dot' allowed!
You may use "#" after any Domain/Sub-Domain to add comments. IE: (example.com|x.x.x.x # TLD Whitelist)
This List is stored as 'Base64' format in the config.xml file."

what does your white/blacklisting in TLD look like if you can show without too much being displayed?

A Former User

@wolfsden3 said in TLD white list not working:

@sparkyMcpenguin said in TLD white list not working:

'packets' at

OK great! I didn't know that in the widget this was a thing. I had 98 million unbound queries since the last clear LOL! I have a lot of DNSBL's.

Mine had a trash can next to the word "packets" and once I clicked that it wiped the count out. I also clicked the gear and the pop under, under the table (widget) showed up where you can configure it to clear the count daily. I did that now too.

Cool!

I'm reloading both IP and DNSbl lists now. It takes like 10 minutes :P

i just now saw this 98 MILLION? wowzers scoob

A Former User

the ordering of the lists really affected me one day with maxmind downloads.
a newly added blocklist i had added had dnsbl's default ip listed.
needless to say it broke more than just maxmind downloads as i then had problems with DNSBL and other things (why that ip is on blocklist i dont know or really care too much, but on the blocklist it was listed as maxmind not dnsbl - to me sounded like a bad cached entry from the blocklist, which i confirmed under the 'reports' page in pfblockerng). disabling that list, doing a packet clear force update reload resolved this

wolfsden3

Well thanks for the discussion, I learned a few things that I'll implement at other locations. Looks like they have 760k DNS queries per day on that FW. I'm not sure if that's a lot or not.

Minimizing DNS queries is my next project although the FW is doing it's job and fairly well I think.

I'll fart around with this. I'm not sure if other sites are experiencing this too. They might very well be.

Thanks again.

A Former User

@wolfsden3 said in TLD white list not working:

Well thanks for the discussion, I learned a few things that I'll implement at other locations. Looks like they have 760k DNS queries per day on that FW. I'm not sure if that's a lot or not.

Minimizing DNS queries is my next project although the FW is doing it's job and fairly well I think.

I'll fart around with this. I'm not sure if other sites are experiencing this too. They might very well be.

Thanks again.

in just my home network with maybe 5 devices that really put out data, ive seen 600K before, 700 doesn't seem much of a stretch over the course of a week ish (edit: depending on number of clients this is a factor i dont know lol)

A Former User

@wolfsden3 said in TLD white list not working:

Well thanks for the discussion, I learned a few things that I'll implement at other locations. Looks like they have 760k DNS queries per day on that FW. I'm not sure if that's a lot or not.

Minimizing DNS queries is my next project although the FW is doing it's job and fairly well I think.

I'll fart around with this. I'm not sure if other sites are experiencing this too. They might very well be.

Thanks again.

last thing i promise.

below i have screenshot and posted my firewall rules:

Floating:

WAN:

LAN:

GUESTVLAN:

blacked out information is just rules for my openvpn