Whois domains some time not parsing [Resolved]

dragoangel

Can somebody help, want to block outbound to domains by creating ipv4 and ipv6 lists, already used this not one time, but sometime have errors:


cat /var/db/pfblockerng/original/coinhive.orig
### Domain: coinhive.com ###
94.130.129.239
94.130.102.124
94.130.90.167
94.130.128.151
78.46.102.214
94.130.129.243
94.130.90.154
94.130.129.235
94.130.90.152
94.130.128.243

cat /var/db/pfblockerng/deny/coinhive_com.txt
1.1.1.1

But at same time working:


cat /var/db/pfblockerng/original/ppoi_org.orig
### Domain: ppoi.org ###
104.18.43.108
104.18.42.108

cat /var/db/pfblockerng/deny/ppoi_org.txt
104.18.42.108
104.18.43.108

One question why? Why some whois records works and some not? I'm confused :-\

dragoangel

No info? :-X Nobody know to troubleshoot this? I've already created DNSBL list with custom entries and it work good, but really interesting why some list of IP's parsed OK, and some not, whois method on firewall is so good.

BBcan177

I would assume the you have Deduplication enabled, so it won't add IPs that are in another Feed, or are in a CIDR that is already in the Feeds.

grep "^94\.130\." /var/db/pfblockerng/deny/*

dragoangel

Tnx for reply, i'll check tomorrow

dragoangel

Yep, like you said.


grep "^94\.130\." /var/db/pfblockerng/deny/*

/var/db/pfblockerng/deny/blocklist.txt:94.130.127.188
/var/db/pfblockerng/deny/blocklist.txt:94.130.175.54
/var/db/pfblockerng/deny/blocklist.txt:94.130.87.22
/var/db/pfblockerng/deny/dshield_1k.txt:94.130.9.56
/var/db/pfblockerng/deny/et_compromised.txt:94.130.150.118
/var/db/pfblockerng/deny/et_compromised.txt:94.130.162.146
/var/db/pfblockerng/deny/et_compromised.txt:94.130.173.202

But they from other 94.130.* segment, it because of CIDR aggregation + deduplication, or this deduplication work like this?
What do you recommend? Better then disable deduplication (and CIDR aggregation to?) for knowing that all going on like needed, especially I use "Alias Native" for use them at custom rules, if them have duplicates it mean that some lists can be particularly not full? And tnx for help :D

BBcan177

Just noticed that you are not comparing the same Feeds here:

cat /var/db/pfblockerng/original/coinhive.orig
cat /var/db/pfblockerng/deny/coinhive_com.txt

Deduplication is doing it job… So when you use "Alias Native" there is no deduplication... So always depends what you want to achieve...

dragoangel

Thank you for clean answer. About not same feeds, yes it only mistake of copy-paste, this list was not alias native, but I asked about alias native in clarifying question. :)