What's up with the whitelist not working on DNSBL?


  • Banned

    What am I not understanding here?

    In this case I have the domain I want in the Custom Domain Whitelist section, saved and reloaded (been on the list for weeks).

    .icloud.com
    

    However, I'm still getting the DNSBL certificate when I visit this site.

    When I go to my DNSBL Alerts, I see the traffic being flagged, and I also see that it is correctly identified as already being on the whitelist.
    So what is happening here? DNSBL recognizes the traffic as being whitelisted but is still blackholing it? This makes no sense to me.




  • Moderator

    It might not be that domain that is the issue… You can see if the domain is still listed in DNSBL:

    grep ".icloud.com" /var/db/pfblockerng/dnsbl/*
    grep ".icloud.com" /var/unbound/pfb_dnsbl.conf

    It might be another domain, check F12 in the browser and goto Dev mode to see the console error msgs. Or review the Alerts tab logs...

    Sometimes its a CNAME that also needs to be whitelisted.


  • Banned

    I tried clearing console errors, then going to the webpage, no console errors. My alerts tab only shows one entry when I go to the page and it's the one I posted for icloud.com.

    Both of those greps returned a long list of entries.



  • Do a DNS query in pfSense.  That will return either a CNAME or an IP address.

    If the result is a CNAME then requery on that until a query returns the IP of pfBlockerNG's web server.

    Whatever domain resolves to that IP is the one you need to whitelist.

    As an aside, I have noticed many of the list authors are quite harsh on blocking MS and Apple domains, yet are quite content to allow google to be vastly more invasive in terms of privacy and tracking; I wonder if that is more of a reflection on google's wholesale monopoly abuse, or the list authors' preferences?


  • Banned

    I ran the DNS lookup, got four IP's. I re-queried each IP and none resolved 10.10.10.1.

    Is that what you meant? I'm not getting any returns for CNAME's.






  • It is what I meant - I checked it here, the problem is you're checking the wrong domain.

    HTTP queries for icloud.com issue an 301 (permanently moved) response redirecting to (drum roll) www.icloud.com - try searching on that and see how you get on?

    (Yes, I could just tell you which domain to whitelist, but I'm trying to teach you to fish here in case it happens again.)


  • Banned

    Well, doing a DNS lookup for www.icloud.com does resolve 10.10.10.1…. But I'm not seeing what good this is doing?

    In my Custom Whitelist I placed

    .icloud.com
    

    Which whitelist all subdomains of icloud.com.

    Prefix Domain with a "." to Whitelist all Sub-Domains.  IE: (.example.com)

    I tried adding

    www.icloud.com
    

    saving & force reloading.
    But it doesn't make any difference.

    DNSBL is already identifying that the site is whitelisted, but is still redirecting to a blackhole.


  • Moderator

    This command will list any CNAMES for a domain… it will use @8.8.8.8 (You can change that to any external NS server)... You don't want to use the pfSense resolver as it would return the DNSBL VIP.

    I don't see any of these domains/CNAMES in any Feed... But try to grep for those and see if they come up... grep for the start of the domain name. IE:  ".akadns.net" etc...

    drill @8.8.8.8 www.icloud.com
    

    ;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 4609
    ;; flags: qr rd ra ; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 0
    ;; QUESTION SECTION:
    ;; www.icloud.com.      IN      A

    ;; ANSWER SECTION:
    www.icloud.com. 3119    IN      CNAME  www-cdn.icloud.com.akadns.net.
    www-cdn.icloud.com.akadns.net.  73      IN      CNAME  www.icloud.com.edgekey.net.
    www.icloud.com.edgekey.net.    17250  IN      CNAME  e4478.a.akamaiedge.net.
    e4478.a.akamaiedge.net. 9      IN      A      23.15.152.140

    ;; AUTHORITY SECTION:

    ;; ADDITIONAL SECTION:

    ;; Query time: 17 msec
    ;; SERVER: 8.8.8.8
    ;; WHEN: Thu Jun 29 19:13:56 2017
    ;; MSG SIZE  rcvd: 161

    You can also run the following and if it replies back with the DNSBL IP, then its blocked… Did you try to clear your browser and /OS cache... Or reboot the LAN device?

    host -t A www.icloud.com
    

  • Banned

    I had already added

    www-cdn.icloud.com.akadns.net
    

    With no effect, I'll see if I can find any more to add, try the rest of what you mentioned and report back.

    Thanks for the help!


  • Banned

    [2.4.0-BETA][admin@netbox.network]/root: grep ".akadns.net" /var/db/pfblockerng/dnsbl/*
    /var/db/pfblockerng/dnsbl/Cameleon.txt:local-data: "adfarm.mplx.akadns.net 60 IN A 10.10.10.1"
    /var/db/pfblockerng/dnsbl/Cameleon.txt:local-data: "img.mplx.akadns.net 60 IN A 10.10.10.1"
    /var/db/pfblockerng/dnsbl/Cameleon.txt:local-data: "www.burstnet.akadns.net 60 IN A 10.10.10.1"
    /var/db/pfblockerng/dnsbl/SBh_p.txt:local-data: "sls.update.microsoft.com.akadns.net 60 IN A 10.10.10.1"
    /var/db/pfblockerng/dnsbl/SBh_p.txt:local-data: "statsfe2.update.microsoft.com.akadns.net 60 IN A 10.10.10.1"
    /var/db/pfblockerng/dnsbl/SWC.txt:local-data: "ads.adxpose.mpire.akadns.net 60 IN A 10.10.10.1"
    /var/db/pfblockerng/dnsbl/SWC.txt:local-data: "ads1.perfadbrite.com.akadns.net 60 IN A 10.10.10.1"
    /var/db/pfblockerng/dnsbl/SWC.txt:local-data: "lb1.www.ms.akadns.net 60 IN A 10.10.10.1"
    /var/db/pfblockerng/dnsbl/SWC.txt:local-data: "schemas.microsoft.akadns.net 60 IN A 10.10.10.1"
    /var/db/pfblockerng/dnsbl/SWC.txt:local-data: "track-apmebf.cj.akadns.net 60 IN A 10.10.10.1"
    /var/db/pfblockerng/dnsbl/sh2pfB_0.txt:local-data: "ads.as4x.tmcs.akadns.net 60 IN A 10.10.10.1"
    [2.4.0-BETA][admin@netbox.network]/root: grep ".akadns.net" /var/unbound/pfb_dnsbl.conf
    local-data: "adfarm.mplx.akadns.net 60 IN A 10.10.10.1"
    local-data: "img.mplx.akadns.net 60 IN A 10.10.10.1"
    local-data: "www.burstnet.akadns.net 60 IN A 10.10.10.1"
    local-data: "ads.adxpose.mpire.akadns.net 60 IN A 10.10.10.1"
    local-data: "ads1.perfadbrite.com.akadns.net 60 IN A 10.10.10.1"
    local-data: "lb1.www.ms.akadns.net 60 IN A 10.10.10.1"
    local-data: "schemas.microsoft.akadns.net 60 IN A 10.10.10.1"
    local-data: "track-apmebf.cj.akadns.net 60 IN A 10.10.10.1"
    local-data: "ads.as4x.tmcs.akadns.net 60 IN A 10.10.10.1"
    local-data: "sls.update.microsoft.com.akadns.net 60 IN A 10.10.10.1"
    local-data: "statsfe2.update.microsoft.com.akadns.net 60 IN A 10.10.10.1"
    [2.4.0-BETA][admin@netbox.network]/root: host -t A www.icloud.com
    www.icloud.com has address 10.10.10.1
    

  • Banned

    I've added all of these so far and still blackholing.

    www.icloud.com
    .www.icloud.com
    icloud.com
    .icloud.com
    www-cdn.icloud.com.akadns.net #CNAME for icloud.com
    .icloud.com.akadns.net
    .icloud.com.edgekey.net
    www.icloud.com.edgekey.net
    e4478.a.akamaiedge.net
    

  • Moderator

    Hmm I checked again and that domain is listed in hpHosts_PSH Feed… Will have to contact the maintainer of that feed.

    grep "www.icloud.com" /var/db/pfblockerng/dnsblorig/*

    /var/db/pfblockerng/dnsblorig/hpHosts_PSH.orig:127.0.0.1        www.icloud.com
    /var/db/pfblockerng/dnsblorig/hpHosts_PSH.orig:127.0.0.1        www.icloud.com-findi.top
    /var/db/pfblockerng/dnsblorig/hpHosts_PSH.orig:127.0.0.1        www.icloud.com-ios9.cc
    /var/db/pfblockerng/dnsblorig/hpHosts_PSH.orig:127.0.0.1        www.icloud.com-manage.net
    /var/db/pfblockerng/dnsblorig/hpHosts_PSH.orig:127.0.0.1        www.icloud.com-na.cc
    /var/db/pfblockerng/dnsblorig/hpHosts_PSH.orig:127.0.0.1        www.icloud.com.21.0x7.pn.ci.fmip-12.in
    /var/db/pfblockerng/dnsblorig/hpHosts_PSH.orig:127.0.0.1        www.icloud.com.ht
    /var/db/pfblockerng/dnsblorig/hpHosts_PSH.orig:127.0.0.1        www.icloud.com.iphonc.win
    /var/db/pfblockerng/dnsblorig/hpHosts_PSH.orig:127.0.0.1        www.icloud.com.reported.me
    

    I used the Alerts Tab to whitelist "www.icloud.com" and it automatically added the following to the DNSBL Whitelist:

    .www.icloud.com
    .www-cdn.icloud.com.akadns.net # CNAME for (www.icloud.com)
    .www.icloud.com.edgekey.net # CNAME for (www.icloud.com)
    .e4478.a.akamaiedge.net # CNAME for (www.icloud.com)
    

    It was blocked before whitelisting it but now replies back with:

    host -t A www.icloud.com

    www.icloud.com is an alias for www-cdn.icloud.com.akadns.net.
    www-cdn.icloud.com.akadns.net is an alias for www.icloud.com.edgekey.net.
    www.icloud.com.edgekey.net is an alias for e4478.a.akamaiedge.net.
    e4478.a.akamaiedge.net has address 173.222.186.46

    Remove all those Whitelist entries that you manually added. Then browse to www.icloud.com, then whitelist it from the Alerts tab and see how that goes…


  • Banned

    Remove all those Whitelist entries that you manually added. Then browse to www.icloud.com, then whitelist it from the Alerts tab and see how that goes…

    Perfect, that did it!