2 Questions: Whitelist and UT1

[Stewart]

I'm running the latest pfBlockerNG-devel and pfSense.

Question 1: I feel like I've asked this question before but not sure and can't find it if I did. I'm trying to whitelist certain domains and IPs so that pfBlocker doesn't scoop them up and block them. When I attempt to create the whitelist I get the warning of:

I'm sure it's on purpose and is stopping me from doing something stupid. I'm not looking to specifically whitelist these sites in the pfSense firewall, I just don't want them blocked by pfBlocker.

Question 2: I'm trying to use DNSBL categories from both Shallalist and UT1 but when I try to update the list I get:

===[  DNSBL Process  ]================================================

 Loading DNSBL Statistics... completed
 Loading DNSBL Whitelist... completed

Downloading Blacklist Database(s) [ ut1 (~8.5MB) ] ... Please wait ...
	UT1 ... Failed

[ UT1_bitcoin ]			 Downloading update [ 02/19/20 17:10:38 ] .
[ UT1_bitcoin ] file_get_contents(/var/db/pfblockerng/ut1/ut1_bitcoin): failed to open stream: No such file or directory


 [ DNSBL_UT1 - UT1_bitcoin ] Download FAIL
   Local File Failure

[ UT1_dangerous_material ]	 Downloading update .
[ UT1_dangerous_material ] file_get_contents(/var/db/pfblockerng/ut1/ut1_dangerous_material): failed to open stream: No such file or directory


 [ DNSBL_UT1 - UT1_dangerous_material ] Download FAIL
   Local File Failure

[ UT1_ddos ]			 Downloading update .
[ UT1_ddos ] file_get_contents(/var/db/pfblockerng/ut1/ut1_ddos): failed to open stream: No such file or directory


 [ DNSBL_UT1 - UT1_ddos ] Download FAIL
   Local File Failure

[ UT1_dialer ]			 Downloading update .
[ UT1_dialer ] file_get_contents(/var/db/pfblockerng/ut1/ut1_dialer): failed to open stream: No such file or directory


 [ DNSBL_UT1 - UT1_dialer ] Download FAIL
   Local File Failure

[ UT1_drogue ]			 Downloading update .
[ UT1_drogue ] file_get_contents(/var/db/pfblockerng/ut1/ut1_drogue): failed to open stream: No such file or directory


 [ DNSBL_UT1 - UT1_drogue ] Download FAIL
   Local File Failure

[ UT1_gambling ]		 Downloading update .
[ UT1_gambling ] file_get_contents(/var/db/pfblockerng/ut1/ut1_gambling): failed to open stream: No such file or directory


 [ DNSBL_UT1 - UT1_gambling ] Download FAIL
   Local File Failure

[ UT1_hacking ]			 Downloading update .
[ UT1_hacking ] file_get_contents(/var/db/pfblockerng/ut1/ut1_hacking): failed to open stream: No such file or directory


 [ DNSBL_UT1 - UT1_hacking ] Download FAIL
   Local File Failure

[ UT1_malware ]			 Downloading update .
[ UT1_malware ] file_get_contents(/var/db/pfblockerng/ut1/ut1_malware): failed to open stream: No such file or directory


 [ DNSBL_UT1 - UT1_malware ] Download FAIL
   Local File Failure

[ UT1_phishing ]		 Downloading update .
[ UT1_phishing ] file_get_contents(/var/db/pfblockerng/ut1/ut1_phishing): failed to open stream: No such file or directory


 [ DNSBL_UT1 - UT1_phishing ] Download FAIL
   Local File Failure

[ UT1_reaffected ]		 Downloading update .
[ UT1_reaffected ] file_get_contents(/var/db/pfblockerng/ut1/ut1_reaffected): failed to open stream: No such file or directory


 [ DNSBL_UT1 - UT1_reaffected ] Download FAIL
   Local File Failure

[ UT1_warez ]			 Downloading update .
[ UT1_warez ] file_get_contents(/var/db/pfblockerng/ut1/ut1_warez): failed to open stream: No such file or directory


 [ DNSBL_UT1 - UT1_warez ] Download FAIL
   Local File Failure

My totals look like this:

===[ DNSBL Domain/IP Counts ] ===================================

  169034 total
   31560 /var/db/pfblockerng/dnsbl/AntiSocial_BD.txt
   26277 /var/db/pfblockerng/dnsbl/MDS.txt
   18128 /var/db/pfblockerng/dnsbl/Shallalist_spyware.txt
   14510 /var/db/pfblockerng/dnsbl/Shallalist_gamble.txt
   10951 /var/db/pfblockerng/dnsbl/Shallalist_drugs.txt
    9506 /var/db/pfblockerng/dnsbl/Cameleon.txt
    9209 /var/db/pfblockerng/dnsbl/SFS_Toxic_BD.txt
    8483 /var/db/pfblockerng/dnsbl/hpHosts_ATS.txt
    6693 /var/db/pfblockerng/dnsbl/SWC.txt
    6455 /var/db/pfblockerng/dnsbl/Spam404.txt
    5697 /var/db/pfblockerng/dnsbl/Adaway.txt
    2539 /var/db/pfblockerng/dnsbl/MDS_Immortal.txt
    2399 /var/db/pfblockerng/dnsbl/EasyPrivacy.txt
    2285 /var/db/pfblockerng/dnsbl/ISC_SDH.txt
    2006 /var/db/pfblockerng/dnsbl/Shallalist_warez.txt
    1459 /var/db/pfblockerng/dnsbl/D_Me_ADs.txt
    1413 /var/db/pfblockerng/dnsbl/Shallalist_costtraps.txt
    1335 /var/db/pfblockerng/dnsbl/Shallalist_spyware_v4.ip
    1085 /var/db/pfblockerng/dnsbl/Shallalist_fortunetelling.txt
    1052 /var/db/pfblockerng/dnsbl/EasyList.txt
    1050 /var/db/pfblockerng/dnsbl/MDL.txt
    1027 /var/db/pfblockerng/dnsbl/Yoyo.txt
    1017 /var/db/pfblockerng/dnsbl/Shallalist_tracker.txt
     597 /var/db/pfblockerng/dnsbl/Shallalist_hacking.txt
     547 /var/db/pfblockerng/dnsbl/Shallalist_drugs_v4.ip
     450 /var/db/pfblockerng/dnsbl/MVPS.txt
     450 /var/db/pfblockerng/dnsbl/BBC_DC2.txt
     308 /var/db/pfblockerng/dnsbl/Shallalist_hacking_v4.ip
     269 /var/db/pfblockerng/dnsbl/Shallalist_warez_v4.ip
     179 /var/db/pfblockerng/dnsbl/Shallalist_violence.txt
      42 /var/db/pfblockerng/dnsbl/Shallalist_gamble_v4.ip
      22 /var/db/pfblockerng/dnsbl/Shallalist_violence_v4.ip
      15 /var/db/pfblockerng/dnsbl/D_Me_Tracking.txt
      11 /var/db/pfblockerng/dnsbl/Shallalist_tracker_v4.ip
       5 /var/db/pfblockerng/dnsbl/EasyList_v4.ip
       1 /var/db/pfblockerng/dnsbl/Shallalist_costtraps_v4.ip
       1 /var/db/pfblockerng/dnsbl/EasyPrivacy_v4.ip
       1 /var/db/pfblockerng/dnsbl/D_Me_Malv.txt
       0 /var/db/pfblockerng/dnsbl/UT1_warez.fail
       0 /var/db/pfblockerng/dnsbl/UT1_reaffected.fail
       0 /var/db/pfblockerng/dnsbl/UT1_phishing.fail
       0 /var/db/pfblockerng/dnsbl/UT1_malware.fail
       0 /var/db/pfblockerng/dnsbl/UT1_hacking.fail
       0 /var/db/pfblockerng/dnsbl/UT1_gambling.fail
       0 /var/db/pfblockerng/dnsbl/UT1_drogue.fail
       0 /var/db/pfblockerng/dnsbl/UT1_dialer.fail
       0 /var/db/pfblockerng/dnsbl/UT1_ddos.fail
       0 /var/db/pfblockerng/dnsbl/UT1_dangerous_material.fail
       0 /var/db/pfblockerng/dnsbl/UT1_bitcoin.fail
       0 /var/db/pfblockerng/dnsbl/D_Me_Malw.txt
       0 /var/db/pfblockerng/dnsbl/Abuse_URLBL.txt
       0 /var/db/pfblockerng/dnsbl/Abuse_DOMBL.txt

Looks like Shallalist is working but UT1 isn't. Is there a problem with UT1 or are we unable to cross use the 2 lists?

Thanks for the help!
Edit: for clarity

[NollipfSense]

@Stewart Add the white list here Firewall>pfBlockerNG-DNSBL...the shallalist and the UT1 site might just be down...give it a day or two.

[Stewart]

Thanks. That takes care of DNS blocks. What about IP blocks? How do I whitelist those? Is it the IPv4 Suppression section?

Also, UT1 is still not downloading for me. Is it up for anyone else?

[NollipfSense]

@Stewart said in 2 Questions: Whitelist and UT1:

Thanks. That takes care of DNS blocks. What about IP blocks? How do I whitelist those? Is it the IPv4 Suppression section?

Also, UT1 is still not downloading for me. Is it up for anyone else?

That's correct on the IPv4...remember to force reload after you saved. I don't use the UT1 feed so I cannot say...give it a little more time.

[Stewart]

Well, the DNSBL_Whitelist didn't work for me. Had a site trying to get to na2.docusign.net which is blocked by the Malicious > Antisocial list. We put in na2.docusign.net and docusign.net into the DNSBL_Whitelist area and went to Update > Update > Run but the sites were still blocked. We disabled the antisocial list for now and ran the update and it's working but is it required to run a reload as opposed to an update to get the whitelist updates to take effect?

[NollipfSense]

@Stewart said in 2 Questions: Whitelist and UT1:

but is it required to run a reload as opposed to an update to get the whitelist updates to take effect?

That's what I said... (remember to force reload after you saved.)...else you just updating the same blocked database with no change. Here, I chose all; however, you can reload just the DNSBL.

[Stewart]

@NollipfSense Thanks. I was just confirming. They needed it up and running right away which is why I shut it off before coming back to check. I'll be switching it back in a couple of hours and wanted to make sure of the process before I do. Looking at that list, though, I'm not sure if I even want it. It blocks OneDrive (and by extension O365 Sharepoint) as well as Docusign. It seems like those are pretty important to many business workloads.

[RonpfS]

@NollipfSense Your Whitlelist should have only domain names, no URLs or http://

[NollipfSense]

@RonpfS said in 2 Questions: Whitelist and UT1:

@NollipfSense Your Whitlelist should have only domain names, no URLs or http://

That's what I have...see second post...WAIT, I see the mistake...thanks!