PfBlockerNG v2.0 w/DNSBL

rattis · Apr 19, 2016, 4:20 AM

@rattis:

Like this plugin, first and only one I'm using so far but I'm having a problem.

Some of the feeds block properly, while some do not. At first I thought this was just Easylist, but I just found another.

p.s. BBcan177 I was the guy on twitter talking about it on Sunday April 17th

Hey… welcome!

If you ping those domains from the pfSense box do they all resolve to the DNSBL VIP address?

Either your desktop has other DNS servers defined or the browser is bypassing the DNS server... what browser are you using? Ensure that the desktops only have pfSense set as its DNS Server. Might also need to exit and reopen the browser...

Flush the Browser DNS cache, and also flush the OS cache... Another option might be to create a Reject rule in the LAN interface to block any outgoing port 53 other than from the pfSense Box itself...

already have the re-direct rule in place, it's been catching my chromecasts. (learned that by reading this and other threads / reddit, which was an awesome trick to learn btw).

Only working DNS server is the pfSense Box (cygwin):

chrisj@bikatol ~
$ ipconfig /all | grep -i "DNS Servers"
   DNS Servers . . . . . . . . . . . : fec0:0:0:ffff::1%1
   DNS Servers . . . . . . . . . . . : fec0:0:0:ffff::1%1
   DNS Servers . . . . . . . . . . . : 192.168.1.254

The 2 ipv6 ones are the equiv of the 169 addresses Microsoft used to use for ipv4. Even when I ran the windows hotfix to disable ipv6, the problem remained.

Testing in Cygwin for NSlookup and IE for the sites (because Chrome and Fire Fox have ad blockers already). Also using Chrome on my Android phone.

Ping from pfSense gets the 10.10.10.1 addresses like expected. However even after flushing DNS again the clients still go to the actual ip addresses.

[2.3-RELEASE][admin@pfSense.localdomain]/root: ping 188server.com
PING 188server.com (10.10.10.1): 56 data bytes
64 bytes from 10.10.10.1: icmp_seq=0 ttl=64 time=0.169 ms
64 bytes from 10.10.10.1: icmp_seq=1 ttl=64 time=0.115 ms
64 bytes from 10.10.10.1: icmp_seq=2 ttl=64 time=0.149 ms
^C
--- 188server.com ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.115/0.144/0.169/0.022 ms
[2.3-RELEASE][admin@pfSense.localdomain]/root:

Same from windows client with flush:

C:\Users\chrisj>ipconfig /flushdns

Windows IP Configuration

Successfully flushed the DNS Resolver Cache.

C:\Users\chrisj>ping 188server.com

Pinging 188server.com [69.162.80.55] with 32 bytes of data:
Reply from 69.162.80.55: bytes=32 time=47ms TTL=50

BBcan177 · Apr 19, 2016, 4:30 AM

@rattis:

Ping from pfSense gets the 10.10.10.1 addresses like expected. However even after flushing DNS again the clients still go to the actual ip addresses.

[2.3-RELEASE][admin@pfSense.localdomain]/root: ping 188server[.]com
PING 188server[.]com (10.10.10.1): 56 data bytes
64 bytes from 10.10.10.1: icmp_seq=0 ttl=64 time=0.169 ms
64 bytes from 10.10.10.1: icmp_seq=1 ttl=64 time=0.115 ms
64 bytes from 10.10.10.1: icmp_seq=2 ttl=64 time=0.149 ms
^C
--- 188server.com ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.115/0.144/0.169/0.022 ms
[2.3-RELEASE][admin@pfSense.localdomain]/root:

Same from windows client with flush:

C:\Users\chrisj>ipconfig /flushdns

Windows IP Configuration

Successfully flushed the DNS Resolver Cache.

C:\Users\chrisj>ping 188server[.]com

Pinging 188server.com [69.162.80.55] with 32 bytes of data:
Reply from 69.162.80.55: bytes=32 time=47ms TTL=50

Ya you can see that the pfSense has no issue in resolving that domain to 10.10.10.1

What does a tracert 188server[.]com (kinda dangerous to post those domains as-is :) )

What happens if you clear out those two IPv6 DNS settings?

BBcan177 · Apr 19, 2016, 6:12 PM

The following PR #107 for pfBlockerNG has been merged for pfSense 2.3 only:

pfBlockerNG v2.0.10 - Changelog:

Cosmetic Improvements to the Widget.
Add "Gateway Groups" to "Custom Gateway" Advanced In/Outbound Firewall customizations.
Advanced Outbound Firewall Rule customizations:
- Switched "Custom Port" option from "Source" to "Destination" port.
- Fix issue with "Custom Protocol".
Added a patch to fix an issue with 4 missing Countries in the MaxMind Database.
MaxMind is aware of the issue. (Countries = Africa - SS, N.America - BQ,CW,SX)
Other under-the-hood improvements.

(Note - Double check all of your firewall Rules after this update!)

rattis · Apr 20, 2016, 2:31 AM

Good news, and I gives me something to look at / test. I just tried ping from my android phone, and did an nslookup on my old netbook.

both are reporting 10.10.10.1 for 188server[.]com.

After more testing on my primary system, trying different browsers (I have 2 versions of Firefox installed, 1 chrome, 1 ie, 1 edge) I got the same result regardless of what was turned on.

Turns out my problem was my AV software. Avast has an option called SecureDNS which prevents dns "hijacking" so the address your going to is really the address Avast has in their DNS Servers. It's done over a tunnel so it was never caught by Wireshark or others.

trumee · Apr 20, 2016, 2:55 AM

@BBcan177:

Hi robwalker5561,

Run these commands and it will show if there is a loopback address (Won't be the first time that IBlock does this :) ):
grep "127\.0\.0\." /var/db/pfblockerng/deny/*
grep "127\.0\.0\." /var/db/pfblockerng/dnsbl/*
grep "127\.0\.0\." /var/db/aliastables/*
If it finds a loopback, then enable "Suppression" in the General Tab, and run a "Force Reload" to clear it out…

I am get a loopback address in a list but there is now Suppress option next to the ip address. How do i suppress this rule?

BBcan177 · Apr 20, 2016, 3:09 AM

It shows "no match" in the list column, so that IP is not currently in any list. Enable "Suppression" in the General Tab, and run a "Force Reload". If it's not repeating then just ignore.

As a note, when you enable Suppression, it will also filter out any RFC1918 and loopback addresses along with the whitelisted IPs in the pfBlockerNGSuppress Alias.

docjay · Apr 20, 2016, 4:01 AM

BBCAN,

Hey- thanks for all of your hard work with this package! I'm setting up a 2nd PFsense box for a client and I'm wanting to copy over my 'pfblockerNG' config. Is there an easy way to do it other than doing it all by hand?

thanks

Jamie

Abhishek · Apr 20, 2016, 4:39 AM

Hello , i find issue opening somewebsites , ie when i try to open it takes long time when pfblockerNG is enabled

i uploaded a Picture of tata docomo site , showing slow load time and ping showing some connections are redirected to 10.x.x.x but its waiting there

when this happens do i need to manually whitelist all those domains

note sure if this crash is related to above error but


					Crash report begins.  Anonymous machine information:

amd64
10.3-RELEASE
FreeBSD 10.3-RELEASE #6 05adf0a(RELENG_2_3_0): Mon Apr 11 18:52:07 CDT 2016     root@ce23-amd64-builder:/builder/pfsense-230/tmp/obj/builder/pfsense-230/tmp/FreeBSD-src/sys/pfSense

Crash report details:

PHP Errors:
[20-Apr-2016 09:34:31 Asia/Kolkata] PHP Stack trace:
[20-Apr-2016 09:34:31 Asia/Kolkata] PHP   1\. {main}() /usr/local/www/pfblockerng/pfblockerng_update.php:0
[20-Apr-2016 09:34:31 Asia/Kolkata] PHP   2\. pfb_cron_update() /usr/local/www/pfblockerng/pfblockerng_update.php:425
[20-Apr-2016 09:34:31 Asia/Kolkata] PHP   3\. header() /usr/local/www/pfblockerng/pfblockerng_update.php:90
[20-Apr-2016 09:36:02 Asia/Kolkata] PHP Stack trace:
[20-Apr-2016 09:36:02 Asia/Kolkata] PHP   1\. {main}() /usr/local/www/pfblockerng/pfblockerng_update.php:0
[20-Apr-2016 09:36:02 Asia/Kolkata] PHP   2\. pfb_cron_update() /usr/local/www/pfblockerng/pfblockerng_update.php:425
[20-Apr-2016 09:36:02 Asia/Kolkata] PHP   3\. header() /usr/local/www/pfblockerng/pfblockerng_update.php:90
[20-Apr-2016 09:37:35 Asia/Kolkata] PHP Stack trace:
[20-Apr-2016 09:37:35 Asia/Kolkata] PHP   1\. {main}() /usr/local/www/pfblockerng/pfblockerng_update.php:0
[20-Apr-2016 09:37:35 Asia/Kolkata] PHP   2\. pfb_cron_update() /usr/local/www/pfblockerng/pfblockerng_update.php:419
[20-Apr-2016 09:37:35 Asia/Kolkata] PHP   3\. header() /usr/local/www/pfblockerng/pfblockerng_update.php:90


Alias	Count	Packets	Updated	 
pfB_DNSBLIP	37930	0	Apr 20 09:59	  (2)
pfB_PS_v4	944	0	Apr 20 09:50	  (2)
pfB_Top_v4	4524	695	Apr 20 09:50	  (2)
pfB_Top_v6	630	0	Apr 20 09:50	  (2)
pfB_adserverlist	10883	9	Apr 20 09:50	  (2)
DNSBL_PrivacyFraudDNSGroup	95896	0	Apr 20 9:55	
DNSBL_MalwareExploitDNSGroup	178126	0	Apr 20 9:57	
DNSBL_SPAMDNSGroup	6689	0	Apr 20 9:57	
DNSBL_MaliciousDNSGroup	20831	0	Apr 20 9:58	
DNSBL_BambenekConsultingDNSGroup	732404	0	Apr 20 9:59	
DNSBL_EasyList	4837	0	Apr 20 9:59	
DNSBL_DNDBL	83273	0	Apr 20 9:54	
Logging is disabled

BBcan177 · Apr 20, 2016, 4:45 AM

@docjay:

BBCAN,

Hey- thanks for all of your hard work with this package! I'm setting up a 2nd PFsense box for a client and I'm wanting to copy over my 'pfblockerNG' config. Is there an easy way to do it other than doing it all by hand?

thanks

Jamie

Did you try the "Sync" Tab in the package?

BBcan177 · Apr 20, 2016, 4:58 AM

@Merchant:

Hello , I find issue opening somewebsites , ie when i try to open it takes long time when pfblockerNG is enabled

Usually when sites are slow to load with DNSBL, is that you have a multi-segmented LAN, and the desktop doesn't have full access to the DNSBL VIP address… Can you ping the DNSBL VIP address? There is a setting in the DNSBL tab to auto-create a floating permit rule, then select all of the LAN subnets that need access to the DNSBL VIP…

when this happens do i need to manually whitelist all those domains

Goto the Alerts tab, and you will see all of the Domains that are blocked… You can click the "+" icon to suppress this domain... You can also manually add domains to the "Custom Suppression" list in the DNSBL tab at the bottom of that page. When you manually add domains, you need to run a "Force Reload - DNSBL" for those domains to be whitelisted...

note sure if this crash is related to above error but
/usr/local/www/pfblockerng/pfblockerng_update.php:90
/usr/local/www/pfblockerng/pfblockerng_update.php:425
/usr/local/www/pfblockerng/pfblockerng_update.php:419

This error is from the "Update" Tab… Do you notice any issues when viewing that page? When you see this error, the next time... Please review the pfblockerng.log and see if you find any errors around the same time and send those to me...

I also notice this from your post:
pfB_DNSBLIP 37930 0 Apr 20 09:59

Its quite strange for the DNSBLIP alias to have so many IPs… If you have a IP based List, add those to the IPv4 tab instead of DNSBL which is domain based. The DNSBLIP is to collect any IPs that are found in a DNSBL feed.... Its not going to cause any issues, but its better in the IPv4 tab....

Abhishek · Apr 20, 2016, 5:05 AM


This error is from the "Update" Tab... Do you notice any issues when viewing that page? When you see this error, the next time... Please review the pfblockerng.log and see if you find any errors around the same time and send those to me...

sure if i get same crash error again i will will upload the log


Can you ping the DNSBL VIP address?  There is a setting in the DNSBL tab to auto-create a floating permit rule, then select all of the LAN subnets that need access to the DNSBL VIP...

Connection is

WAN –- PFSENSE---LAN ( Single LAN ( No van or multiple network )
i uploaded a pic in above post showing ping , i am able to ping with no delay

Pinging secure.adnxs.com [10.10.10.1] with 32 bytes of data:
Reply from 10.10.10.1: bytes=32 time<1ms TTL=64
Reply from 10.10.10.1: bytes=32 time<1ms TTL=64
Reply from 10.10.10.1: bytes=32 time<1ms TTL=64
Reply from 10.10.10.1: bytes=32 time<1ms TTL=64


Its quite strange for the DNSBLIP alias to have so many IPs... If you have a IP based List, add those to the IPv4 tab instead of DNSBL which is domain based. The DNSBLIP is to collect any IPs that are found in a DNSBL feed.... Its not going to cause any issues, but its better in the IPv4 tab....

yes i will move IP based list from DNSBL to IPV4 tab

BBcan177 · Apr 20, 2016, 5:11 AM

@Merchant:

Can you ping the DNSBL VIP address? There is a setting in the DNSBL tab to auto-create a floating permit rule, then select all of the LAN subnets that need access to the DNSBL VIP…

Pinging secure.adnxs.com [10.10.10.1] with 32 bytes of data:
Reply from 10.10.10.1: bytes=32 time<1ms TTL=64
Reply from 10.10.10.1: bytes=32 time<1ms TTL=64
Reply from 10.10.10.1: bytes=32 time<1ms TTL=64
Reply from 10.10.10.1: bytes=32 time<1ms TTL=64

You need to run this command to check for proper connectivity to the DNSBL VIP:
You should also be able to browse to the DNSBL VIP and get the 1x1…

ping 10.10.10.1

Goto the Alerts tab, and see what domains are being blocked… You can also hit F12 in the browser and open Dev Mode... Goto "console" and look for the domains that are blocked... You can try to suppress those domains that are causing you issue... Finally ensure that your desktops only have the pfSense DNS IP defined and no other DNS servers listed...

Abhishek · Apr 20, 2016, 5:22 AM

update

all my DNSBL list are names


http://adaway.org/hosts.txt
http://someonewhocares.org/hosts/
http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&mimetype=plaintext
http://winhelp2002.mvps.org/hosts.txt
http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&startdate[day]=&startdate[month]=&startdate[year]=&useip=0.0.0.0
http://www.securemecca.com/Downloads/hosts.txt

http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&mimetype=plaintextoup
http://hosts-file.net/ad_servers.txt
http://adaway.org/hosts.txt
http://sysctl.org/cameleon/hosts
http://adblock.gjtech.net/?format=unix-hosts

https://s3.amazonaws.com/lists.disconnect.me/simple_tracking.txt
http://hosts-file.net/fsa.txt
http://hosts-file.net/hjk.txt
http://hosts-file.net/pha.txt
https://www.dshield.org/feeds/suspiciousdomains_High.txt
https://s3.amazonaws.com/lists.disconnect.me/simple_malvertising.txt
http://www.malwaredomainlist.com/hostslist/hosts.txt
http://mirror1.malwaredomains.com/files/justdomains
https://s3.amazonaws.com/lists.disconnect.me/simple_malware.txt
http://hosts-file.net/emd.txt
http://hosts-file.net/exp.txt
http://hosts-file.net/mmt.txt
http://lists.malwarepatrol.net/cgi/getfile?receipt=f1442112770&product=8&list=dansguardian
http://zeustracker.abuse.ch/blocklist.php?download=domainblocklist
https://malc0de.com/bl/BOOT

http://hosts-file.net/grm.txt
http://hosts-file.net/hfs.txt

https://raw.githubusercontent.com/Dawsey21/Lists/master/main-blacklist.txt

http://hosts-file.net/hphosts-partial.txt

http://hosts-file.net/hphosts-partial.txt
http://someonewhocares.org/hosts/hosts
http://winhelp2002.mvps.org/hosts.txt
http://gist.githubusercontent.com/BBcan177/4a8bf37c131be4803cb2/raw

http://osint.bambenekconsulting.com/feeds/dga-feed.gz
http://osint.bambenekconsulting.com/feeds/c2-dommasterlist.txt


Pinging 10.10.10.1 with 32 bytes of data:
Reply from 10.10.10.1: bytes=32 time<1ms TTL=64
Reply from 10.10.10.1: bytes=32 time<1ms TTL=64
Reply from 10.10.10.1: bytes=32 time<1ms TTL=64
Reply from 10.10.10.1: bytes=32 time<1ms TTL=64

Ping statistics for 10.10.10.1:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 0ms, Maximum = 0ms, Average = 0ms

But when i tried to open 10.10.10.1 in webbrowser i am getting pfsense login page instead of 1x1 image , so i guess that might b the problem

so i post my detail settings

SINGLE WAN LINK to NIC 1 PFSEnse ( wan dhcp ) –-----PFsense WAN Port --------PFsense LAN ( SINGLE LAN 192.168.x.x Network )

we dont have Multi WAN or MUlti LAN /Vlan
ie two NIC Cards ( 1 connected ISP LINK ) and other NIC Card ( Connected to Switch )

HTTPS is enabled for Pfsense
Installation is Fresh 2.3 install with xml backup restored

edit


nginx: 2016/04/20 15:09:06 [error] 65374#0: *476 open() "/usr/local/www/track/click/30047178/community.spiceworks.com" failed (2: No such file or directory), client: 10.10.10.1, server: , request: "GET /track/click/30047178/community.spiceworks.com?p=eyJzIjoiYnIzelB6UlBqRVFBSXNvajVBX0k5VVY0SExzIiwidiI6MSwicCI6IntcInVcIjozMDA0NzE3OCxcInZcIjoxLFwidXJsXCI6XCJodHRwczpcXFwvXFxcL2NvbW11bml0eS5zcGljZXdvcmtzLmNvbVxcXC90b3BpY1xcXC8xNTY3NTk1LWNhbi1pLXAydi1hLWRlYWQtcGVyc29uYWwtY29tcHV0ZXI_dXRtX2NhbXBhaWduPWRpZ2VzdCZ1dG1fbWVkaXVtPWVtYWlsJnV0bV9zb3VyY2U9ZGlnZXN0JnV0bWU9Z3JvdXBzK3BvcHVsYXIzXCIsXCJpZFwiOlwiNzFiODNmZjE4MzU3NDhlNzkzZDE3MmYzNDY5MTQ2MzhcIixcInVybF9pZHNcIjpbXCI1Zjg4NGU0OGIyYzY3OTZlMmExYzY3ZjNhNWQ2MDY3NDljZGJhMjFjXCJdfSJ9 HTTP/1.1", host: "track.spiceworks.com"

docjay · Apr 20, 2016, 1:47 PM

@BBcan177:

@docjay:

BBCAN,

Hey- thanks for all of your hard work with this package! I'm setting up a 2nd PFsense box for a client and I'm wanting to copy over my 'pfblockerNG' config. Is there an easy way to do it other than doing it all by hand?

thanks

Jamie

Did you try the "Sync" Tab in the package?

LOL, nope. Sorry, I didn't see it or think it would do what I would expect. I'll give it a shot this evening!

Thanks!

BBcan177 · Apr 20, 2016, 4:01 PM

@Merchant:

update

An obersevation: why use three different yoyo feeds:
http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&mimetype=plaintext
http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&startdate[day]=&startdate[month]=&startdate[year]=&useip=0.0.0.0
http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&mimetype=plaintextoup

Is there a reason why you seperated hpHosts, instead of using the single download for them all?
http://hosts-file.net/download/hosts.zip

http://hosts-file.net/fsa.txt
http://hosts-file.net/hjk.txt
http://hosts-file.net/pha.txt
http://hosts-file.net/emd.txt
http://hosts-file.net/exp.txt
http://hosts-file.net/mmt.txt
http://hosts-file.net/grm.txt
http://hosts-file.net/hfs.txt

As a note: Look at the pfblockerng.log, it will show a section "DNSBL Domain/IP Counts" which will show you which DNSBL Feeds have IPs… look for the .ip extension...

But when i tried to open 10.10.10.1 in webbrowser i am getting pfsense login page instead of 1x1 image , so i guess that might b the problem

Yes your problem is due to Squid… I don't use Squid, and can't really help you workaround this issue.... Others have had a similar issue, but haven't seen anyone reply with a solution yet...

docjay · Apr 20, 2016, 11:07 PM

@docjay:

@BBcan177:

@docjay:

BBCAN,

Hey- thanks for all of your hard work with this package! I'm setting up a 2nd PFsense box for a client and I'm wanting to copy over my 'pfblockerNG' config. Is there an easy way to do it other than doing it all by hand?

thanks

Jamie

Did you try the "Sync" Tab in the package?

LOL, nope. Sorry, I didn't see it or think it would do what I would expect. I'll give it a shot this evening!

Thanks!

I tried the sync tab and it worked bad ass! I should have looked at it before, sorry for bothering you & thanks for thinking of this option!

Abhishek · Apr 21, 2016, 5:21 AM

@BBcan177:

@Merchant:

update

An obersevation: why use three different yoyo feeds:
http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&mimetype=plaintext
http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&startdate[day]=&startdate[month]=&startdate[year]=&useip=0.0.0.0
http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&mimetype=plaintextoup

Is there a reason why you seperated hpHosts, instead of using the single download for them all?
http://hosts-file.net/download/hosts.zip

http://hosts-file.net/fsa.txt
http://hosts-file.net/hjk.txt
http://hosts-file.net/pha.txt
http://hosts-file.net/emd.txt
http://hosts-file.net/exp.txt
http://hosts-file.net/mmt.txt
http://hosts-file.net/grm.txt
http://hosts-file.net/hfs.txt

As a note: Look at the pfblockerng.log, it will show a section "DNSBL Domain/IP Counts" which will show you which DNSBL Feeds have IPs… look for the .ip extension...

But when i tried to open 10.10.10.1 in webbrowser i am getting pfsense login page instead of 1x1 image , so i guess that might b the problem

Yes your problem is due to Squid… I don't use Squid, and can't really help you workaround this issue.... Others have had a similar issue, but haven't seen anyone reply with a solution yet...

thank you , i will update the host's list
i will try modify my wpad so that host wont use proxy for 10.x.x.x network , will check today , thank you as usual , u rocks :)

update

Updated my wpad and now its working :D , thank you . its loading (1x1)
Now all websites are loading fast

GoldServe · Apr 21, 2016, 5:55 AM

Has anyone noticed the dnsbl alert entries have the wrong time? Seems to be a few hours ahead from all the other firewall entries.

Abhishek · Apr 21, 2016, 6:08 AM

@GoldServe:

Has anyone noticed the dnsbl alert entries have the wrong time? Seems to be a few hours ahead from all the other firewall entries.

mine is showing correct

BBcan177 · Apr 21, 2016, 6:36 AM

@GoldServe:

Has anyone noticed the dnsbl alert entries have the wrong time? Seems to be a few hours ahead from all the other firewall entries.

Which DNSBL alerts? HTTP or HTTPS. For HTTP, it takes the request time from the remote Lan device. Could the clock be off on one of your desktops?