PfBlockerNG v2.0 w/DNSBL

luckman212

pfBlockerNG v2.0 …When a DNS request is made for a domain that is listed in DNSBL, the request is redirected to a Virtual IP address where an instance of Lighttpd Web Server will collect the packet statistics and push a '1x1' GIF image to the Browser…

Just curious why you chose to use lighttpd vs. the nginx webserver that is already built in. Was this for backwards compatibility with older versions?

renegade

I think i found out, why my network is so slow and unresponseful sometimes.
The Unbound seems to restart pretty often :(

Jul 17 13:57:29	unbound	64290:0	notice: Restart of unbound 1.5.9.
Jul 17 13:57:29	unbound	64290:0	info: 1.000000 2.000000 4
Jul 17 13:57:29	unbound	64290:0	info: 0.524288 1.000000 3
Jul 17 13:57:29	unbound	64290:0	info: 0.262144 0.524288 2
Jul 17 13:57:29	unbound	64290:0	info: 0.131072 0.262144 14
Jul 17 13:57:29	unbound	64290:0	info: 0.065536 0.131072 10
Jul 17 13:57:29	unbound	64290:0	info: 0.032768 0.065536 13
Jul 17 13:57:29	unbound	64290:0	info: 0.016384 0.032768 1
Jul 17 13:57:29	unbound	64290:0	info: lower(secs) upper(secs) recursions
Jul 17 13:57:29	unbound	64290:0	info: [25%]=0.0598646 median[50%]=0.127795 [75%]=0.236398
Jul 17 13:57:29	unbound	64290:0	info: histogram of recursion processing times
Jul 17 13:57:29	unbound	64290:0	info: average recursion processing time 0.298118 sec
Jul 17 13:57:29	unbound	64290:0	info: server stats for thread 1: requestlist max 14 avg 7.29787 exceeded 0 jostled 0
Jul 17 13:57:29	unbound	64290:0	info: server stats for thread 1: 73 queries, 26 answers from cache, 47 recursions, 0 prefetch
Jul 17 13:57:29	unbound	64290:0	info: 0.262144 0.524288 1
Jul 17 13:57:29	unbound	64290:0	info: 0.131072 0.262144 2
Jul 17 13:57:29	unbound	64290:0	info: 0.065536 0.131072 2
Jul 17 13:57:29	unbound	64290:0	info: 0.032768 0.065536 1
Jul 17 13:57:29	unbound	64290:0	info: 0.016384 0.032768 1
Jul 17 13:57:29	unbound	64290:0	info: lower(secs) upper(secs) recursions
Jul 17 13:57:29	unbound	64290:0	info: [25%]=0.057344 median[50%]=0.114688 [75%]=0.212992
Jul 17 13:57:29	unbound	64290:0	info: histogram of recursion processing times
Jul 17 13:57:29	unbound	64290:0	info: average recursion processing time 0.156474 sec
Jul 17 13:57:29	unbound	64290:0	info: server stats for thread 0: requestlist max 0 avg 0 exceeded 0 jostled 0
Jul 17 13:57:29	unbound	64290:0	info: server stats for thread 0: 18 queries, 11 answers from cache, 7 recursions, 0 prefetch
Jul 17 13:57:29	unbound	64290:0	info: service stopped (unbound 1.5.9).
Jul 17 13:55:09	unbound	64290:0	info: start of service (unbound 1.5.9).
Jul 17 13:55:08	unbound	64290:0	notice: init module 0: iterator
Jul 17 13:54:33	unbound	64290:0	notice: Restart of unbound 1.5.9.
Jul 17 13:54:33	unbound	64290:0	info: server stats for thread 1: requestlist max 31 avg 16.4314 exceeded 0 jostled 0
Jul 17 13:54:33	unbound	64290:0	info: server stats for thread 1: 53 queries, 2 answers from cache, 51 recursions, 0 prefetch
Jul 17 13:54:33	unbound	64290:0	info: server stats for thread 0: requestlist max 1 avg 0.5 exceeded 0 jostled 0
Jul 17 13:54:33	unbound	64290:0	info: server stats for thread 0: 2 queries, 0 answers from cache, 2 recursions, 0 prefetch
Jul 17 13:54:33	unbound	64290:0	info: service stopped (unbound 1.5.9).
Jul 17 13:54:33	unbound	64290:0	info: start of service (unbound 1.5.9).
Jul 17 13:54:33	unbound	64290:0	notice: init module 0: iterator
Jul 17 13:54:15	filterdns		clearing entry 192.30.253.113 from table debian_updates on host github.com
Jul 17 13:54:15	filterdns		adding entry 192.30.253.112 to table debian_updates on host github.com
Jul 17 13:53:58	unbound	64290:0	notice: Restart of unbound 1.5.9.
Jul 17 13:53:58	unbound	64290:0	info: 0.262144 0.524288 14
Jul 17 13:53:58	unbound	64290:0	info: 0.131072 0.262144 26
Jul 17 13:53:58	unbound	64290:0	info: 0.065536 0.131072 18
Jul 17 13:53:58	unbound	64290:0	info: 0.032768 0.065536 9
Jul 17 13:53:58	unbound	64290:0	info: 0.016384 0.032768 4
Jul 17 13:53:58	unbound	64290:0	info: lower(secs) upper(secs) recursions
Jul 17 13:53:58	unbound	64290:0	info: [25%]=0.0828302 median[50%]=0.153758 [75%]=0.243239
Jul 17 13:53:58	unbound	64290:0	info: histogram of recursion processing times
Jul 17 13:53:58	unbound	64290:0	info: average recursion processing time 0.165017 sec
Jul 17 13:53:58	unbound	64290:0	info: server stats for thread 1: requestlist max 12 avg 4.98592 exceeded 0 jostled 0
Jul 17 13:53:58	unbound	64290:0	info: server stats for thread 1: 82 queries, 11 answers from cache, 71 recursions, 0 prefetch

BBcan177

@luckman212:

@BBcan177:

pfBlockerNG v2.0 …When a DNS request is made for a domain that is listed in DNSBL, the request is redirected to a Virtual IP address where an instance of Lighttpd Web Server will collect the packet statistics and push a '1x1' GIF image to the Browser…

Just curious why you chose to use lighttpd vs. the nginx webserver that is already built in. Was this for backwards compatibility with older versions?

Hi Luke,

When 2.3 was in testing, I attempted to switch to NGINX; however, ran into issues on DNSBL logging for HTTPS alerts. Lighttpd has an error-conditional-log that contains some details that the pkg uses to provide some Alert details for HTTPS alerts. With NGINX, when the browser drops the connection to the DNSBL webserver due the incorrect Certificate provided, no other logging is completed by NGINX. I managed to get the Devs to add LUA support to NGINX, but there were issues with OpenSSL at the time that prevented further testing. I may revisit this as time permits, but so far, there hasn't been any issues with both NGINX and Lighttpd running at the same time.

BBcan177

@renegade:

I think i found out, why my network is so slow and unresponseful sometimes.
The Unbound seems to restart pretty often

Do you have DHCP Registration enabled in the Resolver? Maybe try it without those two checkboxes… Are you running it in Resover or Forwarder mode? Is DNSSEC enabled?

BBcan177

pfBlockerNG v2.1.1 w/TLD - Pull request has been posted:
https://forum.pfsense.org/index.php?topic=115357.0

renegade

@BBcan177:

@renegade:

I think i found out, why my network is so slow and unresponseful sometimes.
The Unbound seems to restart pretty often

Do you have DHCP Registration enabled in the Resolver? Maybe try it without those two checkboxes… Are you running it in Resover or Forwarder mode? Is DNSSEC enabled?

I am running the DNS in Resolver Mode. Forwarding is disabled. DNSSEC is enabled; but was disabled before and i had the same problems.
Yes, i have both boxes checked. (I need this).
I can try to disable the checkboxes. This morning my whole network had problems, because the Unbound stopped and did not restart.

pfcode

@renegade:

@BBcan177:

@renegade:

I think i found out, why my network is so slow and unresponseful sometimes.
The Unbound seems to restart pretty often

Do you have DHCP Registration enabled in the Resolver? Maybe try it without those two checkboxes… Are you running it in Resover or Forwarder mode? Is DNSSEC enabled?

I am running the DNS in Resolver Mode. Forwarding is disabled. DNSSEC is enabled; but was disabled before and i had the same problems.
Yes, i have both boxes checked. (I need this).
I can try to disable the checkboxes. This morning my whole network had problems, because the Unbound stopped and did not restart.

There is a serious issue in the unbound service package, you will have to disable DNSBL, so that the line "server:include: /var/unbound/pfb_dnsbl.conf" is removed from Custom Options in the Unbound, to make unbound restart properly when your modem is rebooted or your IPv4/IPv6 WAN IP got renewed.

BBcan177

@renegade:

@BBcan177:

@renegade:

I think i found out, why my network is so slow and unresponseful sometimes.
The Unbound seems to restart pretty often

Do you have DHCP Registration enabled in the Resolver? Maybe try it without those two checkboxes… Are you running it in Resover or Forwarder mode? Is DNSSEC enabled?

I am running the DNS in Resolver Mode. Forwarding is disabled. DNSSEC is enabled; but was disabled before and i had the same problems.
Yes, i have both boxes checked. (I need this).
I can try to disable the checkboxes. This morning my whole network had problems, because the Unbound stopped and did not restart.

Check the Resolver.log for additional details… You might have to increase the Log Verbosity to get it to show more details to help diagnose the issue.

BBcan177

@pfcode:

There is a serious issue in the unbound service package, you will have to disable DNSBL, so that the line "server:include: /var/unbound/pfb_dnsbl.conf" is removed from Custom Options in the Unbound, to make unbound restart properly when your modem is rebooted or your IPv4/IPv6 WAN IP got renewed.

Please post this issue in Redmine. Its the best way to help get this resolved.

https://redmine.pfsense.org/projects/pfsense

renegade

So i would have to disable DNSBL every morning after my 24h disconnect, restart unbound and enable DNSBL again…?
I do not really understand this bug, so i think that i am not the eligible person to file an issue.

trumee

I have the following list in 'Whitelist' for slickdeals, but still the links are not being whitelisted:


dotomi.com
dpbolvw.net
evyy.net
doubleclick.net
avantlink.com
tkqlhce.com
pjatr.com
shareasale.com
pntrack.com
ojrq.net
clickserve.cc-dt.com
jdoqocy.com
pntrac.com
affiliatefuture.com
anrdoezrs.net
gan.doubleclick.net
emjcd.com
commission-junction.com
cj.com
apmebf.com
gopjn.com
bfast.com
redirect.at
redirectingat.com
7eer.net
linksynergy.com
kqzyfj.com
cc-dt.com
altfarm.mediaplex.com
onenetworkdirect.net
go2jump.org
pntra.com
pjtra.com
qksrv.net
ad.doubleclick.net
cj.dotomi.com

The following link failed:


http://www.tkqlhce.com/click-4485850-11886886-1423272259000?sid=941dcd3e509111e69c477a73c7e879150INT&url=http%3A%2F%2Fwww.newegg.com

BBcan177

@trumee:

I have the following list in 'Whitelist' for slickdeals, but still the links are not being whitelisted:

The following link failed:
http://www.tkqlhce.com/click-4485850-11886886-1423272259000?sid=941dcd3e509111e69c477a73c7e879150INT&url=http%3A%2F%2Fwww.newegg.com

You also need to Whitelist:
www.tkqlhce.com

add that to the Whitelist, and run a "Force Reload - DNSBL"

trumee

Is it possible to have a regex like *tkqlhce.com to block all the subdomains?

tjbaudio

Hello, new member here. Long time linux user with a solid understanding of networking and Vlans but I am brand new to pfsense and complex firewall configs. Currently I am trying to set up a new firewall for my home. Ultimate goals are to filter ads, block M$ spying and IOT back doors, separate Vlan or nic for IP cameras, and VPN access for said cameras.

Currently I am stuck at the ad filtering. Ad's are blocked but page loads are very long and none are logged. The VIP is timing out on :80 and :443, connection reset error on :8443. :8081 serves up just fine and is logged in alerts correctly. Given the service is up and running and :8081 works and ads are blocked I think DNSBL is up and running just fine. I suspect a firewall rule or NAT is not correct. I have done the same install twice with the same results both times. Since this package is fairly new GOOGLE is coming up short but I will keep trying.

I am using P4 1.8 GHz with 768 meg ram. Fresh install of PFsense 2.3.1 upgraded to 2.3.1-RELEASE-p5 (i386). Basic ip and interface settings working. Installed the latest pfBlockerNG package 2.0.17. Configured per post 18 of this thread skipping Alxea. The only oops was I enabled pfBNG a couple of steps early. Reboot after initial setup for good measure.

Again, every thing works as expected other than browsing to 10.10.10.1:80 or 10.10.10.1:443 has to time out during page loads. I have looked around and I think I see the rules in place for this but I still think one is wrong or I am missing something simple. Included below are some snips from the config XML.

Thank you very much for any help or hints you can give.

 <nat><outbound><mode>automatic</mode></outbound> 
 <rule><source>
 <any><destination><address>10.10.10.1</address>

<port>80</port></destination> 
<protocol>tcp</protocol>
<target>127.0.0.1</target>
<local-port>8081</local-port>
<interface>lan</interface>

 <associated-rule-id><natreflection>purenat</natreflection></associated-rule-id></any></rule> 
 <rule><source>
 <any><destination><address>10.10.10.1</address>

<port>443</port></destination> 
<protocol>tcp</protocol>
<target>127.0.0.1</target>
<local-port>8443</local-port>
<interface>lan</interface>

 <associated-rule-id><natreflection>purenat</natreflection></associated-rule-id></any></rule> 
 <separator></separator></nat> 
 <filter><rule><type>pass</type>
<ipprotocol>inet</ipprotocol>

<interface>lan</interface>
<tracker>0100000101</tracker>
<source>
<network>lan</network>

 <destination><any></any></destination></rule> 
 <rule><type>pass</type>
<ipprotocol>inet6</ipprotocol>

<interface>lan</interface>
<tracker>0100000102</tracker>
<source>
<network>lan</network>

 <destination><any></any></destination></rule></filter> 
 <shaper><ipsec><aliases><alias><name>pfB_DNSBLIP</name>
 <url>https://127.0.0.1:443/pfblockerng/pfblockerng.php?pfb=pfB_DNSBLIP</url> 
<updatefreq>32</updatefreq>

<address>

<type>urltable</type>
 <detail></detail> 

 <proxyarp>```
 <pfblockerngdnsblsettings><config><pfb_dnsbl>on</pfb_dnsbl>
<pfb_dnsvip>10.10.10.1</pfb_dnsvip>
<pfb_dnsport>8081</pfb_dnsport>
<pfb_dnsport_ssl>8443</pfb_dnsport_ssl>
<dnsbl_interface>lan</dnsbl_interface>
 <pfb_dnsbl_rule><dnsbl_allow_int>lan</dnsbl_allow_int>
<action>Deny_Both</action>
<aliaslog>enabled</aliaslog>
 <autoaddrnot_in><autoports_in><aliasports_in><autoaddr_in><autonot_in><aliasaddr_in><autoproto_in><agateway_in>default</agateway_in>
 <autoaddrnot_out><autoports_out><aliasports_out><autoaddr_out><autonot_out><aliasaddr_out><autoproto_out><agateway_out>default</agateway_out>
 <alexa_enable><alexa_count>1000</alexa_count>
<alexa_inclusion>ca,co,com,io,net,org</alexa_inclusion></alexa_enable></autoproto_out></aliasaddr_out></autonot_out></autoaddr_out></aliasports_out></autoports_out></autoaddrnot_out></autoproto_in></aliasaddr_in></autonot_in></autoaddr_in></aliasports_in></autoports_in></autoaddrnot_in></pfb_dnsbl_rule></config></pfblockerngdnsblsettings>

 <pfblockerng><config><enable_cb>on</enable_cb>
<pfb_keep>on</pfb_keep>
<pfb_interval>24</pfb_interval>
<pfb_min>0</pfb_min>
<pfb_hour>0</pfb_hour>
<pfb_dailystart>0</pfb_dailystart>
 <enable_dup><enable_agg><suppression><enable_log><database_cc><skipfeed>0</skipfeed>
<log_maxlines>20000</log_maxlines>
<inbound_interface>lan</inbound_interface>
<inbound_deny_action>block</inbound_deny_action>
<outbound_interface>wan</outbound_interface>
<outbound_deny_action>reject</outbound_deny_action>
 <openvpn_action><enable_float><pass_order>order_0</pass_order>
<autorule_suffix>autorule</autorule_suffix>
 <killstates></killstates></enable_float></openvpn_action></database_cc></enable_log></suppression></enable_agg></enable_dup></config></pfblockerng>

 <virtualip><vip><mode>ipalias</mode>
<interface>lan</interface>

<type>single</type>
<subnet_bits>32</subnet_bits>
<subnet>10.10.10.1</subnet></vip></virtualip> 

```</proxyarp></address></alias></aliases></ipsec></shaper>

BBcan177

@trumee:

Is it possible to have a regex like *tkqlhce.com to block all the subdomains?

The pending version 2.1.1 has the option to Whitelist all Sub-Domains by prefixing the Whitelisted Domain with a "dot"… Just waiting for the Devs to merge the new changes... See this thread:

https://forum.pfsense.org/index.php?topic=115357.0

BBcan177

@tjbaudio:

Again, every thing works as expected other than browsing to 10.10.10.1:80 or 10.10.10.1:443 has to time out during page loads. I have looked around and I think I see the rules in place for this but I still think one is wrong or I am missing something simple. Included below are some snips from the config XML

Since you are using multiple VLANS, you need to ensure that all VLANs can hit the DNSBL VIP… There is an option in the DNSBL Tab, to add a Permit Floating rule... Ensure that you select all of the VLANs in the selection menu... You will have slow browsing issues if you can't get ping the DNSBL VIP and Browse to the DNSBL VIP and get the 1x1 pix....

tjbaudio

@BBcan177:

@tjbaudio:

Again, every thing works as expected other than browsing to 10.10.10.1:80 or 10.10.10.1:443 has to time out during page loads. I have looked around and I think I see the rules in place for this but I still think one is wrong or I am missing something simple. Included below are some snips from the config XML

Since you are using multiple VLANS, you need to ensure that all VLANs can hit the DNSBL VIP… There is an option in the DNSBL Tab, to add a Permit Floating rule... Ensure that you select all of the VLANs in the selection menu... You will have slow browsing issues if you can't get ping the DNSBL VIP and Browse to the DNSBL VIP and get the 1x1 pix....

My apologies. That is my ultimate goal. At this point I just have a stock install with 2 nics, WAN and LAN nics. No Vlan's. Just default settings+ pfBNG installed.
Edit to add:
I can ping 10.10.10.1 from LAN.
In states I get:
LAN tcp 10.10.10.1:8081 <- 192.168.1.101:50268 ESTABLISHED:ESTABLISHED 5 / 4 587 B / 576 B
When loading 10.10.10.1:8081 in a browser.

LAN tcp 127.0.0.1:8443 (10.10.10.1:443) <- 192.168.1.135:50120 CLOSED:SYN_SENT 3 / 0 152 B / 0 B
When loading 10.10.10.1:80 in a browser.

LAN tcp 127.0.0.1:8443 (10.10.10.1:443) <- 192.168.1.101:35768 CLOSED:SYN_SENT 2 / 0 120 B / 0 B
LAN tcp 127.0.0.1:8443 (10.10.10.1:443) <- 192.168.1.101:35770 CLOSED:SYN_SENT 2 / 0 120 B / 0 B
When loading 10.10.10.1 in a browser.

renegade

@BBcan177:

@pfcode:

There is a serious issue in the unbound service package, you will have to disable DNSBL, so that the line "server:include: /var/unbound/pfb_dnsbl.conf" is removed from Custom Options in the Unbound, to make unbound restart properly when your modem is rebooted or your IPv4/IPv6 WAN IP got renewed.

Please post this issue in Redmine. Its the best way to help get this resolved.

https://redmine.pfsense.org/projects/pfsense

Okay guys, what can i do here?
I am really sad that i cannot use DNSBL, because my provider cuts my line every day…
Do i have to post a bounty and refer to that forum post?

tjbaudio

I think I have traced my problem to NAT reflection. DNSBL is responding with the VIP address when tested with a known add site. The VIP server is up and responds as it should when using ports 8081 and 8443.
The firewall states show it forwarding when using 80 and 443. but nothing ever comes back.
I have tested this with wget, firefox, and chrom.

I love how DNSBL and pfBlockerNG are working. I am just going to have to go back over my pfsense setup to figure out the NAT reflection problem.

BBcan177

@renegade:

Okay guys, what can i do here?
I am really sad that i cannot use DNSBL, because my provider cuts my line every day…
Do i have to post a bounty and refer to that forum post?

Hi renegade,

You don't need to post a bounty. The redmine link is a site to post bugs/features for pfSense. The issue at hand seems to be related to the base pfSense software and not with the package. I don't have a similar network environment to debug this issue. So that's why I suggested that someone with the issue post in redmine to help sort out this issue. In the meantime try to reduce the number of feeds used for DNSBL which might reduce the unbound resolver load time.