PfBlockerNG v2.0 w/DNSBL

[wolfsden3]

@BBcan177:

If you are in a multi-segmented Lan network, there is a in the DNSBL tab to auto create a Floating allow rule for the selected Lan networks.

That is already checked and created

[renegade]

I have added all lists on my apu.
Now my unbound is very slow. It takes more than 800ms to resolve a hostname.
The flicker.conf list is about 60mb.
I have increased some values like Message Cache Size and Number of Hosts to Cache. 50mb and 100000. But it is still so sloooow.
Any ideas?
Thank you

[BBcan177]

@renegade:

I have added all lists on my apu.
Now my unbound is very slow. It takes more than 800ms to resolve a hostname.
The flicker.conf list is about 60mb.
I have increased some values like Message Cache Size and Number of Hosts to Cache. 50mb and 100000. But it is still so sloooow.

Ensure that your LAN devices can ping the DNSBL VIP, and also that they can browse to the DNSBL VIP and get the 1x1 pix.  If any of those two tests fail, you will see slowness as the browser is timing out waiting for the DNS resolution to the blocked Domains.

[renegade]

Thats works fine.
I tested the dns resolution time in the Diagnostic menu.
With top I can see that unbound takes a lot of cpu for some time.

[BBcan177]

Is the Resolver in "Forwarder" or "Resolver" mode?

What DNS settings are defined in the pfSense General Tab? As these settings are used by the pfSense box itself…

When testing the ping/browse to the DNSBL VIP, do that from the LAN clients and not from pfSense itself.

If you're in a multi-segmented LAN, ensure to use the 'DNSBL Permit Rule' option in the DNSBL Tab. Also clear your Browser/OS cache.

[renegade]

Resolver Mode.
No multi segmented LAN.
VIP is tested from remote.
I tested a dns resolution like www.IBM.com on pfsense box.
The slow network is reproducible on many devices.
Maybe there is a mistake in my config which unbound makes that slow.

[wolfsden3]

@BBcan177:

Is the Resolver in "Forwarder" or "Resolver" mode?

What DNS settings are defined in the pfSense General Tab? As these settings are used by the pfSense box itself…

When testing the ping/browse to the DNSBL VIP, do that from the LAN clients and not from pfSense itself.

If you're in a multi-segmented LAN, ensure to use the 'DNSBL Permit Rule' option in the DNSBL Tab. Also clear your Browser/OS cache.

Unfortunately not 100% sure of what fixed it, but the DNSBL alerts are current. On to the next…

[BBcan177]

@hulleyrob:

Ive just noticed an annoying bug when pressing save on the Firewall/pfBlockerNG/Edit/IPv4 page if there are any errors all your data gets wiped out :-(

Can you please leave the data so if can be corrected?

FYI, this bug was fixed by the following pfSense commit for pkg_edit.php:
https://github.com/pfsense/pfsense/commit/a654d899cd5d288501fea1ec52dba2e3f0e479ba

Its in the pfSense Dev version and will be in the next pfSense 2.3.2 release.

[luckman212]

@BBcan177:

pfBlockerNG v2.0 …When a DNS request is made for a domain that is listed in DNSBL, the request is redirected to a Virtual IP address where an instance of Lighttpd Web Server will collect the packet statistics and push a '1x1' GIF image to the Browser…

Just curious why you chose to use lighttpd vs. the nginx webserver that is already built in. Was this for backwards compatibility with older versions?

[renegade]

I think i found out, why my network is so slow and unresponseful sometimes.
The Unbound seems to restart pretty often :(

Jul 17 13:57:29	unbound	64290:0	notice: Restart of unbound 1.5.9.
Jul 17 13:57:29	unbound	64290:0	info: 1.000000 2.000000 4
Jul 17 13:57:29	unbound	64290:0	info: 0.524288 1.000000 3
Jul 17 13:57:29	unbound	64290:0	info: 0.262144 0.524288 2
Jul 17 13:57:29	unbound	64290:0	info: 0.131072 0.262144 14
Jul 17 13:57:29	unbound	64290:0	info: 0.065536 0.131072 10
Jul 17 13:57:29	unbound	64290:0	info: 0.032768 0.065536 13
Jul 17 13:57:29	unbound	64290:0	info: 0.016384 0.032768 1
Jul 17 13:57:29	unbound	64290:0	info: lower(secs) upper(secs) recursions
Jul 17 13:57:29	unbound	64290:0	info: [25%]=0.0598646 median[50%]=0.127795 [75%]=0.236398
Jul 17 13:57:29	unbound	64290:0	info: histogram of recursion processing times
Jul 17 13:57:29	unbound	64290:0	info: average recursion processing time 0.298118 sec
Jul 17 13:57:29	unbound	64290:0	info: server stats for thread 1: requestlist max 14 avg 7.29787 exceeded 0 jostled 0
Jul 17 13:57:29	unbound	64290:0	info: server stats for thread 1: 73 queries, 26 answers from cache, 47 recursions, 0 prefetch
Jul 17 13:57:29	unbound	64290:0	info: 0.262144 0.524288 1
Jul 17 13:57:29	unbound	64290:0	info: 0.131072 0.262144 2
Jul 17 13:57:29	unbound	64290:0	info: 0.065536 0.131072 2
Jul 17 13:57:29	unbound	64290:0	info: 0.032768 0.065536 1
Jul 17 13:57:29	unbound	64290:0	info: 0.016384 0.032768 1
Jul 17 13:57:29	unbound	64290:0	info: lower(secs) upper(secs) recursions
Jul 17 13:57:29	unbound	64290:0	info: [25%]=0.057344 median[50%]=0.114688 [75%]=0.212992
Jul 17 13:57:29	unbound	64290:0	info: histogram of recursion processing times
Jul 17 13:57:29	unbound	64290:0	info: average recursion processing time 0.156474 sec
Jul 17 13:57:29	unbound	64290:0	info: server stats for thread 0: requestlist max 0 avg 0 exceeded 0 jostled 0
Jul 17 13:57:29	unbound	64290:0	info: server stats for thread 0: 18 queries, 11 answers from cache, 7 recursions, 0 prefetch
Jul 17 13:57:29	unbound	64290:0	info: service stopped (unbound 1.5.9).
Jul 17 13:55:09	unbound	64290:0	info: start of service (unbound 1.5.9).
Jul 17 13:55:08	unbound	64290:0	notice: init module 0: iterator
Jul 17 13:54:33	unbound	64290:0	notice: Restart of unbound 1.5.9.
Jul 17 13:54:33	unbound	64290:0	info: server stats for thread 1: requestlist max 31 avg 16.4314 exceeded 0 jostled 0
Jul 17 13:54:33	unbound	64290:0	info: server stats for thread 1: 53 queries, 2 answers from cache, 51 recursions, 0 prefetch
Jul 17 13:54:33	unbound	64290:0	info: server stats for thread 0: requestlist max 1 avg 0.5 exceeded 0 jostled 0
Jul 17 13:54:33	unbound	64290:0	info: server stats for thread 0: 2 queries, 0 answers from cache, 2 recursions, 0 prefetch
Jul 17 13:54:33	unbound	64290:0	info: service stopped (unbound 1.5.9).
Jul 17 13:54:33	unbound	64290:0	info: start of service (unbound 1.5.9).
Jul 17 13:54:33	unbound	64290:0	notice: init module 0: iterator
Jul 17 13:54:15	filterdns		clearing entry 192.30.253.113 from table debian_updates on host github.com
Jul 17 13:54:15	filterdns		adding entry 192.30.253.112 to table debian_updates on host github.com
Jul 17 13:53:58	unbound	64290:0	notice: Restart of unbound 1.5.9.
Jul 17 13:53:58	unbound	64290:0	info: 0.262144 0.524288 14
Jul 17 13:53:58	unbound	64290:0	info: 0.131072 0.262144 26
Jul 17 13:53:58	unbound	64290:0	info: 0.065536 0.131072 18
Jul 17 13:53:58	unbound	64290:0	info: 0.032768 0.065536 9
Jul 17 13:53:58	unbound	64290:0	info: 0.016384 0.032768 4
Jul 17 13:53:58	unbound	64290:0	info: lower(secs) upper(secs) recursions
Jul 17 13:53:58	unbound	64290:0	info: [25%]=0.0828302 median[50%]=0.153758 [75%]=0.243239
Jul 17 13:53:58	unbound	64290:0	info: histogram of recursion processing times
Jul 17 13:53:58	unbound	64290:0	info: average recursion processing time 0.165017 sec
Jul 17 13:53:58	unbound	64290:0	info: server stats for thread 1: requestlist max 12 avg 4.98592 exceeded 0 jostled 0
Jul 17 13:53:58	unbound	64290:0	info: server stats for thread 1: 82 queries, 11 answers from cache, 71 recursions, 0 prefetch

[BBcan177]

@luckman212:

@BBcan177:

pfBlockerNG v2.0 …When a DNS request is made for a domain that is listed in DNSBL, the request is redirected to a Virtual IP address where an instance of Lighttpd Web Server will collect the packet statistics and push a '1x1' GIF image to the Browser…

Just curious why you chose to use lighttpd vs. the nginx webserver that is already built in. Was this for backwards compatibility with older versions?

Hi Luke,

When 2.3 was in testing, I attempted to switch to NGINX; however, ran into issues on DNSBL logging for HTTPS alerts. Lighttpd has an error-conditional-log that contains some details that the pkg uses to provide some Alert details for HTTPS alerts. With NGINX, when the browser drops the connection to the DNSBL webserver due the incorrect Certificate provided, no other logging is completed by NGINX. I managed to get the Devs to add LUA support to NGINX, but there were issues with OpenSSL at the time that prevented further testing. I may revisit this as time permits, but so far, there hasn't been any issues with both NGINX and Lighttpd running at the same time.

[BBcan177]

@renegade:

I think i found out, why my network is so slow and unresponseful sometimes.
The Unbound seems to restart pretty often

Do you have DHCP Registration enabled in the Resolver? Maybe try it without those two checkboxes… Are you running it in Resover or Forwarder mode? Is DNSSEC enabled?

[BBcan177]

pfBlockerNG v2.1.1 w/TLD - Pull request has been posted:
    https://forum.pfsense.org/index.php?topic=115357.0

[renegade]

@BBcan177:

@renegade:

I think i found out, why my network is so slow and unresponseful sometimes.
The Unbound seems to restart pretty often

Do you have DHCP Registration enabled in the Resolver? Maybe try it without those two checkboxes… Are you running it in Resover or Forwarder mode? Is DNSSEC enabled?

I am running the DNS in Resolver Mode. Forwarding is disabled. DNSSEC is enabled; but was disabled before and i had the same problems.
Yes, i have both boxes checked. (I need this).
I can try to disable the checkboxes. This morning my whole network had problems, because the Unbound stopped and did not restart.

[pfcode]

@renegade:

@BBcan177:

@renegade:

I think i found out, why my network is so slow and unresponseful sometimes.
The Unbound seems to restart pretty often

Do you have DHCP Registration enabled in the Resolver? Maybe try it without those two checkboxes… Are you running it in Resover or Forwarder mode? Is DNSSEC enabled?

I am running the DNS in Resolver Mode. Forwarding is disabled. DNSSEC is enabled; but was disabled before and i had the same problems.
Yes, i have both boxes checked. (I need this).
I can try to disable the checkboxes. This morning my whole network had problems, because the Unbound stopped and did not restart.

There is a serious issue in the unbound service package, you will have to disable DNSBL, so that the line "server:include: /var/unbound/pfb_dnsbl.conf" is removed from Custom Options in the Unbound, to make unbound restart properly when your modem is rebooted or your IPv4/IPv6 WAN IP got renewed.

[BBcan177]

@renegade:

@BBcan177:

@renegade:

I think i found out, why my network is so slow and unresponseful sometimes.
The Unbound seems to restart pretty often

Do you have DHCP Registration enabled in the Resolver? Maybe try it without those two checkboxes… Are you running it in Resover or Forwarder mode? Is DNSSEC enabled?

I am running the DNS in Resolver Mode. Forwarding is disabled. DNSSEC is enabled; but was disabled before and i had the same problems.
Yes, i have both boxes checked. (I need this).
I can try to disable the checkboxes. This morning my whole network had problems, because the Unbound stopped and did not restart.

Check the Resolver.log for additional details… You might have to increase the Log Verbosity to get it to show more details to help diagnose the issue.

[BBcan177]

@pfcode:

There is a serious issue in the unbound service package, you will have to disable DNSBL, so that the line "server:include: /var/unbound/pfb_dnsbl.conf" is removed from Custom Options in the Unbound, to make unbound restart properly when your modem is rebooted or your IPv4/IPv6 WAN IP got renewed.

Please post this issue in Redmine. Its the best way to help get this resolved.

https://redmine.pfsense.org/projects/pfsense

[renegade]

So i would have to disable DNSBL every morning after my 24h disconnect, restart unbound and enable DNSBL again…?
I do not really understand this bug, so i think that i am not the eligible person to file an issue.

[trumee]

I have the following list in 'Whitelist' for slickdeals, but still the links are not being whitelisted:

dotomi.com
dpbolvw.net
evyy.net
doubleclick.net
avantlink.com
tkqlhce.com
pjatr.com
shareasale.com
pntrack.com
ojrq.net
clickserve.cc-dt.com
jdoqocy.com
pntrac.com
affiliatefuture.com
anrdoezrs.net
gan.doubleclick.net
emjcd.com
commission-junction.com
cj.com
apmebf.com
gopjn.com
bfast.com
redirect.at
redirectingat.com
7eer.net
linksynergy.com
kqzyfj.com
cc-dt.com
altfarm.mediaplex.com
onenetworkdirect.net
go2jump.org
pntra.com
pjtra.com
qksrv.net
ad.doubleclick.net
cj.dotomi.com

The following link failed:

http://www.tkqlhce.com/click-4485850-11886886-1423272259000?sid=941dcd3e509111e69c477a73c7e879150INT&url=http%3A%2F%2Fwww.newegg.com

[BBcan177]

@trumee:

I have the following list in 'Whitelist' for slickdeals, but still the links are not being whitelisted:

The following link failed:

http://www.tkqlhce.com/click-4485850-11886886-1423272259000?sid=941dcd3e509111e69c477a73c7e879150INT&url=http%3A%2F%2Fwww.newegg.com

You also need to Whitelist:
    www.tkqlhce.com

add that to the Whitelist, and run a "Force Reload - DNSBL"