@talaverde said in Still having classic problem of blocked URLs with 'unknown' feed:

Even after a completely fresh reinstall, I keep getting unwanted URLs on the DNSBL block list with 'unknown' feed. Here are some examples:
wsapi.skype.com
static.asm.skype.com
consumer.entitlement.skype.com
in.appcenter.ms

All of these domains above have a CNAME. Is it possible that these CNAMES are in your Blocklists?

drill @8.8.8.8 wsapi.skype.com wsapi.skype.com. 2995 IN CNAME client-ws.gateway.messenger.geo.msnmessenger.msn.com.akadns.net. client-ws.gateway.messenger.geo.msnmessenger.msn.com.akadns.net. 59 IN CNAME eus-wsapi.cloudapp.net. eus-wsapi.cloudapp.net. 58 IN A 13.92.27.116 drill @8.8.8.8 static.asm.skype.com static.asm.skype.com. 1657 IN CNAME static-asm-skype.trafficmanager.net. static-asm-skype.trafficmanager.net. 299 IN CNAME nus1-authgw.cloudapp.net. nus1-authgw.cloudapp.net. 52 IN A 40.77.16.143 drill @8.8.8.8 consumer.entitlement.skype.com consumer.entitlement.skype.com. 1969 IN CNAME sconsentit9.trafficmanager.net. sconsentit9.trafficmanager.net. 299 IN CNAME sconsentit903.cloudapp.net. sconsentit903.cloudapp.net. 8 IN A 40.122.44.183 drill @8.8.8.8 in.appcenter.ms in.appcenter.ms. 732 IN CNAME in-secondary-prod-east-us2.prod.avalanch.es. in-secondary-prod-east-us2.prod.avalanch.es. 129 IN CNAME 0e6fa46e-9c94-4256-b449-4f54c1f1e69f.cloudapp.net. 0e6fa46e-9c94-4256-b449-4f54c1f1e69f.cloudapp.net. 47 IN A 13.68.31.193 drill @8.8.8.8 download.windowsupdate.com download.windowsupdate.com. 1303 IN CNAME 2-01-3cf7-0009.cdx.cedexis.net. 2-01-3cf7-0009.cdx.cedexis.net. 239 IN CNAME b1ns.au-msedge.net. b1ns.au-msedge.net. 27 IN CNAME b1ns.c-0001.c-msedge.net. b1ns.c-0001.c-msedge.net. 27 IN CNAME c-0001.c-msedge.net. c-0001.c-msedge.net. 27 IN A 13.107.4.50

grep -r -l "wsapi.skype.com" /var/db/pfblockerng/*
When I run this command, these files show up
/var/db/pfblockerng/dnsbl_cache.sqlite
/var/db/pfblockerng/pfbalexawhitelist.txt
/var/db/pfblockerng/top-1m.csv
Even if I delete those files and run a reload update, they still show up. They won't go away. I figure out how to keep these from being blocked or show up on the list. it seems like every .skype.com subdomain is being blocked. I've added skype.com, .skype.com and even the subdomains themselves to my whitelist. Still, no difference.
Any thoughts?
What is dnsbl_cache.sqlite? Is that just a log?

The dnsbl_cache.sqlite is a database to show the last blocked event. You don't need to delete that file. And definitely don't need to delete the TOP1M Database (Whitelist).

You need to grep for DNSBL events as:

grep "example.com" /var/db/pfblockerng/dnsbl/*

Thanks for the tip. I used the developer tool to see the url path. All is fine now.

https://www.youtube.com/watch?v=g0KOcfGicjM

Ah, yes, that's what I was understanding.

I don't know if anyone else would be interested, but I would certainly use TLD blocking with the alternate model (blacklist by default, whitelist desired TLDs, and then process exceptions to the whitelist by adding in specifically blacklisted domains).

Generally, I think that would give me a more maintainable list. I assume most of the newer .tlds are junk (at this point in time, anyway). Rather than trying to keep up with that list, I'd rather have the option to define the list of known good (and most widely used), and go from there.

Anyway -- just a thought for the future -- maybe others would use that as well.

The package is great, thanks for all of the work!

Grimson,

I did search. Both via google and in forums and did not see it. Thanks for the link

@ronpfs said in My new pfBlockerNG is showing 100% on the dashboard. That's not right is it?:

With DHCP registration checked, unbound restart with every new lease.
When you save DHCP settings, it also restart unbound. So it "normal" behaviour to see the 100% in the Widget.

At this point I can live without an accurate % as long as nothing else is happening that would cause negative effects. At this point I am going to leave settings where they are and see what happens. Right now I'm getting an accurate measurement of blocking at 22.36% (4,382 of 19,608), and my dns names are showing up in BandwidthD which I expect will disappear at some point and revert to "configure dns to see names" If/when it does I'll decide which is more important, knowing % blocked or resolving names.

Roveer

I agree that the RAID-0 could be considered 'overkill'. This is why I originally was using RAID-1. However, I started to see significant performance degradation. Then I learned that INTEL RAID only supports TRIM on RAID 0, not on RAID 1. So, it was more out of necessity. I suppose I could have had separate non-raid SSDs, but I chose to have a single volume, to keep it simple. The extra performance doesn't help. I'm getting a full 1000 MB/s read/write.

If I were buying new hardware, I would buy ONE NVMe SSD (non RAID), but I have to work with what I have.

After a few weeks with this setup, I've been quite happy with the performance and stability.

Now, I'm trying to fine-tune exactly which feeds I add. The biggest performance hit I see now is when I add too many feeds, or the very large feeds (BBC, hpHosts). I think I'm noticing excessive latency with large lists like those. Since I get very few hits on those lists, I've dropped them for now. I may add them back slowly to see if things change.

@BBcan177 Maybe enable the ability for the web server to also run on IPv6 and add AAAA records.

@isaacfl said in Does pfBlockerNG work in pure ipv6 environment?:

@nogbadthebad

When I nslookup adservice.google.com I get:

Name: adservice.google.com
Address: 10.10.10.1

So there is no AAAA record.

ping adservice.google.com gives me:
Ping request could not find host adservice.google.com. Please check the name and try again

Nothing ever shows in the pfBlockerNg logs though.

So probably not going to work very well in an ipv6 only environment.
My prior adblocker would always respond with both an A and an AAAA record for blocked sites.

I did the steps to no avail.
I have uploaded my unbound.conf and remotecontrol.conf. hopefully you can help me figure out what setting is wrong.
0_1542452701420_conf.zip

Running pfSense 2.4.4-RELEASE (amd64) and latest pfBlockerNG devel and can confirm this same issue.

Example: "192.168.10.10 - blops3 udp port" which appears to be one of my NAT port forward descriptions.

Thanks, didn't know Feodo lists were hosted by abuse.ch, too. Bit sad to read, that a simple dist-upgrade causes multi-day failures... our customers would kill us for that ;)

@rico said in How to install pfBlockerNG if you don't want to upgrade to pfSense v2.4.4:

Shrew Soft is obsolete, there is not even some official Windows 10 Client.
5 year old VPN client is allowed by IT restrictions but no top of the line and free OpenVPN? Weird company...

-Rico

Had the same restriction with an financial sector customer (SAP consultants) and told them the same things. ShrewSoft is allowed for some IPSec dial-ins (with chosen cipher suites that are equally old as the software, 3DES and such) or - even worse - guys that stubbornly told me, that they used PPTP(!) to "VPN into that bank customer" - sometimes reality is more satirical as any magazine/show/internet blog you can imagine ;)

chrome dns page is blank ☹

@johnpoz

Yup

Upgrade looks like it went really smooth!

Disable service. Keep config ticked. Uninstall package. Reinstall package.

Config was retained and looks like it's fully working (even the locally cron downloaded shallalist-lists are still working). To be modified is the custom feeds, migration to a selection from the predefined list now available.
And lots of new features to spend some time with a see as a padawan! 😊

brgs,

Ah, that explains it!

Quick feedback about that:

In a cluster setup I see that as bit of a problem, as you will setup the standby node with all things you have to setup there at first, then activate sync and then hope you have to never touch it again ;)
In pfBNG terms that would mean you have to not only install the package but also configure it the first time the same time as the primary node because otherwise the standby will throw errors because it can't find the aliases the primary uses in its ruleset (e.g. pri1...). That's something that perhaps you should keep in mind. I'd suggest a slightly different approach: In the wizard screen ask if it is installed on a CARP cluster. If so, tell them to install the package on the standby node first but to NOT run the installer, just let it rest. Then proceed with the installer, let them create the DNSBL IP as Alias on a CARP address they created themself so you don't have to deal with CARP at all and then after the setup do the initial update and sync it to slave. :)

With that the first time pfBNG runs its update, all lists are updated, created etc. and you can then sync all to the standby node. Perhaps than trigger a force update there via XMLRPC so the standby node also gets the IP/DNS lists correctly. After that the cron update is set to - an hour per default? - so that it should be somewhat safe to leave it alone and have it sync only every hour. But I'd go like the freeradius package for example and just push the configs to the slave every time you save it on the primary and only let the cron do the list updating on both, not the syncing. But that would be my opinion only :)

upgrading to the dev version seems to have fixed the problem.

Thank you.