DNS Blacklist, New Package! Check it out.

Supermule

Nice….Not superurgent, but a small feedback to you.

:)

I suppose one can update the package via an uninstall....

jigpe

Thank you xa0z! Great job :)

Thanks for the heads up!

jigp

keeper 0

thanks, more power to your team guys :)

vreid473

Hello,

Thanks for implementing this package. I'm trying it out , and I think it has a nice feature set and is super easy to configure. I have a few suggestions for additions:

1. The ability to add or remove domains and url's to a whitelist and to a blacklist.
2. The ability to specify the web page url to which users are redirected for each category.
3. The ability to specify the blacklist to use. Or, alternatively, some documentation about what's on the existing blocklist.

I noticed that the dns blacklist package mentions porn in the notes at the top of the page, but there appears to be not porn category (at least on the embedded machine on which I'm doing my initial testing). Is this an error or a feature to reduce the footprint on embedded devices?

I appreciate the time and effort that you have placed into creating this package. I think it definitely fits the need for an additional filtering plugin for PFSense. :)

Vaughn Reid III

xa0z

Thanks for the comments. The next version of this script does allow users to add their own custom blacklist entries. As of right now there is no way to specify the URL that a blacklisted item is sent to, that is why it is forwarding to a Google IP. Doing the method you are requesting requires a proxy. This is being looked into for the future.

As for being able to select your own blacklist database, I have already thought of this and think it would be a great idea but right now the scripting is ~~static~~ …it was static, it is dynamic now... to use preset categories/directories for the database. In the near future I would like to have this set to load the categories from a variable after reading the available directories containing the blacklists.

In a beta version of DNS Blacklist we had two separate categories… "Porn" and "Adult". That blacklist database is not what we're using right now so the categories are different and now contained under the same category. The configuration page still lists the two categories in the NOTE because I forgot to remove it, just a self error.

We're going to release a "fixed" version here soon, but a major release with additions will be a little later.

cybrsrfr

Did some improvements to the package today.
1. wording changes.
2. uninstall has been fixed
3. all categories can be deselected.

You can edit categories by using pfSense's command page:
Diagnostics -> Command

ls /usr/local/www/packages/dnsblacklist/blacklists

Lets say you wanted to edit the 'adult' list, you can download the list using:
File to download -> /usr/local/www/packages/dnsblacklist/blacklists/adult/domains

You would then edit the file using an editor that deals well with large files.
If you are on windows don't use the standard notepad.exe instead use pspad or notepad++

To upload the file from the command page use:
File to upload

The file will be uploaded to the /tmp directory.

Then use 'Execute Shell command' to cp the file to the correct directory.
cp /tmp/domains /usr/local/www/packages/dnsblacklist/blacklists/adult/domains

If you follow the folder structure consistently you can add additional categories.
You also would need to carefully edit the /usr/local/www/packages/dnsblacklist/blacklists/global_usage file which list the categories and provides the description for the categories. You can edit the global_usage file using Diagnostics -> Edit File

xa0z

Tomorrow I will begin working on our own Blacklist Database. It will work in the same manor as current… We will have a blacklist directory, and within that directory we will have a directory with the respective category name, and then within that directory we have the "domains" file which contains the domains that will be added to the active blacklist when that category is selected.

Today mcrane did a great job in updating the code to make sure everything was working as normal as we can. Remember we're still in BETA so give it some time, make your requests for addons, report any bugs you might see and we will do our best to make sure things go as smooth as possible.

As for right now the only concern I have is making sure we can get a really good, clean, host-database for the blacklisting.

cybrsrfr

The current list comes from here:
http://cri.univ-tlse1.fr/blacklists/index_en.php

Its just not as large as xa0z would like.

xa0z

The size isn't what bothers me… I think that a proper database should contain categories that are more limited to specific matching items, and better defined categories at that.

That was the whole point of me starting this project.

cybrsrfr

Here is the original blacklist that was going to be used.
http://urlblacklist.com/

I didn't use it in the final version of the package because I didn't feel that redistribution of that list was the ethical thing to do when the website that provides the list is trying to sell it. They provide the entire list for download and payment is on an honor basis as stated on their website. Their intent is really to sell it for a subscription. I did not notice this when I started building this package. When I noticed they were trying to sell the list they had compiled I could not with good conscious redistribute urlblacklist.com's list. If someone wants to use that list and pay for the subscription they can do so by editing the domain lists and adding the categories as I described in an earlier post.
http://urlblacklist.com/?sec=subscribe

Here is a few more choices of DNS blacklists.
http://www.squidguard.org/blacklists.html

Anyone can edit the list that the package comes with. In order for an alternative customized list to be used in the DNS blacklist package it must be compiled ethically and legally or I will not commit it to the package. For example it wouldn't be ethical to use lists from urlblacklist.com or other lists that don't offer the list in freely. Unless you obtain permission directly from them to use the list for such a purpose.

The list from http://cri.univ-tlse1.fr/blacklists/index_en.php says on their website 'can be used with many commercial or free software' therefore it is ethical to use their list for the DNS Blacklist package.

xa0z

Hmm, so I guess there isn't a need for me to continue on with the project. It is just pointless for me to try and do one thing and then have it all changed to not be the way it was when I started it.

I already said I was going to build my own database list but what's the point, I don't need to supply something that anyone can just grab from all over the net to use.

I'm glad for the help I got when I started writing the project but if I am being phased out, which it so seems I am, then I guess my input, or work is useless.

Supermule

If this isn't supported and maintained, pls. inform me how to delete it completely! It wont go away…..any way I do it.

xa0z

Okay, what you need to do is "Remove" the package like normal… then Re-Install it like normal. Then the new version has the fixes to take care of the problems you're having. You can then remove it to completely remove it, or you can keep using it.

My only problem is that this is starting to move a direction in which I don't seem to have anymore say so in the project. I'm not saying anything bad, and I'm not about to put mcrane down, he's a great guy and has helped me a great deal with stuff, but I just wish the project I started, and wrote part of... I had some control over.

Supermule

That is quite understandable…..:)

You have my support for the project, but it seems that the way the list is generated and the source of the data, is the big hurdle???

xa0z

I want to compile my own database. I want to get it as simple as possible where specific rated categories are available for certin things. I want to remove all the IPs in the database and use only hostnames (for now) to help keep it clean. Web Browsers don't do reverse lookups so having all those extra useless items is a waste of RAM.

I will try my best to keep with the program, just want to make sure it stays on track. I mean the whole point to me starting this was to help out, and if it's not helping me, how can it help others?

Supermule

Exactly. :)

Keep it up! You are on the right track….IP's change. Domainnames do not as often....So it is a good argument.

cybrsrfr

The problem is the list. As I stated before it has to be created ethically. That is compiled from free lists where the owners give their permission.

I will not commit something to pfSense that makes me an accessory to stealing. That is why I refused to commit the package with the previous list. If I did this I would expect to get my commit authority revoked.

It is not impossible to find lists that are free for any use and supplement them, and improve them with your own domains you and others find while searching.

xa0z

That is exactly what I'm doing. I'm not worried about the urlBlacklists database anymore. I am just saying that I'm getting the database taken care of on my own, and it will be greatly categorizied.

Davc

I think this is a good project, if the DNS Blacklist is good and effectively use. Surely there will be people supported and willing to donate some subscription fee to maintain the Blacklist and the project itself in this Pfsense forum/members.

I am no expert in here, but there are people who might (already) using OpenDNS to filter, as long as this project don't run the same track as the OpenDNS or able to offer more than OpenDNS. I think there are light on this projects.

Good Work !! ;)

jan.gestre

Hi Davc,

I've tried DNS blacklist in one of my pfSense box but it broke the dns forwarder service, after installation the dnsmasq service stopped and can't be restarted even after repeated tries and reboot. I uninstalled the aforementioned package but I still can't start the dnsmasq service,I had to examine other working boxes to see what files have been added or changed by the DNS blacklist package, it added the dnsmasq.conf which I've deleted and my dnsmasq service finally started.

Did I missed something?

Regards,

Jan