How To: Manually create SquidGuard blacklist from EasyList, etc.

mirkwoody

Here is how you can create custom blacklists from popular list already on the internet. I've asked a lot of questions here, now I want to try and add something back.

It is by no means a definitive guide on "How to", but gives some idea. Originally I wanted something automated but to me it seems none have successfully gotten that to work 100%.
.. There will be a lot of Searching and replacing in NotePad++.

If anyone has some ninja skills and ideas of how to make this easier, then please tell.

Step 1. - Find a list you want.
You could for example find some here https://filterlists.com/. Click "View" and copy all or parts of it.
Sometimes lists like "EasyList" can be cumbersome and you only want parts of the list, then you can head to https://github.com/easylist/easylist and find only the parts you want.

Step 2. - Download "Notepad++"
Link here: https://notepad-plus-plus.org/
It will be much easier and faster to work with than for example Word.

Step 3. - Copy & Edit.
Copy the stuff you want into NotePad++.
For SquidGuard to accept you list you need to have a simple list with domain followed by a space "Google.com " for example, not "?Google.com=what." or something odd.

Some lists are simple and easy, If it is then you can maybe skip to end of this Step.
Other list are not, let's take example with some mess it won't accept.

||zugo.com^$third-party
||z???waar.org^$third-party
||zxxds.net^$third-party
foengeri.
f34f3.
f43..sfs.
||zyiis.net^$third-party
||zypenetwork.com^$third-party
! adsterra

Open the Replace tool by pressing [Ctrl + H]
Find: "||"
Replace with: … nothing.
Replace all.

Do this with everything you can, also "^$third-party".
Also remove the double dot
Find: ".."
Replace with: "."

You should get something like this..

zugo.com
z????waar.org
zxxds.net
foengeri.
f34f3.
f43.sfs.
zyiis.net
zypenetwork.com
! adsterra

To remove the "! adsterra" line use…
Find: "!.*$"
Replace: nothing.
Search mode: Regular expression.

So search for everything invalid and delete accordingly. Same with anything like "_ ? = ( ) & # , * ..." any special character and symbol.

Now you have this.

zugo.com
zwaar.org
zxxds.net
foengeri.
f34f3.
f43.sfs.
zyiis.net
zypenetwork.com

Now there are top domains missing "foengeri." for example. (This can be a pest to find in large lists)

Select "Regular expression" at the bottum on replace tool window.
Find "$"
Replace: " " .. with a space.. you will now add a space behind everything
Click replace all.

Now switch it back to "Normal" search mode.
Find: ". " dot followed by space.
Replace with ".com" or other… maybe just delete since you wont know which top domain was suppose to be there.

..

Now Find " " all spaces
Replace with nothing to remove all the spaces you made.

...

Now add a space behind everything again. Switch to "Regular expression" again,
Find: "$"
Replace with: " " a space (So that SquidGuard can understand)

zugo.com 
zwaar.org 
zxxds.net 
zyiis.net 
zypenetwork.com

Now there should be a space behind every domain.

Step 4. - Insert into SquidGuard and test

Services - SquidGuard Proxy Filter - Target catagories - "+ Add"
Type name and copy text into "Domain list"
Hit Save… (If all is well it lets you save.. otherwise it tells you what is wrong. If only a few errors, then just do in browser search, usually Ctrl + F)
Go to "Common ACL", hit the "+" at target rule list, and choose to "deny" or other.
... Apply and save in General settings tab.

Important tip/note

Don't spend hours looking thru a list before trying to save, SquidGuard will tell you exactly whats wrong in the list, and you can easily search for the incorrect parts in NotePad++.

….

A lot of this could probably be automated more, and there are many search and replace options I am not familiar with.
Also.. when it is not auto updating it's maybe not the best, but worth some.

=== Useful regular expressions in Notepad++ ===

Wiki how to: http://docs.notepad-plus-plus.org/index.php/Regular_Expressions

$ End of line.
^ Front of line.
/.$ All after test.com/
\b(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(.(?1)){3}\b Remove IPv4 IP adresses.
[.

to something else if you like… < > ( )...

^.{0,3}$ Any line shorter than 3 ...
^(.?)$\s+?^(?=.^\1$) Duplicate finding and replacing.

Remove empty lines: Edit > Line Operations > Remove empty lines.