shalla list off

JonathanLee · Apr 23, 2022, 8:02 PM

@waqar-uk Dang The original version I used I had to reimage after it installed it broke my system. I got this link from someone else I have not tested it yet because its exam week

stephenw10 · Apr 23, 2022, 9:27 PM

@jonathanlee said in shalla list off:

http://dsi.ut-capitole.fr/blacklists/download/blacklists_for_pfsense_reducted.tar.gz

The file downloads fine for me. I haven't tried it in Squid.

Waqar.UK · Apr 24, 2022, 5:57 AM

@stephenw10 said in shalla list off:

http://dsi.ut-capitole.fr/blacklists/download/blacklists_for_pfsense_reducted.tar.gz

Now it works for me. I wonder why before it did not?

Does it work in Suricata?

JonathanLee · Aug 3, 2022, 6:06 PM

@waqar-uk

This works!!!
I just tested this after the summer semester has been completed!!

login-to-view
(Image: Blacklist loading)

login-to-view
(Image: Blacklist Loaded)

login-to-view
(Image: SquidGuard Catagories restored

nimrod · Aug 3, 2022, 6:47 PM

@jonathanlee said in shalla list off:

@waqar-uk

This works!!!
I just tested this after the summer semester has been completed!!

login-to-view
(Image: Blacklist loading)

login-to-view
(Image: Blacklist Loaded)

login-to-view
(Image: SquidGuard Catagories restored

UT1 is not working in pfBlockerNG because its using this link:

 ftp://ftp.ut-capitole.fr/pub/reseau/cache/squidguard_contrib/blacklists.tar.gz

First i thought its the issue with pfBlockerNG, but if i try to manually download this file from my desktop machine by using wget, this is what i get:

 wget ftp://ftp.ut-capitole.fr/pub/reseau/cache/squidguard_contrib/blacklists.tar.gz
--2022-08-03 20:27:53--  ftp://ftp.ut-capitole.fr/pub/reseau/cache/squidguard_contrib/blacklists.tar.gz
           => ‘blacklists.tar.gz’
Resolving ftp.ut-capitole.fr (ftp.ut-capitole.fr)... 193.49.48.249
Connecting to ftp.ut-capitole.fr (ftp.ut-capitole.fr)|193.49.48.249|:21... connected.
Logging in as anonymous ... Logged in!
==> SYST ... done.    ==> PWD ... done.
==> TYPE I ... done.  ==> CWD (1) /pub/reseau/cache/squidguard_contrib ... done.
==> SIZE blacklists.tar.gz ... 30615325
==> PASV ...

And thats where its stuck. I edited /usr/local/pkg/pfblockerng/ut1_global_usage and changed the link to point to

https://dsi.ut-capitole.fr/blacklists/download/blacklists.tar.gz

It gets downloaded without any issues, but it fails to install. I downloaded blacklists.tar.gz manually and unpacked it with tar, and there is another tar archive inside. pfBlockerNG extracts the content of the archive and expects folders containing category lists, but instead, its just another archive and thats where it fails.

I can extract the initial archive, and upload extracted tar file somewhere else, and update the link in ut1_global_usage file, and that works. But that needs to be done very time there is a update.

Is it safe to say that both Shallalist and UT1 are dead on pfBlockerNG ?

JonathanLee · Aug 3, 2022, 7:23 PM

@nimrod http://dsi.ut-capitole.fr/blacklists/download/blacklists_for_pfsense_reducted.tar.gz

Try this it works for me and I have lists again inside of squid guard. The website has support I have emailed to have URLs added and they add them also.

This was the same issue I had it would download and not install, this list works it ends with reducted. Yes shalla list is gone, I wanted to save the last list that was available but I didn't get to in time.

Squidguard is different than PfblockerNG

nimrod · Aug 3, 2022, 7:29 PM

@jonathanlee said in shalla list off:

@nimrod http://dsi.ut-capitole.fr/blacklists/download/blacklists_for_pfsense_reducted.tar.gz

Try this it works for me and I have lists again inside of squid guard. The website has support I have emailed to have URLs added and they add them also.

This was the same issue I had it would download and not install, this list works it ends with reducted. Yes shalla list is gone, I wanted to save the last list that was available but I didn't get to in time.

Squidguard is different than PfblockerNG

I know.

But i dont use Squidguard. I use pfBlockerNG. And these lists are not working in pfBlockerNG. That was the whole pont of my post.

JonathanLee · Aug 3, 2022, 7:39 PM

@nimrod this list states PfSense

nimrod · Aug 3, 2022, 8:27 PM

@jonathanlee said in shalla list off:

@nimrod this list states PfSense

Yes. And that can be anything from pfBlocker, to Snort, Suricata or Squidguard.

JonathanLee · Aug 4, 2022, 1:39 AM

@nimrod

Hello, this list is different, and does in fact work for Squidguard inside of a Netgate 2100 Max running PfSense software version 22.05. I just tested URL shorteners and it shows blacklist blocked. I understand that you are only using PfBlockNG. However I noticed the items your testing show a different blacklist URL over the URL that I have provided. Please note, "Squidguard in Pfsense has a problem with the size of the database. pfblockerNG (a specific package or pfsense) might be the solution: an explanation" (Per BlackList Website).

I am sorry I thought this URL would also work for the other package as it is not branded as a specific package, only branded as PfSense and marked as reduced for home firewalls to provide a fix for the size of the database issue.

URLs that you showed for your post are that do not work for me also:
https://dsi.ut-capitole.fr/blacklists/download/blacklists.tar.gz

ftp://ftp.ut-capitole.fr/pub/reseau/cache/squidguard_contrib/blacklists.tar.gz

The above URLs also did not work on my Pfsense with Squidguard, they would not load because of the database size issue.

URL that works for my Pfsense:

http://dsi.ut-capitole.fr/blacklists/download/blacklists_for_pfsense_reducted.tar.gz

I thought it was worth posting this URL as it might work for your package PfBlockNG. This is a different URL.

login-to-view

(Image: URL Blacklist running successful blocks for websites by category blocks provided by the URL above)

nimrod · Aug 4, 2022, 2:24 AM

@jonathanlee said in shalla list off:

@nimrod

Hello, this list is different, and does in fact work for Squidguard inside of a Netgate 2100 Max running PfSense software version 22.05. I just tested URL shorteners and it shows blacklist blocked. I understand that you are only using PfBlockNG. However I noticed the items your testing show a different blacklist URL over the URL that I have provided. Please note, "Squidguard in Pfsense has a problem with the size of the database. pfblockerNG (a specific package or pfsense) might be the solution: an explanation" (Per BlackList Website).

I am sorry I thought this URL would also work for the other package as it is not branded as a specific package, only branded as PfSense and marked as reduced for home firewalls to provide a fix for the size of the database issue.

URLs that you showed for your post are that do not work for me also:
https://dsi.ut-capitole.fr/blacklists/download/blacklists.tar.gz

ftp://ftp.ut-capitole.fr/pub/reseau/cache/squidguard_contrib/blacklists.tar.gz

The above URLs also did not work on my Pfsense with Squidguard, they would not load because of the database size issue.

URL that works for my Pfsense:

http://dsi.ut-capitole.fr/blacklists/download/blacklists_for_pfsense_reducted.tar.gz

I thought it was worth posting this URL as it might work for your package PfBlockNG. This is a different URL.

login-to-view

(Image: URL Blacklist running successful blocks for websites by category blocks provided by the URL above)

None of those files in that web folder will will work with pfBlockerNG because they are all tar archives inside the tar archive. Such format is not supported.

dauhee · Jan 3, 2023, 12:36 AM

@nimrod said in shalla list off:

http://dsi.ut-capitole.fr/blacklists/download/blacklists_for_pfsense_reducted.tar.gz

http://dsi.ut-capitole.fr does not appear to be alive any more

JonathanLee · Jan 3, 2023, 12:58 AM

@dauhee It seems to be working still for me in California

login-to-view
(Image: Testing blacklist)

login-to-view
(Image: Download complete)

Email: fabrice.prigent@ut-capitole.fr
he manages this blacklist if you are having issues, I recently emailed a update for DoH addresses to add to the blacklist he is returning to check emails Jan 3rd I think the auto response said.

JonathanLee · Jan 3, 2023, 1:03 AM

@dauhee said in shalla list off:

http://dsi.ut-capitole.fr

login-to-view

It seems to be working email that email address if you want to add URL items.

I recently emailed to add DoH

JonathanLee · Jan 3, 2023, 4:31 PM

@dauhee

https://forum.netgate.com/topic/176693/dns-over-443/6?_=1672707245355

I was having major issues with DoH bypassing my DNS and just doing whatever it wanted port hopping to DNS on HTTPS. If you want, I can send you that blacklist and you just have to use an Apache server to download it on pfSense.

Link to a working url blacklist for pfsense squidguard package:

Squidguard Package: I saved the download of the open-source software just in case zip of tar gz file
squidGuard-1.3.tar.zip

Google Drive Link to a working URL blacklist for Pfsense Squidguard package that I use for a 2100 Max:
https://drive.google.com/file/d/1KsE8H5Q_wa7cT2knU0iQynCQ_A_xPIVc/view?usp=share_link

The official source of the blacklist listed above:
http://dsi.ut-capitole.fr/blacklists/index_en.php

Keep in mind the list above is the pfsense reducted version for lower memory firewalls:
http://dsi.ut-capitole.fr/blacklists/download/blacklists_for_pfsense_reducted.tar.gz

Loading this, you could load this blacklist on the Linux box. Just type at a prompt:

service apache2 start

make sure you copy the blacklist over to /var/www/foldernamehere
store that blacklist inside that folder.

now use the ip address of the linux box with http://192.168.1.10/foldernamehere/blacklistfile

Now download it into the firewall box with your custom apache server.

jdeloach · Jan 3, 2023, 4:49 PM

@jonathanlee

Just an FYI, I know this is not a popular option, but myself I have updates turned off for Shallalist and UT1 but still have the list enabled. All the files are stored on your computer unless you deleted them.

It is outdated but it still catches several hundred hits for the categories that I have selected. I think it is given that there is probably not a replacement for this list so you either use the outdated list as is or you just disable it and move on. Folks have been complaining about this for a year now so it is time to let it die. The world is not going to end if this list does not get resurrected.

If we, the users, can't find a replacement for this list, I think it is unfair to expect the pfBlockerNG maintainer to pull a replacement for it out of his ass.

JonathanLee · Jan 3, 2023, 4:57 PM

@jdeloach Do you possibly still have the shalla list in .tar.gz? I want to save it. We should make a github with the lists for use with both squidguard and pfblocking.

jdeloach · Jan 3, 2023, 5:58 PM

@jonathanlee said in shalla list off:

@jdeloach Do you possibly still have the shalla list in .tar.gz? I want to save it. We should make a github with the lists for use with both squidguard and pfblocking.

You probably already have these files on your pfsense computer. Check the following directories for the compressed files:
/var/db/pfblockerng/shallalist.tar.gz
/var/db/pfblockerng/ut1.tar.gz

The uncompressed files in following directories:
/var/db/pfblockeng/shallalist/(uncompressed category files)
/var/pfblockerng/ut1/(uncompressed category files)

The above files should be on anyone's computer that has ever had shallalist and ut1 enabled in pfBlockerNG. Keep in mind these files will be outdated.

JonathanLee · Jan 3, 2023, 6:09 PM

@jdeloach I had to format my system shortly after shalla went down :(

dauhee · Jan 4, 2023, 1:02 AM

@jonathanlee oh wow, thank you very much. yes http://dsi.ut-capitole.fr/blacklists/download/blacklists_for_pfsense_reducted.tar.gz is working for me now. it was late at night and maybe I didn't check the link properly or in the middle of me setting up squid proxy I did something wrong

really appreciated for the information