Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login
    Introducing Netgate Nexus: Multi-Instance Management at Your Fingertips.

    Squidguard blacklist possible update issue

    Scheduled Pinned Locked Moved Cache/Proxy
    2 Posts 1 Posters 1.3k Views 1 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • JonathanLeeJ Offline
      JonathanLee
      last edited by

      SquidGuard meltdown cause: The reduced blacklist from UT Capitole had a broken nested folder structure that recursively doubled every category, creating hundreds of duplicate database entries instead of 144. This caused massive RAM consumption and the "too many queued redirectors" crash.

      Fix: Switching to the full blacklists_for_pfsense.tar.gz which has the correct flat folder structure resolved the SquidGuard instability completely.

      some of the errors you might see:

      FATAL: Too many queued redirector requests; see on-persistent-overload
      WARNING: All 48/48 redirector processes are busy
      WARNING: 96 pending requests queued
      WARNING: Consider increasing the number of redirector processes
      Squid Cache (Version 5.8): Terminated abnormally
      

      Others once you find it and reload the list are... this was attempted load with everything turned off

      kernel		pid 99405 (squidGuard), uid 0 inumber 9875 on /tmp/squidGuard: filesystem full
      Jun 9 08:24:40	kernel		pid 99405 (squidGuard), uid 0 inumber 9875 on /tmp/squidGuard: filesystem full
      Jun 9 08:24:39	kernel		pid 99405 (squidGuard), uid 0 inumber 9875 on /tmp/squidGuard: filesystem full
      Jun 9 08:24:38	kernel		pid 99405 (squidGuard), uid 0 inumber 9875 on /tmp/squidGuard: filesystem full
      Jun 9 08:24:37	kernel		pid 99405 (squidGuard), uid 0 inumber 9875 on /tmp/squidGuard: filesystem full
      Jun 9 08:24:36	kernel		pid 99405 (squidGuard), uid 0 inumber 9875 on /tmp/squidGuard: filesystem full
      

      Reason:
      This is because the list is doubled for some reason light now this is what loads....

      [blk_blacklists_adult]	access	
      ----
      [blk_blacklists_agressif]	access	
      ----
      [blk_blacklists_ai]	access	
      ----
      [blk_blacklists_arjel]	access	
      ----
      [blk_blacklists_associations_religieuses]	access	
      ----
      [blk_blacklists_astrology]	access	
      ----
      [blk_blacklists_audio-video]	access	
      ----
      [blk_blacklists_bank]	access	
      ----
      [blk_blacklists_bitcoin]	access	
      ----
      [blk_blacklists_blacklists_adult]	access	
      ----
      [blk_blacklists_blacklists_agressif]	access	
      ----
      [blk_blacklists_blacklists_ai]	access	
      ----
      [blk_blacklists_blacklists_arjel]	access	
      ----
      [blk_blacklists_blacklists_associations_religieuses]	access	
      ----
      [blk_blacklists_blacklists_astrology]	access	
      ----
      [blk_blacklists_blacklists_audio-video]	access	
      ----
      [blk_blacklists_blacklists_bank]	access	
      ----
      [blk_blacklists_blacklists_bitcoin]	access	
      ----
      [blk_blacklists_blacklists_blog]	access	
      ----
      [blk_blacklists_blacklists_catalogue-biu-toulouse]	access	
      ----
      [blk_blacklists_blacklists_celebrity]	access	
      ----
      [blk_blacklists_blacklists_chat]	access	
      ----
      [blk_blacklists_blacklists_child]	access	
      ----
      [blk_blacklists_blacklists_cleaning]	access	
      ----
      [blk_blacklists_blacklists_cooking]	access	
      ----
      [blk_blacklists_blacklists_cryptojacking]	access	
      ----
      [blk_blacklists_blacklists_dangerous_material]	access	
      ----
      [blk_blacklists_blacklists_dating]	access	
      ----
      [blk_blacklists_blacklists_ddos]	access	
      ----
      [blk_blacklists_blacklists_dialer]	access	
      ----
      [blk_blacklists_blacklists_doh]	access	
      ----
      [blk_blacklists_blacklists_download]	access	
      ----
      [blk_blacklists_blacklists_drogue]	access	
      ----
      [blk_blacklists_blacklists_dynamic-dns]	access	
      ----
      [blk_blacklists_blacklists_educational_games]	access	
      ----
      [blk_blacklists_blacklists_examen_pix]	access	
      ----
      [blk_blacklists_blacklists_exceptions_liste_bu]	access	
      ----
      [blk_blacklists_blacklists_fakenews]	access	
      ----
      [blk_blacklists_blacklists_filehosting]	access	
      ----
      [blk_blacklists_blacklists_financial]	access	
      ----
      [blk_blacklists_blacklists_forums]	access	
      ----
      [blk_blacklists_blacklists_gambling]	access	
      ----
      [blk_blacklists_blacklists_games]	access	
      ----
      [blk_blacklists_blacklists_good]	access	
      ----
      [blk_blacklists_blacklists_hacking]	access	
      ----
      [blk_blacklists_blacklists_indisponible]	access	
      ----
      [blk_blacklists_blacklists_jobsearch]	access	
      ----
      [blk_blacklists_blacklists_jstor]	access	
      ----
      [blk_blacklists_blacklists_lingerie]	access	
      ----
      [blk_blacklists_blacklists_liste_blanche]	access	
      ----
      [blk_blacklists_blacklists_liste_bu]	access	
      ----
      [blk_blacklists_blacklists_local]	access	
      ----
      [blk_blacklists_blacklists_malware]	access	
      ----
      [blk_blacklists_blacklists_manga]	access	
      ----
      [blk_blacklists_blacklists_marketingware]	access	
      ----
      [blk_blacklists_blacklists_mixed_adult]	access	
      ----
      [blk_blacklists_blacklists_mobile-phone]	access	
      ----
      [blk_blacklists_blacklists_phishing]	access	
      ----
      [blk_blacklists_blacklists_press]	access	
      ----
      [blk_blacklists_blacklists_publicite]	access	
      ----
      [blk_blacklists_blacklists_radio]	access	
      ----
      [blk_blacklists_blacklists_reaffected]	access	
      ----
      [blk_blacklists_blacklists_redirector]	access	
      ----
      [blk_blacklists_blacklists_remote-control]	access	
      ----
      [blk_blacklists_blacklists_residential-proxies]	access	
      ----
      [blk_blacklists_blacklists_sect]	access	
      ----
      [blk_blacklists_blacklists_sexual_education]	access	
      ----
      [blk_blacklists_blacklists_shopping]	access	
      ----
      [blk_blacklists_blacklists_shortener]	access	
      ----
      [blk_blacklists_blacklists_social_networks]	access	
      ----
      [blk_blacklists_blacklists_special]	access	
      ----
      [blk_blacklists_blacklists_sports]	access	
      ----
      [blk_blacklists_blacklists_stalkerware]	access	
      ----
      [blk_blacklists_blacklists_strict_redirector]	access	
      ----
      [blk_blacklists_blacklists_strong_redirector]	access	
      ----
      [blk_blacklists_blacklists_translation]	access	
      ----
      [blk_blacklists_blacklists_tricheur]	access	
      ----
      [blk_blacklists_blacklists_tricheur_pix]	access	
      ----
      [blk_blacklists_blacklists_update]	access	
      ----
      [blk_blacklists_blacklists_verisign]	access	
      ----
      [blk_blacklists_blacklists_vpn]	access	
      ----
      [blk_blacklists_blacklists_warez]	access	
      ----
      [blk_blacklists_blacklists_webhosting]	access	
      ----
      [blk_blacklists_blacklists_webmail]	access	
      ----
      [blk_blacklists_blog]	access	
      ----
      [blk_blacklists_celebrity]	access	
      ----
      [blk_blacklists_chat]	access	
      ----
      [blk_blacklists_child]	access	
      ----
      [blk_blacklists_cleaning]	access	
      ----
      [blk_blacklists_cooking]	access	
      ----
      [blk_blacklists_cryptojacking]	access	
      ----
      [blk_blacklists_dangerous_material]	access	
      ----
      [blk_blacklists_dating]	access	
      ----
      [blk_blacklists_ddos]	access	
      ----
      [blk_blacklists_dialer]	access	
      ----
      [blk_blacklists_doh]	access	
      ----
      [blk_blacklists_download]	access	
      ----
      [blk_blacklists_drogue]	access	
      ----
      [blk_blacklists_dynamic-dns]	access	
      ----
      [blk_blacklists_educational_games]	access	
      ----
      [blk_blacklists_examen_pix]	access	
      ----
      [blk_blacklists_exceptions_liste_bu]	access	
      ----
      [blk_blacklists_fakenews]	access	
      ----
      [blk_blacklists_filehosting]	access	
      ----
      [blk_blacklists_financial]	access	
      ----
      [blk_blacklists_forums]	access	
      ----
      [blk_blacklists_gambling]	access	
      ----
      [blk_blacklists_games]	access	
      ----
      [blk_blacklists_hacking]	access	
      ----
      [blk_blacklists_jobsearch]	access	
      ----
      [blk_blacklists_lingerie]	access	
      ----
      [blk_blacklists_liste_blanche]	access	
      ----
      [blk_blacklists_liste_bu]	access	
      ----
      [blk_blacklists_malware]	access	
      ----
      [blk_blacklists_manga]	access	
      ----
      [blk_blacklists_marketingware]	access	
      ----
      [blk_blacklists_mixed_adult]	access	
      ----
      [blk_blacklists_mobile-phone]	access	
      ----
      [blk_blacklists_phishing]	access	
      ----
      [blk_blacklists_press]	access	
      ----
      [blk_blacklists_publicite]	access	
      ----
      [blk_blacklists_radio]	access	
      ----
      [blk_blacklists_reaffected]	access	
      ----
      [blk_blacklists_redirector]	access	
      ----
      [blk_blacklists_remote-control]	access	
      ----
      [blk_blacklists_residential-proxies]	access	
      ----
      [blk_blacklists_sect]	access	
      ----
      [blk_blacklists_sexual_education]	access	
      ----
      [blk_blacklists_shopping]	access	
      ----
      [blk_blacklists_shortener]	access	
      ----
      [blk_blacklists_social_networks]	access	
      ----
      [blk_blacklists_special]	access	
      ----
      [blk_blacklists_sports]	access	
      ----
      [blk_blacklists_stalkerware]	access	
      ----
      [blk_blacklists_strict_redirector]	access	
      ----
      [blk_blacklists_strong_redirector]	access	
      ----
      [blk_blacklists_translation]	access	
      ----
      [blk_blacklists_tricheur]	access	
      ----
      [blk_blacklists_tricheur_pix]	access	
      ----
      [blk_blacklists_update]	access	
      ----
      [blk_blacklists_vpn]	access	
      ----
      [blk_blacklists_warez]	access	
      ----
      [blk_blacklists_webhosting]	access	
      ----
      [blk_blacklists_webmail]	access	
      

      And if you check the list manually you can see what occurs with the reduced file
      inside of blacklist for this tar file right now is another blacklist a recursive list
      e0647071-e3f4-4178-9702-830f90b2a76d-image.png

      And it has all the same files:

      adedb3f0-9cde-416c-a9dd-f27d584d8d04-image.png

      The other list does not have this issue
      USE THIS ONE!

      http://dsi.ut-capitole.fr/blacklists/download/blacklists_for_pfsense.tar.gz
      

      Use this one :

      1a233b77-d85f-4fdb-a85d-e53b4fb4add4-image.png

      there is just a bad folder in the smaller list right now. Save yourself a headache and swap to the other list, because it is hard to pinpoint this issue it looks like redirector issues

      Make sure to upvote

      1 Reply Last reply Reply Quote 1
      • JonathanLeeJ Offline
        JonathanLee
        last edited by

        YEAH!!! 🎉 The list is officially fixed! A massive thank you to Fabrice Prigent for generously donating his time and expertise to get this sorted for us. We really appreciate your hard work, Fabrice! 🙌 Both work now

        Make sure to upvote

        1 Reply Last reply Reply Quote 2
        • First post
          Last post
        Copyright 2026 Rubicon Communications LLC (Netgate). All rights reserved.