Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Web filtering / blocking - pfBlockerNG DNSBL category memory inquiry - alternative options?

    Scheduled Pinned Locked Moved pfBlockerNG
    2 Posts 2 Posters 1.3k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • M
      mzaknoen
      last edited by

      Looking to block malicious sites on the network (gambling, adult, etc)

      Read about SquidGuard, but it looks like it is being phased out and pfBlockerNG is the way to go.

      Just installed pfBlockerNG DNSBL, did not install the "devel" version, initially I thought it may be development only, but upon reading it looks to be pretty stable, so I can certainly uninstall / re-install.

      XG-2758 (I know, EOL - upgrading soon)

      Intel(R) Atom(TM) CPU C2758 @ 2.40GHz
      Current: 1200 MHz, Max: 2400 MHz
      8 CPUs : 1 package(s) x 8 core(s)

      16GB Ram - memory usage is typically around 8-15%

      100GB Disk space (96% free)

      Looking into setting up category filtering, when checking the "adult" section, I am prompted with the memory warning. Based on my config - thoughts on if I have enough memory to accommodate the download of this giant list? I have read threads/posts on it crashing 8GB systems resulting in the firewall being unusable.

      Dumb question - is there a way for me to download that list of IP's on a computer and copy is over to my Pfsense device to avoid RAM usage of downloading that big file.

      Any alternative options for blocking specific categories? I have read about Cloudflare but haven't done too much research on implementing. Is it a difficult process to setup in Pfsense?

      J 1 Reply Last reply Reply Quote 0
      • J
        jrey @mzaknoen
        last edited by

        @mzaknoen

        Looking to block malicious sites on the network

        generally the issue regardless of what you are trying to do will be based on the list effectiveness

        installed pfBlockerNG DNSBL, did not install the "devel" version

        There is currently no difference and the "non - devel" version is the way to go for most users.

        Looking into setting up category filtering, when checking the "adult" section, I am prompted with the memory warning.

        Memory warning specifically saying?

        What list specifically ? Guessing UT1 -> Adult ? that is something like 4.5 million "domains" with a file size of 122mb but well less than 8gb RAM
        That said the list is also full of bloat.

        is there a way for me to download that list of IP's on a computer and copy is over to my Pfsense device to avoid RAM usage of downloading that big file.

        yes, but downloading and using ram based on what was downloaded are different things.

        Any alternative options for blocking specific categories?

        yes,
        everything from education of users
        to block everything and move to allow certain sites only
        and anything in between.

        there is a balance and that will be different for every use case.

        consider the following sample from the UT1 adult list as I suspect that is what you are running into problems with.

        Ask yourself, do I have a need for anything on blogspot.com?
        Yes, write it down (you will want to whitelist that website(s) if you do)
        No, nice, just continue

        UT1 - Adult domains (raw file) 4,511,799 122mb
        remove all blogspot lines (raw file) is now 891,692 domains and 18mb
        add 1 line containing blogspot.com to the TLD list
        ( a quick DNS scan to others for example blogspot.hr are all cname or redirects to the .com)

        Okay, i have not even tried to load the list, it is not a list I would ever consider using. There are other ways with far less impact.

        However, for the purpose of this example I added only blogspot.com to a TLD for testing. I grabbed one of the URLs from the list (bad me)

        then over to a browser. Don't try this at home kids🤣

        Screen Shot 2023-12-28 at 3.40.32 PM.png

        immediately gets the redirect

        Screen Shot 2023-12-28 at 3.42.44 PM.png

        and in the log we see the original request getting the cname reply
        and the website being blocked by the 1 line added to the TLD
        Screen Shot 2023-12-28 at 3.44.57 PM.png

        A second scan of the original domain list, shows that many of the names don't even resolve, so those are just old and could also be removed.

        Size of list does not equate to "effectiveness" of the list -- also applies to any/all of the available lists.

        If the math is correct I've removed 3,620,107 lines from the file, and effectively have the same blocking with the addition of 1 line. Of course I'm not going to try every single one, although it would be easy enough to script a test.
        This is where the user education can come into play, why on earth would you be going to a website like

        zxaswdserdwokgkmbjnhntbftherhbfokmlplfnvhrfdx.(some TLD)
        

        certainly not by typing that address in.

        Often it is better, to determine what needs to be blocked specifically by reviewing logs. Do I use lists, certainly do. But certainly no need to hit the finishing nail with a sledge hammer.

        Memory is pretty flat lined here - holding at
        Screen Shot 2023-12-28 at 4.28.09 PM.png

        1 Reply Last reply Reply Quote 0
        • First post
          Last post
        Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.