Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    DNSBL Top1M TLD Inclusions not Saving / Restoring

    Scheduled Pinned Locked Moved pfBlockerNG
    6 Posts 2 Posters 221 Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • bartkus05B
      bartkus05
      last edited by

      Hello,

      Any known issues with TLD Inclusions not saving on DNSBL TOP1M? Highlighting the options and saving doesn't do anything. Manually adding "selected" attribute to the <option> elements also doesn't work.

      pfsense+ 24.11
      pfblockerNg 3.2.0_16

      This wasn't happening when I was using pfsense CE 2.7.2 before upgrading yesterday when troubleshooting WAN issues.

      I tried changing it on Edge, Chrome and Firefox (if it was a browser issue), then tried creating a backup of packages, added the alexa_enable and alexa_inclusion to the config, restored and still not working, no inclusions 😢

      For now, I turned off most of the lists, as it was practically blocking everything.

      Thanks!

      GertjanG 1 Reply Last reply Reply Quote 0
      • GertjanG
        Gertjan @bartkus05
        last edited by

        @bartkus05

        A white list blocking everything ?

        Never used it before but just activated it :

        Accepted the defaults
        ,
        But not Alexa as it's out of date / depreciated.
        And just the top 1000 (and not the top 1M as one million domain will break your router. (1M host names needs dozens of Gbytes of RAM, and you DNS system will be way to slow)

        It loads just fine :

        a49513a2-68c2-4078-b822-622afcc62014-image.png

        9fd7a75f-b8da-4eaf-a632-64621feabfc2-image.png

        No "help me" PM's please. Use the forum, the community will thank you.
        Edit : and where are the logs ??

        bartkus05B 1 Reply Last reply Reply Quote 0
        • bartkus05B
          bartkus05 @Gertjan
          last edited by

          @Gertjan

          I use the Top 10k option from Tranco, however without the TLD inclusions, it never builds the db so no whitelist gets applied at all. Regular DNSBL whitelist works perfectly fine.

          af5592b3-d14b-431d-aa55-80fcba3b515a-image.png

          Trying this with the Devel package, its the same

          b31132b8-0c65-4cdd-9420-fd11b6c4b3b2-image.png

          Its like this for all of them, and previously it kept finding up to 100 items to put on the whitelist, for example, googles adclick URLs.

          Also, the same exact issue with 1M option:

          254c144d-b576-4fb0-9268-e5bc005360b2-image.png

          0157ee31-4663-46de-99e9-fbeb7ec38c2a-image.png

          Its like, it refuses to apply anything to the whitelist without the TLDs specified. I have tried doing a fresh start on pfblocker, but no difference on that either. I have the option enabled on each of the groups as well.

          3230f32b-aa53-47c7-baba-f807a6c256ab-image.png

          GertjanG 1 Reply Last reply Reply Quote 0
          • GertjanG
            Gertjan @bartkus05
            last edited by Gertjan

            @bartkus05

            I went for the 10k list.
            I blew right through it, no issue what so ever

            24c144e1-f1a2-4f77-bb64-a17c96fa9865-image.png


            edit :
            Maybe I can't see the issue.

            a9bd376d-9022-45ed-80d3-a795256bc146-image.png

            I just activated this TOP1M Whiltelist option, but ..... :

            1361d8ac-49a3-4eca-8200-96a1a9700720-image.png

            and I checked related files - I think it's these two :

            8715fb60-03a9-480d-8715-5a8428715cdb-image.png

            and they date from last novembre 2024 ?!
            These to files will get updated ones per month (when the option is activated) ?

            I have this impression that 'my) pfBlockerng doesn't download anything (as the top 1000+ sites doesn't really change every day any way).

            The "top-1m.csv" has 1M host names in it, and is waaaay smaller as I thought.
            You have the same files ? same date ?

            No "help me" PM's please. Use the forum, the community will thank you.
            Edit : and where are the logs ??

            bartkus05B 1 Reply Last reply Reply Quote 0
            • bartkus05B
              bartkus05 @Gertjan
              last edited by bartkus05

              @Gertjan

              I have the latest file, compared it with one downloaded directly from Trancos' website

              491bc40b-f473-4ab3-b190-126433dbb1f7-image.png

              So it downloads the file just fine, but then does nothing with it, and reading all the information on this - it only adds domains from downloaded file into the whitelist when TLD Inclusions are selected, but the problem is - I am not able to. When I highlight any of them, click save, it blanks out again, and I tried setting it with the XML config too (modified the XML to add the TLDs I want) and that doesn't pull through either.

              e3d6a786-5a89-4238-9893-64201afa1fe3-image.png

              ec2164c4-d266-4c73-a9fa-75097a7b9407-image.png

              EDIT:

              I have managed to force those in now, when doing a restore of configuration, they get stripped out - however when I modified the config.xml manually, added those values in they stick and allow the top1m list to be processed!

              26c6bfd8-cf0d-475c-88c6-a195d20887be-image.png

              Now, after those have been added, I am unable to edit them, remove them at all. Looks like this might either be something being deprecated, in process of being changed or a genuine bug with pfBlockerNG.

              EDIT 2:

              35a936b2-21fb-4e1e-88c6-5ba58382e2fa-image.png

              Got it fully working now

              GertjanG 1 Reply Last reply Reply Quote 1
              • GertjanG
                Gertjan @bartkus05
                last edited by

                @bartkus05 said in DNSBL Top1M TLD Inclusions not Saving / Restoring:

                Got it fully working now

                👍

                My files also got auto updated after I've left work.

                f3940fb8-86dc-4761-9931-3dbc5cdb0cb6-image.png

                Be ware that "Alexo" might be gone soon, as it isnt' maintained anymore.

                I guess this "TOP1M Whitelist" option isn't used a lot .... so not well tested against "all possible usage modes".

                No "help me" PM's please. Use the forum, the community will thank you.
                Edit : and where are the logs ??

                1 Reply Last reply Reply Quote 0
                • First post
                  Last post
                Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.