Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    New pfblockerNG install Database Sanity check Failed

    Scheduled Pinned Locked Moved pfBlockerNG
    64 Posts 20 Posters 16.5k Views 18 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • J Offline
      JonH @tinfoilmatt
      last edited by

      @tinfoilmatt I did a quick scan of this thread. I'm on 25.07.1 Release and pfblockerng-devel 3.2.10
      I have this same error. I will go back and re-read this thread but FWIW the update mentioned earlier certainly didn't fix it for me.

      J 1 Reply Last reply Reply Quote 0
      • J Offline
        JonH @JonH
        last edited by

        OK, rather than fiddle with editing the file I tried the method mentioned by @Laxarus. It worked perfectly.

        Database Sanity check [ PASSED ]

        Masterfile/Deny folder uniq check
        Deny folder/Masterfile uniq check

        Sync check (Pass=No IPs reported)

        Thanks to all who contributed ideas.

        1 Reply Last reply Reply Quote 0
        • D Offline
          Draco @BBcan177
          last edited by

          @BBcan177 I am runnng pfSense 25.07.1-RELEASE (amd64) on an netgate 5100 box, and pfBBLockerNG-devel v3.2.10. My counts have been off by anywhere from a few dozen to a few thousand.

          I checked /usr/local/pkg/pfblockerng/pfblockerng.sh and line 1232 is already

          s1="$(grep -cv ^${ip_placeholder2}$ ${mastercat})"
          

          A sampling of teh log info around the sanity check after a forced reload:

          ===[ DNSBL Domain/IP Counts ] ===================================
          
            889529 total
            602876 /var/db/pfblockerng/dnsbl/Maltrail_BD.txt
             84614 /var/db/pfblockerng/dnsbl/StevenBlack_ADs.txt
             64269 /var/db/pfblockerng/dnsbl/SFS_Toxic_BD.txt
             40604 /var/db/pfblockerng/dnsbl/EasyList.txt
             40159 /var/db/pfblockerng/dnsbl/EasyPrivacy.txt
             15210 /var/db/pfblockerng/dnsbl/MS_2.txt
             10751 /var/db/pfblockerng/dnsbl/Abuse_urlhaus.txt
             10199 /var/db/pfblockerng/dnsbl/SWC.txt
              9076 /var/db/pfblockerng/dnsbl/Exch_AD_Servers.txt
              6101 /var/db/pfblockerng/dnsbl/Adaway.txt
              2793 /var/db/pfblockerng/dnsbl/Yoyo.txt
              2344 /var/db/pfblockerng/dnsbl/Easylist_Firebog.txt
               235 /var/db/pfblockerng/dnsbl/OpenPhish.txt
               229 /var/db/pfblockerng/dnsbl/NoCoin.txt
                39 /var/db/pfblockerng/dnsbl/MoneroMiner.txt
                30 /var/db/pfblockerng/dnsbl/Manual_BL_custom.txt
          
          ====================[ IPv4/6 Last Updated List Summary ]==============
          
          May 17	2024	Spamhaus_eDrop_v4
          May 17	2024	ARIN_MSFT_ASNs_v4
          May 17	2024	Whitelist_custom_v4
          Aug 14	2024	GreatWallDoH_v4
          Jan 3	2025	Abuse_SSLBL_v4
          Jan 17	2025	Talos_BL_v4
          Jul 31	00:17	MSFT_IPBlocks_v4
          Nov 1	15:08	Spamhaus_Drop6_v6
          Nov 13	21:30	ET_Block_v4
          Nov 14	05:39	Spamhaus_Drop_v4
          Nov 14	13:39	ET_Comp_v4
          Nov 14	15:55	BDS_Ban_v4
          Nov 14	23:48	TOR_Exit_Nodes_v4
          Nov 15	00:03	Public_DNS6_v6
          Nov 15	00:03	DNSServers_v4
          Nov 15	08:13	SFS_IPs_7day_v4
          Nov 15	08:15	HoneyPot_Bad_v4
          Nov 15	08:15	PubMatic_v4
          Nov 15	10:31	CINS_army_v4
          Nov 15	11:45	ISC_Block_v4
          Nov 15	12:03	DNSServers2_v4
          Nov 15	12:10	Abuse_Feodo_C2_v4
          Nov 15	12:11	BotScout_v4
          Nov 15	12:53	pfB_Top_v4
          Nov 15	12:53	pfB_Top_v6
          Nov 15	12:53	PRI1_custom_v4
          
          ====================[ DNSBL Last Updated List Summary ]==============
          
          Apr 30	2023	Adaway
          May 17	2024	Exch_AD_Servers
          Jan 21	2025	MoneroMiner
          Mar 6	2025	NoCoin
          Oct 6	15:16	MS_2
          Oct 28	09:09	SWC
          Nov 12	00:16	StevenBlack_ADs
          Nov 14	05:42	Yoyo
          Nov 14	06:00	Easylist_Firebog
          Nov 14	23:59	SFS_Toxic_BD
          Nov 15	00:04	EasyPrivacy
          Nov 15	00:04	EasyList
          Nov 15	00:10	Abuse_urlhaus
          Nov 15	00:15	Maltrail_BD
          Nov 15	00:16	OpenPhish
          Nov 15	12:52	Manual_BL_custom
          ===============================================================
          
          Database Sanity check [  FAILED  ] ** These two counts should match! **
          ------------
          Masterfile Count    [ 34658 ]
          Deny folder Count   [ 36078 ]
          
          Duplication sanity check (Pass=No IPs reported)
          ------------------------
          Masterfile/Deny folder uniq check
          Deny folder/Masterfile uniq check
          113.161.8.108
          171.25.193.25
          171.25.193.77
          202.166.164.46
          203.146.129.235
          37.228.129.5
          91.203.145.116
          
          Sync check (Pass=No IPs reported)
          ----------
          
          Alias table IP Counts
          -----------------------------
            301737 total
             73769 /var/db/aliastables/pfB_Top_v6.txt
             70200 /var/db/aliastables/pfB_Top_v4.txt
             61107 /var/db/aliastables/pfB_Whitelist_v4.txt
             60313 /var/db/aliastables/pfB_DNS_Sever_List_v4.txt
             18557 /var/db/aliastables/pfB_SFS_v4.txt
             14906 /var/db/aliastables/pfB_PRI1_v4.txt
              2551 /var/db/aliastables/pfB_PRI4_v4.txt
               183 /var/db/aliastables/pfB_DNS_6_v6.txt
                86 /var/db/aliastables/pfB_PRI1_6_v6.txt
                55 /var/db/aliastables/pfB_PRI3_v4.txt
                10 /var/db/aliastables/pfB_ASN_Block_v4.txt
          
          pfSense Table Stats
          -------------------
          table-entries hard limit  2000000
          Table Usage Count         459519
          

          Any suggestions? Thanks!

          BBcan177B 1 Reply Last reply Reply Quote 0
          • BBcan177B Offline
            BBcan177 Moderator @Draco
            last edited by

            @Draco try to goto the General Tab, first ensure that the Keep Settings option is checked. Then unchecked Enable pfBlockerNG so that its disabled. Hit save. Force Update. Then reenable pfBlockerNG and Force update.

            "Experience is something you don't get until just after you need it."

            Website: http://pfBlockerNG.com
            Twitter: @BBcan177  #pfBlockerNG
            Reddit: https://www.reddit.com/r/pfBlockerNG/new/

            D 1 Reply Last reply Reply Quote 0
            • D Offline
              Draco @BBcan177
              last edited by

              @BBcan177 I turned off pFBlocker and hit RUN on Update. Nothing happened except my firewall complaining about the loss of the aliases used in rules. Re-enabled and did another Force Update. Still failing the db sanity check:

              Database Sanity check [  FAILED  ] ** These two counts should match! **
              ------------
              Masterfile Count    [ 33142 ]
              Deny folder Count   [ 34562 ]
              

              What else should I try or do you need from me to diagnose? THanks!

              tinfoilmattT 1 Reply Last reply Reply Quote 0
              • tinfoilmattT Offline
                tinfoilmatt @Draco
                last edited by

                @Draco said in New pfblockerNG install Database Sanity check Failed:

                I turned off pFBlocker and hit RUN on Update.

                You what?

                D 1 Reply Last reply Reply Quote 0
                • D Offline
                  Draco @tinfoilmatt
                  last edited by Draco

                  I turned off pFBlocker and hit RUN on Update.

                  You what?

                  Like BBCan177 said: unchecked pBlocker, hit save. Force update (so click on Update and click the RUN button on the screen).

                  BBcan177B tinfoilmattT 2 Replies Last reply Reply Quote 0
                  • BBcan177B Offline
                    BBcan177 Moderator @Draco
                    last edited by

                    @Draco did you also uncheck "Keep Settings" that would wipe out all downloaded files and start fresh. If not need please repeat those steps and post the whole log output from this Reload.

                    "Experience is something you don't get until just after you need it."

                    Website: http://pfBlockerNG.com
                    Twitter: @BBcan177  #pfBlockerNG
                    Reddit: https://www.reddit.com/r/pfBlockerNG/new/

                    D 1 Reply Last reply Reply Quote 0
                    • tinfoilmattT Offline
                      tinfoilmatt @Draco
                      last edited by

                      @Draco
                      7ecdca98-7c98-427f-9ee0-95e7a2c9309e-image.png

                      1 Reply Last reply Reply Quote 1
                      • D Offline
                        Draco @BBcan177
                        last edited by

                        @BBcan177 🤦 Thank you for the kind reminder; I am so accustomed to ensuring Save Settings is checked that I didn't follow your instructions properly (thanks @tinfoilmatt for uploading and highlighting the screen shot).

                        I've properly followed the instructions and the update did not report and db problems. Thank you again!

                        • drac
                        1 Reply Last reply Reply Quote 0
                        • First post
                          Last post
                        Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.