Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    New pfblockerNG install Database Sanity check Failed

    Scheduled Pinned Locked Moved pfBlockerNG
    58 Posts 20 Posters 16.3k Views 19 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • M Offline
      MidGe48 @Laxarus
      last edited by

      @Laxarus

      Thank you Laxarus,

      It seems to have worked.

      1 Reply Last reply Reply Quote 0
      • M Offline
        mull0r @Laxarus
        last edited by

        @Laxarus This worked for me as well. Though I had to search the web how to edit the file (the easiest way).

        Therefore:

        Addition for anyone struggling to find where to edit files on your pfsense system.

        Go to Diagnostics --> Edit File --> insert the location of the file:

        /usr/local/pkg/pfblockerng/pfblockerng.sh
        

        Go to line number 1232 by filling it in the Go to line field.

        That line should read:

        s1="$(grep -cv ^${ip_placeholder2}$ ${masterfile})"
        

        replace only (leave the rest intact):

        masterfile
        

        to

        mastercat
        

        Then follow the above instructions from @Laxarus https://forum.netgate.com/post/1219635

        J 1 Reply Last reply Reply Quote 0
        • A anthonys referenced this topic on
        • T Offline
          tman222
          last edited by tman222

          Ran into this issue today as well running on 25.07.1 with pfblockerNG-devel 3.2.7. Followed the steps outlined in this thread to edit the pfblockerng.sh file, then deleted/force reloaded all the lists, and all was well again. Thank you everyone in this thread for your your help and great instructions.

          1 Reply Last reply Reply Quote 0
          • S Offline
            slu @BBcan177
            last edited by

            @BBcan177 said in New pfblockerNG install Database Sanity check Failed:

            From:
            s1="$(grep -cv ^${ip_placeholder2}$ ${masterfile})"

            To:
            s1="$(grep -cv ^${ip_placeholder2}$ ${mastercat})"

            Is it possible to update the pfSense package with a bug fix version?

            pfSense Gold subscription

            G 1 Reply Last reply Reply Quote 0
            • G Offline
              Gradius @slu
              last edited by

              FYI, this bug is still present on pfSense v2.8.1-RC and pfBlockerNG-devel v3.2.8

              M 1 Reply Last reply Reply Quote 0
              • N nanda referenced this topic on
              • J jrey referenced this topic on
              • N nanda referenced this topic on
              • M Offline
                mcury Rebel Alliance @Gradius
                last edited by

                Same issue on 25.07.1
                pfBlockerNG-devel 3.2.7

                Database Sanity check [  FAILED  ] ** These two counts should match! **
                ------------
                Masterfile Count    [ 26379 ]
                Deny folder Count   [ 26378 ]
                

                dead on arrival, nowhere to be found.

                1 Reply Last reply Reply Quote 0
                • TommyMooT Offline
                  TommyMoo
                  last edited by

                  Thank you BIG Time! I had the same problem with pfBlocker Database mismatch message, and its now fixed...Again, Thanks! 👍

                  1 Reply Last reply Reply Quote 0
                  • TommyMooT Offline
                    TommyMoo
                    last edited by TommyMoo

                    pfBlocker been updated tonight, now the Sanity Check Passes without need of patching!

                    Thanks to the Developer GREAT! Also other bug fixes been applied with the Update! 👍 👍 👍

                    1 Reply Last reply Reply Quote 0
                    • B Offline
                      borgotech
                      last edited by borgotech

                      HI everyone, i have the same issue:
                      pfBlockerNG-devel 3.2.10
                      Version 25.07.1-RELEASE (amd64)
                      built on Wed Aug 20 15:17:00 EEST 2025
                      FreeBSD 15.0-CURRENT

                      ===============================================================
                      
                      Database Sanity check [  FAILED  ] ** These two counts should match! **
                      ------------
                      Masterfile Count    [ 60322 ]
                      Deny folder Count   [ 60323 ]
                      
                      Duplication sanity check (Pass=No IPs reported)
                      ------------------------
                      Masterfile/Deny folder uniq check
                      Deny folder/Masterfile uniq check
                      
                      Sync check (Pass=No IPs reported)
                      ----------	
                      

                      i tried to enable/diable ... uninstall/install but the error is there :(. Can anybody give me an advice. Thank you.

                      TommyMooT 1 Reply Last reply Reply Quote 0
                      • TommyMooT Offline
                        TommyMoo @borgotech
                        last edited by TommyMoo

                        @borgotech Hello, the bug, has been fixed with the latest update of pfBlocker. Please, check, if you have the latest version. In Menu System -> Package Manager -> see, if its the latest version (Appears then highlighted in color Orange in Packagemanager) if an Update is avaible.

                        If all this doesnt help, you can still patch it manually, please read above post by mull0r above here, who replies to Laxarus, he describes how to patch, in short, you have to login to your pfsense via SSH, and then:

                        sudo nano /usr/local/pkg/pfblockerng/pfblockerng.sh

                        then look for the line that contains. use ctrl w for to use nano editor search, and paste in:

                        s1="$(grep -cv ^${ip_placeholder2}$ ${masterfile})"

                        and change that line to:

                        s1="$(grep -cv ^${ip_placeholder2}$ ${mastercat})"

                        then, save it, with ctrl o, and leave nano, then just restart pfblocker, and the problem should be gone (worked for me, before last update been release a week ago approx.)

                        Thats it. Now, its should be fixed in the new Update of pfblocker, the Trick / Patch, is only necessary if you have a outdatet pfBlocker installed on your System.

                        Good Luck!

                        B 1 Reply Last reply Reply Quote 0
                        • B Offline
                          borgotech @TommyMoo
                          last edited by

                          @TommyMoo
                          Thank you very much for the answer, as I mentioned above I am on the latest version of PfblockerNG devel pfBlockerNG-devel 3.2.10 and the latest stable version of PfSense+ Version 25.07.1-RELEASE (amd64). The patch in your post and the previous ones do not work because the changes have already been made to the latest version.
                          pfblocker_error.jpg
                          Anyway, thanks again..

                          TommyMooT 1 Reply Last reply Reply Quote 0
                          • TommyMooT Offline
                            TommyMoo @borgotech
                            last edited by

                            @borgotech Hello, Im sorry, its doesnt help you, Im on pfsense 2.8.1 CE ... there, it works... hope someone of the Pros, can help you!

                            1 Reply Last reply Reply Quote 0
                            • J Offline
                              JoeNavy @mull0r
                              last edited by

                              @mull0r Thanks for the clear instructions to fix this issue. I am on pfsense+ 25.07.1 and pfBlockerNG 3.2.7.

                              1 Reply Last reply Reply Quote 0
                              • G Offline
                                Gianluca 0 @tinfoilmatt
                                last edited by

                                @tinfoilmatt said in New pfblockerNG install Database Sanity check Failed:

                                if [ "${s1}" == "${s2}" ]; then

                                I had to change back this line from:

                                if [ "${s1}" == "${s2}" ]; then

                                to

                                if [ "${s1} == ${s2}" ]; then

                                and database sanity now pass.

                                I use PfblockerNG 3.2.8

                                J tinfoilmattT 2 Replies Last reply Reply Quote 0
                                • J Offline
                                  JoeNavy @Gianluca 0
                                  last edited by

                                  @Gianluca-0 Interesting that you are on 3.2.8. I do not see this version as a download, yet.

                                  1 Reply Last reply Reply Quote 0
                                  • tinfoilmattT Offline
                                    tinfoilmatt @Gianluca 0
                                    last edited by

                                    @Gianluca-0 What I think you've essentially done here is 'gracefully' broken the function by failing to properly quote the variables s1 and s2.

                                    If the 'sanity check' does not output a list of specifically-checked IP addresses, then the function is merely reporting that it "PASSED" without any errors thrown.

                                    This is all speculation. Someone with actual coding skills would need to confirm.

                                    tinfoilmattT 1 Reply Last reply Reply Quote 1
                                    • tinfoilmattT Offline
                                      tinfoilmatt @tinfoilmatt
                                      last edited by tinfoilmatt

                                      Here's the relevant variables and function from pfBlockerNG-devel 3.2.10 (on CE 2.8.1-RELEASE):

                                      L1232 & L1233:

                                      s1="$(grep -cv ^${ip_placeholder2}$ ${mastercat})"
                                      s2="$(find ${pfbdeny}*.txt ! -name *_v6.txt -type f 2>/dev/null | xargs cat | grep -cv ^${ip_placeholder2}$)"
                                      

                                      L1278 to L1297:

                                      	# Execute when 'de-duplication' is enabled
                                      	if [ "${alias}" == 'on' ]; then
                                      		echo '==============================================================='; echo
                                      		if [ "${s1}" == "${s2}" ]; then
                                      			echo 'Database Sanity check [  PASSED  ]'
                                      		else
                                      			echo 'Database Sanity check [  FAILED  ] ** These two counts should match! **'
                                      			echo '------------'
                                      			echo "Masterfile Count    [ ${s1} ]"
                                      			echo "Deny folder Count   [ ${s2} ]"; echo
                                      			echo 'Duplication sanity check (Pass=No IPs reported)'
                                      		fi
                                      		echo '------------------------'
                                      		echo 'Masterfile/Deny folder uniq check'
                                      		if [ ! -z "${s3}" ]; then echo "${s3}"; fi
                                      		echo 'Deny folder/Masterfile uniq check'
                                      		if [ ! -z "${s4}" ]; then echo "${s4}"; fi
                                      		echo; echo 'Sync check (Pass=No IPs reported)'
                                      		echo '----------'
                                      	fi
                                      

                                      (For reference, the GitHub build of pfBlockerNG-devel appears to be at version 3.2.12 as of October 13, 2025. But neither this version nor 3.2.11 will be available via Package Manager until either is committed to the private Netgate repository.)

                                      J 1 Reply Last reply Reply Quote 0
                                      • J Offline
                                        JonH @tinfoilmatt
                                        last edited by

                                        @tinfoilmatt I did a quick scan of this thread. I'm on 25.07.1 Release and pfblockerng-devel 3.2.10
                                        I have this same error. I will go back and re-read this thread but FWIW the update mentioned earlier certainly didn't fix it for me.

                                        J 1 Reply Last reply Reply Quote 0
                                        • J Offline
                                          JonH @JonH
                                          last edited by

                                          OK, rather than fiddle with editing the file I tried the method mentioned by @Laxarus. It worked perfectly.

                                          Database Sanity check [ PASSED ]

                                          Masterfile/Deny folder uniq check
                                          Deny folder/Masterfile uniq check

                                          Sync check (Pass=No IPs reported)

                                          Thanks to all who contributed ideas.

                                          1 Reply Last reply Reply Quote 0
                                          • D Offline
                                            Draco @BBcan177
                                            last edited by

                                            @BBcan177 I am runnng pfSense 25.07.1-RELEASE (amd64) on an netgate 5100 box, and pfBBLockerNG-devel v3.2.10. My counts have been off by anywhere from a few dozen to a few thousand.

                                            I checked /usr/local/pkg/pfblockerng/pfblockerng.sh and line 1232 is already

                                            s1="$(grep -cv ^${ip_placeholder2}$ ${mastercat})"
                                            

                                            A sampling of teh log info around the sanity check after a forced reload:

                                            ===[ DNSBL Domain/IP Counts ] ===================================
                                            
                                              889529 total
                                              602876 /var/db/pfblockerng/dnsbl/Maltrail_BD.txt
                                               84614 /var/db/pfblockerng/dnsbl/StevenBlack_ADs.txt
                                               64269 /var/db/pfblockerng/dnsbl/SFS_Toxic_BD.txt
                                               40604 /var/db/pfblockerng/dnsbl/EasyList.txt
                                               40159 /var/db/pfblockerng/dnsbl/EasyPrivacy.txt
                                               15210 /var/db/pfblockerng/dnsbl/MS_2.txt
                                               10751 /var/db/pfblockerng/dnsbl/Abuse_urlhaus.txt
                                               10199 /var/db/pfblockerng/dnsbl/SWC.txt
                                                9076 /var/db/pfblockerng/dnsbl/Exch_AD_Servers.txt
                                                6101 /var/db/pfblockerng/dnsbl/Adaway.txt
                                                2793 /var/db/pfblockerng/dnsbl/Yoyo.txt
                                                2344 /var/db/pfblockerng/dnsbl/Easylist_Firebog.txt
                                                 235 /var/db/pfblockerng/dnsbl/OpenPhish.txt
                                                 229 /var/db/pfblockerng/dnsbl/NoCoin.txt
                                                  39 /var/db/pfblockerng/dnsbl/MoneroMiner.txt
                                                  30 /var/db/pfblockerng/dnsbl/Manual_BL_custom.txt
                                            
                                            ====================[ IPv4/6 Last Updated List Summary ]==============
                                            
                                            May 17	2024	Spamhaus_eDrop_v4
                                            May 17	2024	ARIN_MSFT_ASNs_v4
                                            May 17	2024	Whitelist_custom_v4
                                            Aug 14	2024	GreatWallDoH_v4
                                            Jan 3	2025	Abuse_SSLBL_v4
                                            Jan 17	2025	Talos_BL_v4
                                            Jul 31	00:17	MSFT_IPBlocks_v4
                                            Nov 1	15:08	Spamhaus_Drop6_v6
                                            Nov 13	21:30	ET_Block_v4
                                            Nov 14	05:39	Spamhaus_Drop_v4
                                            Nov 14	13:39	ET_Comp_v4
                                            Nov 14	15:55	BDS_Ban_v4
                                            Nov 14	23:48	TOR_Exit_Nodes_v4
                                            Nov 15	00:03	Public_DNS6_v6
                                            Nov 15	00:03	DNSServers_v4
                                            Nov 15	08:13	SFS_IPs_7day_v4
                                            Nov 15	08:15	HoneyPot_Bad_v4
                                            Nov 15	08:15	PubMatic_v4
                                            Nov 15	10:31	CINS_army_v4
                                            Nov 15	11:45	ISC_Block_v4
                                            Nov 15	12:03	DNSServers2_v4
                                            Nov 15	12:10	Abuse_Feodo_C2_v4
                                            Nov 15	12:11	BotScout_v4
                                            Nov 15	12:53	pfB_Top_v4
                                            Nov 15	12:53	pfB_Top_v6
                                            Nov 15	12:53	PRI1_custom_v4
                                            
                                            ====================[ DNSBL Last Updated List Summary ]==============
                                            
                                            Apr 30	2023	Adaway
                                            May 17	2024	Exch_AD_Servers
                                            Jan 21	2025	MoneroMiner
                                            Mar 6	2025	NoCoin
                                            Oct 6	15:16	MS_2
                                            Oct 28	09:09	SWC
                                            Nov 12	00:16	StevenBlack_ADs
                                            Nov 14	05:42	Yoyo
                                            Nov 14	06:00	Easylist_Firebog
                                            Nov 14	23:59	SFS_Toxic_BD
                                            Nov 15	00:04	EasyPrivacy
                                            Nov 15	00:04	EasyList
                                            Nov 15	00:10	Abuse_urlhaus
                                            Nov 15	00:15	Maltrail_BD
                                            Nov 15	00:16	OpenPhish
                                            Nov 15	12:52	Manual_BL_custom
                                            ===============================================================
                                            
                                            Database Sanity check [  FAILED  ] ** These two counts should match! **
                                            ------------
                                            Masterfile Count    [ 34658 ]
                                            Deny folder Count   [ 36078 ]
                                            
                                            Duplication sanity check (Pass=No IPs reported)
                                            ------------------------
                                            Masterfile/Deny folder uniq check
                                            Deny folder/Masterfile uniq check
                                            113.161.8.108
                                            171.25.193.25
                                            171.25.193.77
                                            202.166.164.46
                                            203.146.129.235
                                            37.228.129.5
                                            91.203.145.116
                                            
                                            Sync check (Pass=No IPs reported)
                                            ----------
                                            
                                            Alias table IP Counts
                                            -----------------------------
                                              301737 total
                                               73769 /var/db/aliastables/pfB_Top_v6.txt
                                               70200 /var/db/aliastables/pfB_Top_v4.txt
                                               61107 /var/db/aliastables/pfB_Whitelist_v4.txt
                                               60313 /var/db/aliastables/pfB_DNS_Sever_List_v4.txt
                                               18557 /var/db/aliastables/pfB_SFS_v4.txt
                                               14906 /var/db/aliastables/pfB_PRI1_v4.txt
                                                2551 /var/db/aliastables/pfB_PRI4_v4.txt
                                                 183 /var/db/aliastables/pfB_DNS_6_v6.txt
                                                  86 /var/db/aliastables/pfB_PRI1_6_v6.txt
                                                  55 /var/db/aliastables/pfB_PRI3_v4.txt
                                                  10 /var/db/aliastables/pfB_ASN_Block_v4.txt
                                            
                                            pfSense Table Stats
                                            -------------------
                                            table-entries hard limit  2000000
                                            Table Usage Count         459519
                                            

                                            Any suggestions? Thanks!

                                            BBcan177B 1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.