Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    New pfblockerNG install Database Sanity check Failed

    Scheduled Pinned Locked Moved pfBlockerNG
    58 Posts 20 Posters 16.3k Views 19 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • TommyMooT Offline
      TommyMoo @borgotech
      last edited by

      @borgotech Hello, Im sorry, its doesnt help you, Im on pfsense 2.8.1 CE ... there, it works... hope someone of the Pros, can help you!

      1 Reply Last reply Reply Quote 0
      • J Offline
        JoeNavy @mull0r
        last edited by

        @mull0r Thanks for the clear instructions to fix this issue. I am on pfsense+ 25.07.1 and pfBlockerNG 3.2.7.

        1 Reply Last reply Reply Quote 0
        • G Offline
          Gianluca 0 @tinfoilmatt
          last edited by

          @tinfoilmatt said in New pfblockerNG install Database Sanity check Failed:

          if [ "${s1}" == "${s2}" ]; then

          I had to change back this line from:

          if [ "${s1}" == "${s2}" ]; then

          to

          if [ "${s1} == ${s2}" ]; then

          and database sanity now pass.

          I use PfblockerNG 3.2.8

          J tinfoilmattT 2 Replies Last reply Reply Quote 0
          • J Offline
            JoeNavy @Gianluca 0
            last edited by

            @Gianluca-0 Interesting that you are on 3.2.8. I do not see this version as a download, yet.

            1 Reply Last reply Reply Quote 0
            • tinfoilmattT Offline
              tinfoilmatt @Gianluca 0
              last edited by

              @Gianluca-0 What I think you've essentially done here is 'gracefully' broken the function by failing to properly quote the variables s1 and s2.

              If the 'sanity check' does not output a list of specifically-checked IP addresses, then the function is merely reporting that it "PASSED" without any errors thrown.

              This is all speculation. Someone with actual coding skills would need to confirm.

              tinfoilmattT 1 Reply Last reply Reply Quote 1
              • tinfoilmattT Offline
                tinfoilmatt @tinfoilmatt
                last edited by tinfoilmatt

                Here's the relevant variables and function from pfBlockerNG-devel 3.2.10 (on CE 2.8.1-RELEASE):

                L1232 & L1233:

                s1="$(grep -cv ^${ip_placeholder2}$ ${mastercat})"
                s2="$(find ${pfbdeny}*.txt ! -name *_v6.txt -type f 2>/dev/null | xargs cat | grep -cv ^${ip_placeholder2}$)"
                

                L1278 to L1297:

                	# Execute when 'de-duplication' is enabled
                	if [ "${alias}" == 'on' ]; then
                		echo '==============================================================='; echo
                		if [ "${s1}" == "${s2}" ]; then
                			echo 'Database Sanity check [  PASSED  ]'
                		else
                			echo 'Database Sanity check [  FAILED  ] ** These two counts should match! **'
                			echo '------------'
                			echo "Masterfile Count    [ ${s1} ]"
                			echo "Deny folder Count   [ ${s2} ]"; echo
                			echo 'Duplication sanity check (Pass=No IPs reported)'
                		fi
                		echo '------------------------'
                		echo 'Masterfile/Deny folder uniq check'
                		if [ ! -z "${s3}" ]; then echo "${s3}"; fi
                		echo 'Deny folder/Masterfile uniq check'
                		if [ ! -z "${s4}" ]; then echo "${s4}"; fi
                		echo; echo 'Sync check (Pass=No IPs reported)'
                		echo '----------'
                	fi
                

                (For reference, the GitHub build of pfBlockerNG-devel appears to be at version 3.2.12 as of October 13, 2025. But neither this version nor 3.2.11 will be available via Package Manager until either is committed to the private Netgate repository.)

                J 1 Reply Last reply Reply Quote 0
                • J Offline
                  JonH @tinfoilmatt
                  last edited by

                  @tinfoilmatt I did a quick scan of this thread. I'm on 25.07.1 Release and pfblockerng-devel 3.2.10
                  I have this same error. I will go back and re-read this thread but FWIW the update mentioned earlier certainly didn't fix it for me.

                  J 1 Reply Last reply Reply Quote 0
                  • J Offline
                    JonH @JonH
                    last edited by

                    OK, rather than fiddle with editing the file I tried the method mentioned by @Laxarus. It worked perfectly.

                    Database Sanity check [ PASSED ]

                    Masterfile/Deny folder uniq check
                    Deny folder/Masterfile uniq check

                    Sync check (Pass=No IPs reported)

                    Thanks to all who contributed ideas.

                    1 Reply Last reply Reply Quote 0
                    • D Offline
                      Draco @BBcan177
                      last edited by

                      @BBcan177 I am runnng pfSense 25.07.1-RELEASE (amd64) on an netgate 5100 box, and pfBBLockerNG-devel v3.2.10. My counts have been off by anywhere from a few dozen to a few thousand.

                      I checked /usr/local/pkg/pfblockerng/pfblockerng.sh and line 1232 is already

                      s1="$(grep -cv ^${ip_placeholder2}$ ${mastercat})"
                      

                      A sampling of teh log info around the sanity check after a forced reload:

                      ===[ DNSBL Domain/IP Counts ] ===================================
                      
                        889529 total
                        602876 /var/db/pfblockerng/dnsbl/Maltrail_BD.txt
                         84614 /var/db/pfblockerng/dnsbl/StevenBlack_ADs.txt
                         64269 /var/db/pfblockerng/dnsbl/SFS_Toxic_BD.txt
                         40604 /var/db/pfblockerng/dnsbl/EasyList.txt
                         40159 /var/db/pfblockerng/dnsbl/EasyPrivacy.txt
                         15210 /var/db/pfblockerng/dnsbl/MS_2.txt
                         10751 /var/db/pfblockerng/dnsbl/Abuse_urlhaus.txt
                         10199 /var/db/pfblockerng/dnsbl/SWC.txt
                          9076 /var/db/pfblockerng/dnsbl/Exch_AD_Servers.txt
                          6101 /var/db/pfblockerng/dnsbl/Adaway.txt
                          2793 /var/db/pfblockerng/dnsbl/Yoyo.txt
                          2344 /var/db/pfblockerng/dnsbl/Easylist_Firebog.txt
                           235 /var/db/pfblockerng/dnsbl/OpenPhish.txt
                           229 /var/db/pfblockerng/dnsbl/NoCoin.txt
                            39 /var/db/pfblockerng/dnsbl/MoneroMiner.txt
                            30 /var/db/pfblockerng/dnsbl/Manual_BL_custom.txt
                      
                      ====================[ IPv4/6 Last Updated List Summary ]==============
                      
                      May 17	2024	Spamhaus_eDrop_v4
                      May 17	2024	ARIN_MSFT_ASNs_v4
                      May 17	2024	Whitelist_custom_v4
                      Aug 14	2024	GreatWallDoH_v4
                      Jan 3	2025	Abuse_SSLBL_v4
                      Jan 17	2025	Talos_BL_v4
                      Jul 31	00:17	MSFT_IPBlocks_v4
                      Nov 1	15:08	Spamhaus_Drop6_v6
                      Nov 13	21:30	ET_Block_v4
                      Nov 14	05:39	Spamhaus_Drop_v4
                      Nov 14	13:39	ET_Comp_v4
                      Nov 14	15:55	BDS_Ban_v4
                      Nov 14	23:48	TOR_Exit_Nodes_v4
                      Nov 15	00:03	Public_DNS6_v6
                      Nov 15	00:03	DNSServers_v4
                      Nov 15	08:13	SFS_IPs_7day_v4
                      Nov 15	08:15	HoneyPot_Bad_v4
                      Nov 15	08:15	PubMatic_v4
                      Nov 15	10:31	CINS_army_v4
                      Nov 15	11:45	ISC_Block_v4
                      Nov 15	12:03	DNSServers2_v4
                      Nov 15	12:10	Abuse_Feodo_C2_v4
                      Nov 15	12:11	BotScout_v4
                      Nov 15	12:53	pfB_Top_v4
                      Nov 15	12:53	pfB_Top_v6
                      Nov 15	12:53	PRI1_custom_v4
                      
                      ====================[ DNSBL Last Updated List Summary ]==============
                      
                      Apr 30	2023	Adaway
                      May 17	2024	Exch_AD_Servers
                      Jan 21	2025	MoneroMiner
                      Mar 6	2025	NoCoin
                      Oct 6	15:16	MS_2
                      Oct 28	09:09	SWC
                      Nov 12	00:16	StevenBlack_ADs
                      Nov 14	05:42	Yoyo
                      Nov 14	06:00	Easylist_Firebog
                      Nov 14	23:59	SFS_Toxic_BD
                      Nov 15	00:04	EasyPrivacy
                      Nov 15	00:04	EasyList
                      Nov 15	00:10	Abuse_urlhaus
                      Nov 15	00:15	Maltrail_BD
                      Nov 15	00:16	OpenPhish
                      Nov 15	12:52	Manual_BL_custom
                      ===============================================================
                      
                      Database Sanity check [  FAILED  ] ** These two counts should match! **
                      ------------
                      Masterfile Count    [ 34658 ]
                      Deny folder Count   [ 36078 ]
                      
                      Duplication sanity check (Pass=No IPs reported)
                      ------------------------
                      Masterfile/Deny folder uniq check
                      Deny folder/Masterfile uniq check
                      113.161.8.108
                      171.25.193.25
                      171.25.193.77
                      202.166.164.46
                      203.146.129.235
                      37.228.129.5
                      91.203.145.116
                      
                      Sync check (Pass=No IPs reported)
                      ----------
                      
                      Alias table IP Counts
                      -----------------------------
                        301737 total
                         73769 /var/db/aliastables/pfB_Top_v6.txt
                         70200 /var/db/aliastables/pfB_Top_v4.txt
                         61107 /var/db/aliastables/pfB_Whitelist_v4.txt
                         60313 /var/db/aliastables/pfB_DNS_Sever_List_v4.txt
                         18557 /var/db/aliastables/pfB_SFS_v4.txt
                         14906 /var/db/aliastables/pfB_PRI1_v4.txt
                          2551 /var/db/aliastables/pfB_PRI4_v4.txt
                           183 /var/db/aliastables/pfB_DNS_6_v6.txt
                            86 /var/db/aliastables/pfB_PRI1_6_v6.txt
                            55 /var/db/aliastables/pfB_PRI3_v4.txt
                            10 /var/db/aliastables/pfB_ASN_Block_v4.txt
                      
                      pfSense Table Stats
                      -------------------
                      table-entries hard limit  2000000
                      Table Usage Count         459519
                      

                      Any suggestions? Thanks!

                      BBcan177B 1 Reply Last reply Reply Quote 0
                      • BBcan177B Offline
                        BBcan177 Moderator @Draco
                        last edited by

                        @Draco try to goto the General Tab, first ensure that the Keep Settings option is checked. Then unchecked Enable pfBlockerNG so that its disabled. Hit save. Force Update. Then reenable pfBlockerNG and Force update.

                        "Experience is something you don't get until just after you need it."

                        Website: http://pfBlockerNG.com
                        Twitter: @BBcan177  #pfBlockerNG
                        Reddit: https://www.reddit.com/r/pfBlockerNG/new/

                        1 Reply Last reply Reply Quote 0
                        • First post
                          Last post
                        Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.