• Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login
Netgate Discussion Forum
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login

Dnsbl_error.log growth rate /size

Scheduled Pinned Locked Moved pfBlockerNG
19 Posts 11 Posters 3.3k Views
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • P
    planktonclapped
    last edited by Nov 9, 2017, 9:24 AM

    Hi,

    Came down this morning to find my 2.4 installation was wedged, after a reboot I saw that the root filesystem was full, the culprit seemed to be a 14G dnsbl_error.log.

    I've got the logging option in the DNSBL tab set to disabled but I'm getting, roughly, 200 entries per second written to the dnsbl_error.log - all similar to:

    2017-11-09 09:18:37: (configfile-glue.c.694) === start of condition block === 
    2017-11-09 09:18:37: (configfile-glue.c.350) go parent global/SERVERsocket==0.0.0.0:8443 
    2017-11-09 09:18:37: (configfile-glue.c.622) 2 (cached) result: true 
    2017-11-09 09:18:37: (configfile-glue.c.557) HTTP["host"] ( device-metrics-us.amazon.com ) compare to  .* 
    2017-11-09 09:18:37: (configfile-glue.c.615) 3 (uncached) result: true 
    
    

    Can someone point me in the right direction to understand what the apparent error is that is being logged so frequently or, failing that, control the log files growth.

    1 Reply Last reply Reply Quote 0
    • N
      nickd
      last edited by Jan 20, 2018, 4:50 PM Jan 20, 2018, 4:42 PM

      Also having this exact same issue, the file filled up all 6GB available on my filesystem. Did you ever figure out what was causing it or how to fix it? Interesting that all of my log was all filled with entries related to "device-metrics-us.amazon.com" as well. Sample of beginning of my log file:

      
      2017-11-02 23:01:44: (log.c.217) server started 
      2017-11-02 23:07:46: (configfi202018-01-06 20:33:39: (configfile-glue.c.677) === start of condition block === 
      2018-01-06 20:33:39: (configfile-glue.c.385) 3 global/HTTPhost=~.* not available yet 
      2018-01-06 20:33:39: (configfile-glue.c.589) 1 (uncached) result: unset 
      2018-01-06 20:33:39: (configfile-glue.c.677) === start of condition block === 
      2018-01-06 20:33:39: (configfile-glue.c.531) SERVER["socket"] ( 0.0.0.0:8443 ) compare to  0.0.0.0:8443 
      2018-01-06 20:33:39: (configfile-glue.c.589) 2 (uncached) result: true 
      2018-01-06 20:33:39: (configfile-glue.c.677) === start of condition block === 
      2018-01-06 20:33:39: (configfile-glue.c.342) go parent global/SERVERsocket==0.0.0.0:8443 
      2018-01-06 20:33:39: (configfile-glue.c.596) 2 (cached) res2018-01-07 17:11:33: (2018-01-07 23:33:32: (configfile-glue.c.677) === start of condition block === 
      2018-01-07 23:32018-01-08 01:14:13: (configfile-glue.c.677) === start of condition block === 
      2018-01-08 01:14:13: (configfile-glue.c.531) HTTP["host"] ( device-metrics-us.amazon.com ) compare to  .* 
      2018-01-08 01:14:13: (configfile-glue.c.589) 1 (uncached) result: true 
      2018-01-08 01:14:13: (configfile-glue.c.677) === start of condition block === 
      2018-01-08 01:14:13: (configfile-glue.c.596) 2 (cached) result: true 
      2018-01-08 01:14:13: (configfile-glue.c.677) === start of condition block === 
      2018-01-08 01:14:13: (configfile-glue.c.342) go parent global/SERVERsocket==0.0.0.0:8443 
      2018-01-08 01:14:13: (configfile-glue.c.596) 2 (cached) result: true 
      2018-01-08 01:14:13: (configfile-glue.c.531) HTTP["host"] ( device-metrics-us.amazon.com ) compare to  .* 
      2018-01-08 01:14:13: (configfile-glue.c.589) 3 (uncached) result: true 
      2018-01-08 03:05:18: (configfile-glue.c.677) === start of condition block === 
      2018-01-08 03:05:18: (configfile-glue.c.385) 3 global/HTTPhost=~.* not available yet 
      2018-01-08 03:05:18: (configfile-glue.c.589) 1 (uncached) result: unset 
      2018-01-08 03:05:18: (configfile-glue.c.677) === start of condition block === 
      2018-01-08 03:05:18: (configfile-glue.c.531) SERVER["socket"] ( 0.0.0.0:8443 ) compare to  0.0.0.0:8443 
      2018-01-08 03:05:18: (configfile-glue.c.589) 2 (uncached) result: true 
      2018-01-08 03:05:18: (configfile-glue.c.677) === start of condition block === 
      2018-01-08 03:05:18: (configfile-glue.c.342) go parent global/SERVERsocket==0.0.0.0:8443 
      2018-01-08 03:05:18: (configfile-glue.c.596) 2 (cached) result: true 
      2018-01-08 03:05:18: (configfile-glue.c.385) 3 global/SERVERsocket==0.0.0.0:8443/HTTPhost=~.* not available yet 
      2018-01-08 03:05:18: (configfile-glue.c.589) 3 (uncached) result: unset 
      2018-01-08 03:05:19: (configfile-glue.c.677) === start of condition block === 
      2018-01-08 03:05:19: (configfile-glue.c.531) HTTP["host"] ( device-metrics-us.amazon.com ) compare to  .* 
      
      
      A 1 Reply Last reply Apr 24, 2019, 11:16 PM Reply Quote 0
      • R
        RonpfS
        last edited by Jan 20, 2018, 5:39 PM

        In  Firewall / pfBlockerNG / Log Browser tab you can delete the file to free disk space. You could also download it to a local drive if you want to keep it.

        In Firewall / pfBlockerNG / General tab, there is a setting for log file size.

        2.4.5-RELEASE-p1 (amd64)
        Intel Core2 Quad CPU Q8400 @ 2.66GHz 8GB
        Backup 0.5_5, Bandwidthd 0.7.4_4, Cron 0.3.7_5, pfBlockerNG-devel 3.0.0_16, Status_Traffic_Totals 2.3.1_1, System_Patches 1.2_5

        1 Reply Last reply Reply Quote 0
        • G
          ghorsepower
          last edited by Jan 22, 2018, 2:17 PM

          I am having identical problems. It seems like the dnsbl_error.log keeps trying to stay under the file size limit imposed for minutes to hours then something fails and the log file stops clearing just keeps growing and growing until you run out of disk space.

          If you just delete the log file with rm -rf from shell access it still grows and eats disk space because the file is locked apparently. So i have been rebooting every day to clear it and regain diskspace so my firewall wont crash.

          I figured out today that if you go to Firewall/pfBlockerNG/DNSBL and uncheck "Enable DNSBL", click save. Delete /var/log/pfblockerng/dnsbl_error.log, it actually deletes and regains lost disk space, avoiding a reboot. Then go back to settings and re-enable DNSBL it at least saves a reboot daily? Hope they fix this bug soon because its getting tedious to clean this by hand each day.

          The culprit seems to be "HTTP["host"] ( device-metrics-us.amazon.com ) compare to  .*"

          I am not sure what is causing this, I do have several amazon devices, kindles, fire-tv's etc… They are apparently causing DNSBL to vomit at the rate of 200 lines per second like the gentleman said above? Eventually overloading the clearing function of log management filling the disk.

          1 Reply Last reply Reply Quote 0
          • B
            BBcan177 Moderator
            last edited by Jan 23, 2018, 11:35 PM

            The next release will have a new function to process this log… Just got bogged down since getting back from the holidays... So still working on a couple loose ends...  Thanks!

            "Experience is something you don't get until just after you need it."

            Website: http://pfBlockerNG.com
            Twitter: @BBcan177  #pfBlockerNG
            Reddit: https://www.reddit.com/r/pfBlockerNG/new/

            1 Reply Last reply Reply Quote 0
            • B
              BBcan177 Moderator
              last edited by Jan 23, 2018, 11:39 PM

              You could add these domains to Unbound as a Host override and set them to resolve to 0.0.0.0

              Which will bypass DNSBL completely…

              "Experience is something you don't get until just after you need it."

              Website: http://pfBlockerNG.com
              Twitter: @BBcan177  #pfBlockerNG
              Reddit: https://www.reddit.com/r/pfBlockerNG/new/

              1 Reply Last reply Reply Quote 0
              • G
                ghorsepower
                last edited by Jan 24, 2018, 4:46 PM

                @BBcan177:

                The next release will have a new function to process this log… Just got bogged down since getting back from the holidays... So still working on a couple loose ends...  Thanks!

                Awesome, thanks for the work and looking forward to the next release.

                1 Reply Last reply Reply Quote 0
                • V
                  Valeriy
                  last edited by Feb 20, 2018, 1:14 AM

                  I can confirm that issue exist, 43GB log file after 8 hours ))

                  1 Reply Last reply Reply Quote 0
                  • B
                    beynon
                    last edited by Apr 2, 2018, 1:10 AM

                    I'm seeing this problem now also.  Twice now actually, the latest today.  I'm now on pfsense 2.4.3-RELEASE and pfBlockerNG 2.1.2_2.

                    Anyone know if this log growth problem is fixed in 2.1.2_2 or if the next release will include it?

                    1 Reply Last reply Reply Quote 0
                    • M
                      MingoDynasty
                      last edited by Apr 16, 2018, 3:48 AM

                      I've been having this issue every few days. Last time it filled up in like 3 days? Each time, the /var/log/pfblockerng/dnsbl_error.log file grows to 4.3 GB, which maxes out my Pfsense disk and then causes things to crash and my DNS server to go down.

                      Problem goes away if I restart pfblockerng service, which automatically clears the log file, and then follow that with a reboot. Unfortunately I cannot really keep constant tabs on the disk usage, so I've simply resorted to keeping the pfblockerng service disabled until this issue gets resolved.

                      Strangely I've been running pfblockerng for several months now and never had any issues until now.

                      1 Reply Last reply Reply Quote 0
                      • B
                        bbrendon
                        last edited by May 9, 2018, 7:14 PM

                        Is there a ticket/bug to track this issue?

                        1 Reply Last reply Reply Quote 0
                        • B
                          BBcan177 Moderator
                          last edited by May 11, 2018, 2:26 AM

                          @bbrendon:

                          Is there a ticket/bug to track this issue?

                          This is fixed in the next major release… Its been submitted and is under review. Once approved it will be published as a "devel" release, followed by a full release shortly after...

                          https://github.com/pfsense/FreeBSD-ports/pull/515

                          "Experience is something you don't get until just after you need it."

                          Website: http://pfBlockerNG.com
                          Twitter: @BBcan177  #pfBlockerNG
                          Reddit: https://www.reddit.com/r/pfBlockerNG/new/

                          1 Reply Last reply Reply Quote 0
                          • A
                            ada4u @nickd
                            last edited by Apr 24, 2019, 11:16 PM

                            @nickd hello, still, can't figure out why this logs keeps growing, but in our case to reduce the size, we send this command:
                            echo "..." > /var/log/pfblockerng/dnsbl_error.log every hour from the cron, this way you dont have to reinit or anything,

                            B 1 Reply Last reply Apr 26, 2019, 12:27 AM Reply Quote 0
                            • B
                              BBcan177 Moderator @ada4u
                              last edited by Apr 26, 2019, 12:27 AM

                              @ada4u Install pfBlockerNG-devel which doesn't use that functionality.

                              "Experience is something you don't get until just after you need it."

                              Website: http://pfBlockerNG.com
                              Twitter: @BBcan177  #pfBlockerNG
                              Reddit: https://www.reddit.com/r/pfBlockerNG/new/

                              A 1 Reply Last reply May 6, 2019, 7:58 PM Reply Quote 1
                              • A
                                ada4u @BBcan177
                                last edited by May 6, 2019, 7:58 PM

                                @BBcan177 said in Dnsbl_error.log growth rate /size:

                                pfBlockerNG

                                Thank you so much...

                                that did the trick

                                1 Reply Last reply Reply Quote 0
                                • 4
                                  4o4rh
                                  last edited by Nov 13, 2020, 9:07 PM

                                  This issue is back again. My /var ram drive is becoming full from it, but this has only started since i installed malwarebytes on a PC.

                                  2020-11-13 06:16:12: (configfile-glue.c.581) === start of condition block === 
                                  2020-11-13 06:16:12: (configfile-glue.c.282) go parent global/SERVERsocket==0.0.0.0:8443 
                                  2020-11-13 06:16:12: (configfile-glue.c.500) 2 (cached) result: true 
                                  2020-11-13 06:16:12: (configfile-glue.c.449) HTTP["host"] ( telemetry.malwarebytes.com ) compare to  .* 
                                  2020-11-13 06:16:12: (configfile-glue.c.493) 3 (uncached) result: true
                                  
                                  1 Reply Last reply Reply Quote 0
                                  • R
                                    RonpfS
                                    last edited by RonpfS Nov 13, 2020, 9:14 PM Nov 13, 2020, 9:14 PM

                                    @gwaitsi said in Dnsbl_error.log growth rate /size:

                                    malwarebytes

                                    Search for malwarebytes in the forum :
                                    https://forum.netgate.com/topic/152239/pfblockerng-high-cpu/83

                                    2.4.5-RELEASE-p1 (amd64)
                                    Intel Core2 Quad CPU Q8400 @ 2.66GHz 8GB
                                    Backup 0.5_5, Bandwidthd 0.7.4_4, Cron 0.3.7_5, pfBlockerNG-devel 3.0.0_16, Status_Traffic_Totals 2.3.1_1, System_Patches 1.2_5

                                    4 1 Reply Last reply Nov 13, 2020, 9:31 PM Reply Quote 0
                                    • 4
                                      4o4rh @RonpfS
                                      last edited by Nov 13, 2020, 9:31 PM

                                      @RonpfS terrific, thanks

                                      R 1 Reply Last reply Nov 13, 2020, 9:57 PM Reply Quote 0
                                      • R
                                        RonpfS @4o4rh
                                        last edited by Nov 13, 2020, 9:57 PM

                                        @gwaitsi
                                        Have a look at https://www.reddit.com/r/pfBlockerNG/comments/jt9k89/pfblockerng_malwarebytes_telementery_increased/

                                        2.4.5-RELEASE-p1 (amd64)
                                        Intel Core2 Quad CPU Q8400 @ 2.66GHz 8GB
                                        Backup 0.5_5, Bandwidthd 0.7.4_4, Cron 0.3.7_5, pfBlockerNG-devel 3.0.0_16, Status_Traffic_Totals 2.3.1_1, System_Patches 1.2_5

                                        1 Reply Last reply Reply Quote 0
                                        • First post
                                          Last post
                                        Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.
                                          This community forum collects and processes your personal information.
                                          consent.not_received