Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    dnsbl Crashing

    pfBlockerNG
    3
    6
    559
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • S
      Stewart
      last edited by

      I never used dnsbl with pfBlockerNG but now that I've put pfBlockerNG-devel on a couple of routers I've been trying to use it. Unfortunately, when I do DNS goes up and down and I see Watchdog alerts that dnsbl keeps crashing. Where do I start to troubleshoot it? The error is:

      Nov 26 10:02:01 	php-cgi 		servicewatchdog_cron.php: Service Watchdog detected service dnsbl stopped. Restarting dnsbl (pfBlockerNG DNSBL Web Server)
Nov 26 10:03:00 	php-cgi 		servicewatchdog_cron.php: Service Watchdog detected service dnsbl stopped. Restarting dnsbl (pfBlockerNG DNSBL Web Server)
Nov 26 10:04:01 	php-cgi 		servicewatchdog_cron.php: Service Watchdog detected service dnsbl stopped. Restarting dnsbl (pfBlockerNG DNSBL Web Server)
Nov 26 10:05:01 	php-cgi 		servicewatchdog_cron.php: Service Watchdog detected service dnsbl stopped. Restarting dnsbl (pfBlockerNG DNSBL Web Server)

      Thanks for any help.

      GertjanG 1 Reply Last reply Reply Quote 0
      • GertjanG
        Gertjan @Stewart
        last edited by

        What is this "servicewatchdog_cron" ?
        The Service_Watchdog package ?
        That's a sledge hammer solution. Hitting onto your system until works. You wind up with pure pulp at the end.

        Consider repairing it. Services that go down should be set up correctly so that thy won't go down.
        "correctly " means also that the load of your device corresponds with physical limits like disk size and memory size, processor performance, etc.

        I never saw an instance of "unbound" or "nginx" or anything else go down on pfSense - for a decade or so now.

        Aren't there any more log lines in other logs that give some details about why the process "dnsbl" stops ?
        Or any other messages that look like non-standard ?

        Btw : add you pfSense version, pfBlockerNG version and please confirm that you do not use a boat load of feeds.

        pfBlockerNG-devel 2.2.5_26 on pfSense 2.4.4-p3-AMD runs just fine (for me).

        No "help me" PM's please. Use the forum, the community will thank you.
        Edit : and where are the logs ??

        1 Reply Last reply Reply Quote 0
        • S
          Stewart
          last edited by Stewart

          @Gertjan said in dnsbl Crashing:

          What is this "servicewatchdog_cron" ?
          The Service_Watchdog package ?
          That's a sledge hammer solution. Hitting onto your system until works. You wind up with pure pulp at the end.

          That package monitors services and attempts to restart them if they stop. It's very useful. Sometimes packages don't start up after a definition update or an errant issue. The watchdog service restarts them. If it keeps restarting (like it is doing here) there is certainly an issue with the package.

          Consider repairing it. Services that go down should be set up correctly so that thy won't go down.
          "correctly " means also that the load of your device corresponds with physical limits like disk size and memory size, processor performance, etc.

          Well, yes. I'm trying to repair it and set it up correctly. As I stated it's my first time using this package and I'd like help to know where to check to troubleshoot. And the system isn't overtaxed:
          680bac5e-15e7-4e98-9fa0-7896c4d6a071-image.png

          I never saw an instance of "unbound" or "nginx" or anything else go down on pfSense - for a decade or so now.

          Congratulations. I've seen plenty of instances where c-icap, Squid, SquidGuard, Snort, etc. have crashed. Many times it's because of lack of space, usually because a log file (often Snort or Suricata) gets out of control and fills the entire SSD.

          Aren't there any more log lines in other logs that give some details about why the process "dnsbl" stops ?
          Or any other messages that look like non-standard ?

          Nope, but I'll give more lines from the system log below so you can see.

          Btw : add you pfSense version, pfBlockerNG version and please confirm that you do not use a boat load of feeds.

          84434243-d679-420b-9ba3-9f9576a10638-image.png
          Just the basic ones. And Blacklist is turned off.

          pfBlockerNG-devel 2.2.5_26 on pfSense 2.4.4-p3-AMD runs just fine (for me).

          Again, glad it works for you. It obviously doesn't for me, hence me asking what logs to look at and what to look for.

          2.4.4-RELEASE-p3 (amd64)
          built on Wed May 15 18:53:44 EDT 2019
          FreeBSD 11.2-RELEASE-p10
          pfBlockerNG-devel net 2.2.5_26

          Probably linked to these lines in the pfBlocker error.log file:

           [ pfB_PRI1_v4 - BBC_C2_v4 ] Download FAIL [ 11/25/19 14:00:26 ]
  Firewall and/or IDS (Legacy mode only) are not blocking download.

  Restoring previously downloaded file contents...

 [ pfB_PRI1_v4 - BBC_C2_v4 ] Download FAIL [ 11/25/19 15:01:19 ]
  Firewall and/or IDS (Legacy mode only) are not blocking download.

  Restoring previously downloaded file contents...

 [ pfB_PRI1_v4 - BBC_C2_v4 ] Download FAIL [ 11/25/19 18:02:45 ]
  Firewall and/or IDS (Legacy mode only) are not blocking download.

  Restoring previously downloaded file contents...

 [ pfB_PRI1_v4 - BBC_C2_v4 ] Download FAIL [ 11/25/19 20:01:49 ]
  Firewall and/or IDS (Legacy mode only) are not blocking download.

  Restoring previously downloaded file contents...

 [ pfB_PRI1_v4 - BBC_C2_v4 ] Download FAIL [ 11/25/19 21:01:30 ]
  Firewall and/or IDS (Legacy mode only) are not blocking download.

  Restoring previously downloaded file contents...

 [ pfB_PRI1_v4 - BBC_C2_v4 ] Download FAIL [ 11/26/19 01:01:29 ]
  Firewall and/or IDS (Legacy mode only) are not blocking download.

  Restoring previously downloaded file contents...

          More firewall log lines.

          Nov 26 10:01:08 	suricata 	82524 	[1:2402000:5374] ET DROP Dshield Block Listed Source group 1 [Classification: Misc Attack] [Priority: 2] {TCP} 185.209.0.32:49994 -> x.x.x.x:3382 
Nov 26 10:01:09 	php 		[pfBlockerNG] No changes to Firewall rules, skipping Filter Reload 
Nov 26 10:02:01 	php-cgi 		servicewatchdog_cron.php: Service Watchdog detected service dnsbl stopped. Restarting dnsbl (pfBlockerNG DNSBL Web Server) 
Nov 26 10:03:00 	php-cgi 		servicewatchdog_cron.php: Service Watchdog detected service dnsbl stopped. Restarting dnsbl (pfBlockerNG DNSBL Web Server) 
Nov 26 10:04:01 	php-cgi 		servicewatchdog_cron.php: Service Watchdog detected service dnsbl stopped. Restarting dnsbl (pfBlockerNG DNSBL Web Server) 
Nov 26 10:05:01 	php-cgi 		servicewatchdog_cron.php: Service Watchdog detected service dnsbl stopped. Restarting dnsbl (pfBlockerNG DNSBL Web Server) 
Nov 26 10:05:58 	suricata 	82524 	[1:2403387:53446] ET CINS Active Threat Intelligence Poor Reputation IP group 88 [Classification: Misc Attack] [Priority: 2] {TCP} 92.222.103.113:44251 -> x.x.x.x:445 
Nov 26 10:06:01 	php-cgi 		servicewatchdog_cron.php: Service Watchdog detected service dnsbl stopped. Restarting dnsbl (pfBlockerNG DNSBL Web Server) 
Nov 26 10:06:14 	suricata 	82524 	[1:2403382:53446] ET CINS Active Threat Intelligence Poor Reputation IP group 83 [Classification: Misc Attack] [Priority: 2] {TCP} 89.248.162.143:32946 -> x.x.x.x:8089 
Nov 26 10:07:00 	php-cgi 		servicewatchdog_cron.php: Service Watchdog detected service dnsbl stopped. Restarting dnsbl (pfBlockerNG DNSBL Web Server) 
Nov 26 10:08:00 	php-cgi 		servicewatchdog_cron.php: Service Watchdog detected service dnsbl stopped. Restarting dnsbl (pfBlockerNG DNSBL Web Server) 
Nov 26 10:09:00 	php-cgi 		servicewatchdog_cron.php: Service Watchdog detected service dnsbl stopped. Restarting dnsbl (pfBlockerNG DNSBL Web Server) 
Nov 26 10:09:21 	suricata 	82524 	[1:2403370:53446] ET CINS Active Threat Intelligence Poor Reputation IP group 71 [Classification: Misc Attack] [Priority: 2] {TCP} 80.82.70.239:52621 -> x.x.x.x:3688 
Nov 26 10:09:29 	suricata 	82524 	[1:2403362:53446] ET CINS Active Threat Intelligence Poor Reputation IP group 63 [Classification: Misc Attack] [Priority: 2] {TCP} 71.6.199.23:29011 -> x.x.x.x:5801 
Nov 26 10:09:30 	suricata 	90426 	[1:2007994:21] ET INFO Suspicious User-Agent (1 space) [Classification: Unknown Traffic] [Priority: 3] {TCP} 192.168.0.106:60780 -> 216.69.156.172:80 
Nov 26 10:09:30 	suricata 	82524 	[1:2007994:21] ET INFO Suspicious User-Agent (1 space) [Classification: Unknown Traffic] [Priority: 3] {TCP} x.x.x.x:58144 -> 216.69.156.172:80 
Nov 26 10:10:01 	php-cgi 		servicewatchdog_cron.php: Service Watchdog detected service dnsbl stopped. Restarting dnsbl (pfBlockerNG DNSBL Web Server) 
Nov 26 10:10:23 	suricata 	82524 	[1:2402000:5374] ET DROP Dshield Block Listed Source group 1 [Classification: Misc Attack] [Priority: 2] {TCP} 159.203.201.134:58621 -> x.x.x.x:61105 
Nov 26 10:10:26 	suricata 	82524 	[1:2403386:53446] ET CINS Active Threat Intelligence Poor Reputation IP group 87 [Classification: Misc Attack] [Priority: 2] {TCP} 92.119.160.143:41830 -> x.x.x.x:34758 
Nov 26 10:10:32 	suricata 	82524 	[1:2403308:53446] ET CINS Active Threat Intelligence Poor Reputation IP group 9 [Classification: Misc Attack] [Priority: 2] {TCP} 14.207.152.212:46686 -> x.x.x.x:26 
Nov 26 10:11:00 	php-cgi 		servicewatchdog_cron.php: Service Watchdog detected service dnsbl stopped. Restarting dnsbl (pfBlockerNG DNSBL Web Server) 
Nov 26 10:11:35 	suricata 	82524 	[1:2402000:5374] ET DROP Dshield Block Listed Source group 1 [Classification: Misc Attack] [Priority: 2] {TCP} 89.248.174.215:40016 -> x.x.x.x:8089 
Nov 26 10:11:35 	suricata 	82524 	[1:2403382:53446] ET CINS Active Threat Intelligence Poor Reputation IP group 83 [Classification: Misc Attack] [Priority: 2] {TCP} 89.248.174.215:40016 -> x.x.x.x:8089 
Nov 26 10:12:00 	php-cgi 		servicewatchdog_cron.php: Service Watchdog detected service dnsbl stopped. Restarting dnsbl (pfBlockerNG DNSBL Web Server) 
Nov 26 10:12:17 	suricata 	82524 	[1:2403382:53446] ET CINS Active Threat Intelligence Poor Reputation IP group 83 [Classification: Misc Attack] [Priority: 2] {TCP} 89.248.160.178:54874 -> x.x.x.x:6766 
Nov 26 10:12:56 	suricata 	82524 	[1:2402000:5374] ET DROP Dshield Block Listed Source group 1 [Classification: Misc Attack] [Priority: 2] {TCP} 185.175.93.104:44136 -> x.x.x.x:33012 
Nov 26 10:13:00 	php-cgi 		servicewatchdog_cron.php: Service Watchdog detected service dnsbl stopped. Restarting dnsbl (pfBlockerNG DNSBL Web Server) 
Nov 26 10:14:00 	php-cgi 		servicewatchdog_cron.php: Service Watchdog detected service dnsbl stopped. Restarting dnsbl (pfBlockerNG DNSBL Web Server) 
Nov 26 10:15:01 	php-cgi 		servicewatchdog_cron.php: Service Watchdog detected service dnsbl stopped. Restarting dnsbl (pfBlockerNG DNSBL Web Server) 
Nov 26 10:15:13 	suricata 	82524 	[1:2402000:5374] ET DROP Dshield Block Listed Source group 1 [Classification: Misc Attack] [Priority: 2] {TCP} 185.175.93.101:45607 -> x.x.x.x:5925 
Nov 26 10:15:39 	suricata 	82524 	[1:2402000:5374] ET DROP Dshield Block Listed Source group 1 [Classification: Misc Attack] [Priority: 2] {TCP} 185.176.27.166:59611 -> x.x.x.x:33189 
Nov 26 10:16:00 	php-cgi 		servicewatchdog_cron.php: Service Watchdog detected service dnsbl stopped. Restarting dnsbl (pfBlockerNG DNSBL Web Server) 
Nov 26 10:16:55 	suricata 	82524 	[1:2402000:5374] ET DROP Dshield Block Listed Source group 1 [Classification: Misc Attack] [Priority: 2] {TCP} 92.63.194.148:54988 -> x.x.x.x:55441 
Nov 26 10:16:55 	suricata 	82524 	[1:2403387:53446] ET CINS Active Threat Intelligence Poor Reputation IP group 88 [Classification: Misc Attack] [Priority: 2] {TCP} 92.63.194.148:54988 -> x.x.x.x:55441 
Nov 26 10:17:01 	php-cgi 		servicewatchdog_cron.php: Service Watchdog detected service dnsbl stopped. Restarting dnsbl (pfBlockerNG DNSBL Web Server) 
Nov 26 10:17:26 	suricata 	82524 	[1:2403315:53446] ET CINS Active Threat Intelligence Poor Reputation IP group 16 [Classification: Misc Attack] [Priority: 2] {TCP} 34.93.213.59:52769 -> x.x.x.x:2121 
Nov 26 10:18:00 	php-cgi 		servicewatchdog_cron.php: Service Watchdog detected service dnsbl stopped. Restarting dnsbl (pfBlockerNG DNSBL Web Server) 
Nov 26 10:19:01 	php-cgi 		servicewatchdog_cron.php: Service Watchdog detected service dnsbl stopped. Restarting dnsbl (pfBlockerNG DNSBL Web Server) 
Nov 26 10:19:07 	suricata 	82524 	[1:2402000:5374] ET DROP Dshield Block Listed Source group 1 [Classification: Misc Attack] [Priority: 2] {TCP} 89.248.174.223:18137 -> x.x.x.x:8089 
Nov 26 10:19:07 	suricata 	82524 	[1:2403382:53446] ET CINS Active Threat Intelligence Poor Reputation IP group 83 [Classification: Misc Attack] [Priority: 2] {TCP} 89.248.174.223:18137 -> x.x.x.x:8089 
Nov 26 10:19:15 	suricata 	82524 	[1:2403370:53446] ET CINS Active Threat Intelligence Poor Reputation IP group 71 [Classification: Misc Attack] [Priority: 2] {UDP} 80.82.78.100:56520 -> x.x.x.x:49 
Nov 26 10:20:01 	php-cgi 		servicewatchdog_cron.php: Service Watchdog detected service dnsbl stopped. Restarting dnsbl (pfBlockerNG DNSBL Web Server) 
Nov 26 10:21:01 	php-cgi 		servicewatchdog_cron.php: Service Watchdog detected service dnsbl stopped. Restarting dnsbl (pfBlockerNG DNSBL Web Server) 
Nov 26 10:22:00 	php-cgi 		servicewatchdog_cron.php: Service Watchdog detected service dnsbl stopped. Restarting dnsbl (pfBlockerNG DNSBL Web Server) 
Nov 26 10:22:17 	suricata 	82524 	[1:2402000:5374] ET DROP Dshield Block Listed Source group 1 [Classification: Misc Attack] [Priority: 2] {TCP} 185.175.93.107:59104 -> x.x.x.x:55555 
Nov 26 10:23:01 	php-cgi 		servicewatchdog_cron.php: Service Watchdog detected service dnsbl stopped. Restarting dnsbl (pfBlockerNG DNSBL Web Server) 
Nov 26 10:23:17 	suricata 	82524 	[1:2402000:5374] ET DROP Dshield Block Listed Source group 1 [Classification: Misc Attack] [Priority: 2] {TCP} 185.143.223.148:52886 -> x.x.x.x:25747 
Nov 26 10:23:29 	php-fpm 	15784 	/diag_tables.php: Session timed out for user 'admin' from: y.y.y.y (Local Database) 
Nov 26 10:23:36 	php-fpm 	15784 	/diag_tables.php: Successful login for user 'admin' from: y.y.y.y (Local Database) 
Nov 26 10:24:00 	php-cgi 		servicewatchdog_cron.php: Service Watchdog detected service dnsbl stopped. Restarting dnsbl (pfBlockerNG DNSBL Web Server)
          JeGrJ 1 Reply Last reply Reply Quote 0
          • GertjanG
            Gertjan
            last edited by

            @Stewart said in dnsbl Crashing:

            Squid, SquidGuard, Snort, Suricata

            The big memory eaters.

            You're right when you mention the logs files - packages like Suricata do not use rotating log files so the device will implode if the admin doesn't take care of them.
            I never used any of these packages.

            Note : you system, even if the actual memory usage is low (32 %) does use swap memory. This only happens when there is no space (== RAM) left ....

            There are more log files ! Check also the DHCP - DNS (and other with lots of activity) logs

            No "help me" PM's please. Use the forum, the community will thank you.
            Edit : and where are the logs ??

            S 1 Reply Last reply Reply Quote 0
            • S
              Stewart @Gertjan
              last edited by

              @Gertjan Things can be in swap even if RAM isn't full because a process is limited in the amount of RAM it can use. The only times I'm aware of it is Databases such as MySql so I assume that's the cause. Maybe Squid. I could increase the RAM for Squid memory cache but that'll take it away from other things. It's only about 125MB so I'm not concerned.

              1 Reply Last reply Reply Quote 0
              • JeGrJ
                JeGr LAYER 8 Moderator @Stewart
                last edited by

                @Stewart said in dnsbl Crashing:

                Congratulations. I've seen plenty of instances where c-icap, Squid, SquidGuard, Snort, etc. have crashed. Many times it's because of lack of space, usually because a log file (often Snort or Suricata) gets out of control and fills the entire SSD.

                And how does using the watchdog to restart them makes any sense in that cases? If disk is full the service dies. That's normal. It's just like @Gertjan says: simply restarting with a "dumb" service checker doesn't do any good. I've tested the package myself and simply found no use case at all. All points where one could use it have underlying problems as cause that you have to fix yourself (or by correcting settings etc. etc.) so simply hitting restart after restart doesn't do any good to them.

                But besides that, with Surricata and probably other memory eaters, 4GB seem a bit on the very low side when running DNSBL mode with pfBNG. Do you have other memory intensive settings activated in pfBNG?

                Don't forget to upvote 👍 those who kindly offered their time and brainpower to help you!

                If you're interested, I'm available to discuss details of German-speaking paid support (for companies) if needed.

                1 Reply Last reply Reply Quote 0
                • First post
                  Last post
                Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.