Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Monitoring Gateway

    Scheduled Pinned Locked Moved General pfSense Questions
    55 Posts 5 Posters 8.9k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • stephenw10S
      stephenw10 Netgate Administrator
      last edited by

      Huh. That's.... interesting 🤔

      1 Reply Last reply Reply Quote 0
      • hendiH
        hendi
        last edited by

        He stopped again. I don't know what or how I should search in the logs.

        1 Reply Last reply Reply Quote 0
        • stephenw10S
          stephenw10 Netgate Administrator
          last edited by

          It appears the update script is hitting something it's failing to process but it doesn't do it immediately. So what changes when it fails? Is there anything logged at all at that time?

          1 Reply Last reply Reply Quote 0
          • hendiH
            hendi
            last edited by 

            [2.6.0-RELEASE][admin@ccia1.home.arpa]/root: ls -lt /var/db/rrd
total 402
-rw-r--r--  1 nobody  wheel  393168 Jan 10 17:05 ovpns1-packets.rrd
-rw-r--r--  1 nobody  wheel  393168 Jan 10 17:05 ovpns1-traffic.rrd
-rw-r--r--  1 nobody  wheel  393168 Jan 10 17:05 ipsec-packets.rrd
-rw-r--r--  1 nobody  wheel  393168 Jan 10 17:05 ipsec-traffic.rrd
-rw-r--r--  1 nobody  wheel  393168 Jan 10 17:05 opt1-packets.rrd
-rw-r--r--  1 nobody  wheel  393168 Jan 10 17:05 opt1-traffic.rrd
-rw-r--r--  1 nobody  wheel  393168 Jan 10 17:05 lan-packets.rrd
-rw-r--r--  1 nobody  wheel  393168 Jan 10 17:05 lan-traffic.rrd
-rw-r--r--  1 nobody  wheel  393168 Jan 10 17:05 wan-packets.rrd
-rw-r--r--  1 nobody  wheel  393168 Jan 10 17:05 wan-traffic.rrd
-rw-r--r--  1 root    wheel  147848 Jan 10 17:04 WANGW-quality.rrd
-rw-r--r--  1 nobody  wheel  882048 Jan 10 17:04 ntpd.rrd
-rw-r--r--  1 nobody  wheel  275648 Jan 10 17:04 system-sensors.rrd
-rw-r--r--  1 nobody  wheel  588592 Jan 10 17:04 system-mbuf.rrd
-rw-r--r--  1 nobody  wheel  735320 Jan 10 17:04 system-memory.rrd
-rw-r--r--  1 nobody  wheel  245976 Jan 10 17:04 system-processor.rrd
-rw-r--r--  1 nobody  wheel  245976 Jan 10 17:04 system-states.rrd
-rw-r--r--  1 nobody  wheel   49720 Jan 10 17:04 ovpns1-vpnusers.rrd
-rw-r--r--  1 root    wheel    9205 Jan 10 15:48 updaterrd.sh

            [2.6.0-RELEASE][admin@ccia1.home.arpa]/root: grep "rrd" /var/log/system.log
Jan  8 11:35:05 ccia1 php[415]: rc.bootup: Creating rrd update script
Jan  8 16:46:33 ccia1 php[415]: rc.bootup: Creating rrd update script
Jan  8 17:29:28 ccia1 php-fpm[367]: /services_ntpd.php: Creating rrd update script
Jan  8 17:36:42 ccia1 php-fpm[368]: /status_monitoring.php: Creating rrd update script
Jan  8 17:37:29 ccia1 php-fpm[367]: /status_monitoring.php: Creating rrd update script
Jan  8 17:40:40 ccia1 php-fpm[367]: /status_monitoring.php: Creating rrd update script
Jan  8 17:42:29 ccia1 php-fpm[367]: /status_monitoring.php: Creating rrd update script
Jan  8 17:45:32 ccia1 php-fpm[4876]: /status_monitoring.php: Creating rrd update script
Jan  8 17:46:03 ccia1 php-fpm[368]: /status_monitoring.php: Creating rrd update script
Jan  8 18:00:17 ccia1 php-fpm[368]: /status_monitoring.php: Creating rrd update script
Jan  8 18:39:20 ccia1 php-fpm[367]: /status_monitoring.php: Creating rrd update script
Jan  8 19:02:27 ccia1 php-fpm[368]: /status_monitoring.php: Creating rrd update script
Jan  8 19:04:55 ccia1 php-fpm[367]: /status_monitoring.php: Creating rrd update script
Jan 10 15:48:02 ccia1 php-fpm[368]: /status_monitoring.php: Creating rrd update script

            [2.6.0-RELEASE][admin@ccia1.home.arpa]/root: ps -auxwwd | grep rrd
root    21285   0.0  0.1  11212   2484  0  S+   19:57      0:00.00 |       `-- grep rrd
root    90514   0.0  0.1  11592   2860  -  IN   15:48      0:00.62 |-- /bin/sh /var/db/rrd/updaterrd.sh
root    56237   0.0  0.1  11592   2856  -  IN   17:05      0:00.00 | `-- /bin/sh /var/db/rrd/updaterrd.sh

            Is the problem because updaterrd.sh is duplicated? It seems to coincide with the time when the logs stop. But why does it run twice?

            1 Reply Last reply Reply Quote 0
            • stephenw10S
              stephenw10 Netgate Administrator
              last edited by

              Is there anything in the logs at 17.04/5 though? Not just rrd entries?

              hendiH 1 Reply Last reply Reply Quote 0
              • hendiH
                hendi @stephenw10
                last edited by stephenw10

                @stephenw10

                Jan 10 03:23:00 ccia1 sshguard[90295]: Exiting on signal.
Jan 10 03:23:00 ccia1 sshguard[3397]: Now monitoring attacks.
Jan 10 06:49:00 ccia1 sshguard[3397]: Exiting on signal.
Jan 10 06:49:00 ccia1 sshguard[13163]: Now monitoring attacks.
Jan 10 09:25:00 ccia1 sshguard[13163]: Exiting on signal.
Jan 10 09:25:00 ccia1 sshguard[94432]: Now monitoring attacks.
Jan 10 11:09:00 ccia1 sshguard[94432]: Exiting on signal.
Jan 10 11:09:00 ccia1 sshguard[51714]: Now monitoring attacks.
Jan 10 12:00:00 ccia1 sshguard[51714]: Exiting on signal.
Jan 10 12:00:00 ccia1 sshguard[78531]: Now monitoring attacks.
Jan 10 12:30:00 ccia1 php[94000]: rc.update_urltables: /etc/rc.update_urltables: Starting up.
Jan 10 12:30:00 ccia1 php[94000]: rc.update_urltables: /etc/rc.update_urltables: Sleeping for 58 seconds.
Jan 10 12:30:58 ccia1 php[94000]: rc.update_urltables: /etc/rc.update_urltables: Starting URL table alias updates
Jan 10 12:30:58 ccia1 php[94000]: rc.update_urltables: /etc/rc.update_urltables: pfB_Top_v4 does not need updating.
Jan 10 12:30:58 ccia1 php[94000]: rc.update_urltables: /etc/rc.update_urltables: pfB_Top_v6 does not need updating.
Jan 10 12:30:58 ccia1 php[94000]: rc.update_urltables: /etc/rc.update_urltables: pfB_PRI1_v4 does not need updating.
Jan 10 13:24:00 ccia1 sshguard[78531]: Exiting on signal.
Jan 10 13:24:00 ccia1 sshguard[22732]: Now monitoring attacks.
Jan 10 15:47:52 ccia1 php-fpm[367]: /status_logs_settings.php: Configuration Change: admin@::1 (Local Database): Changed system logging options.
Jan 10 15:47:52 ccia1 check_reload_status[397]: Syncing firewall
Jan 10 15:47:52 ccia1 syslogd: exiting on signal 15
Jan 10 15:47:52 ccia1 syslogd: kernel boot file is /boot/kernel/kernel
Jan 10 15:47:52 ccia1 nginx: 2023/01/10 15:47:52 [error] 776#100532: send() failed (54: Connection reset by peer)
Jan 10 15:48:02 ccia1 php-fpm[368]: /status_monitoring.php: Configuration Change: admin@::1 (Local Database): Status: Monitoring: RRD graphing has been enabled.
Jan 10 15:48:02 ccia1 check_reload_status[397]: Syncing firewall
Jan 10 15:48:02 ccia1 php-fpm[368]: /status_monitoring.php: Creating rrd update script
Jan 10 15:55:35 ccia1 sshguard[22836]: Now monitoring attacks.
Jan 10 15:55:44 ccia1 php-fpm[368]: /index.php: Successful login for user 'admin' from: ::1 (Local Database)
Jan 10 15:56:16 ccia1 php-fpm[368]: /pkg_mgr_install.php: Configuration Change: admin@::1 (Local Database): Creating restore point before package installation.
Jan 10 15:56:16 ccia1 check_reload_status[397]: Syncing firewall
Jan 10 15:56:22 ccia1 pkg-static[67905]: libpaper-1.1.28 installed
Jan 10 15:56:22 ccia1 pkg-static[67905]: jpeg-turbo-2.1.1_1 installed
Jan 10 15:56:22 ccia1 pkg-static[67905]: jbigkit-2.1_1 installed
Jan 10 15:56:22 ccia1 pkg-static[67905]: png-1.6.37_1 installed
Jan 10 15:56:22 ccia1 pkg-static[67905]: uchardet-0.0.7 installed
Jan 10 15:56:22 ccia1 pkg-static[67905]: psutils-1.17_5 installed
Jan 10 15:56:23 ccia1 pkg-static[67905]: tiff-4.3.0 installed
Jan 10 15:56:24 ccia1 pkg-static[67905]: giflib-5.2.1 installed
Jan 10 15:56:24 ccia1 pkg-static[67905]: libfontenc-1.1.4 installed
Jan 10 15:56:24 ccia1 pkg-static[67905]: freetype2-2.11.0 installed
Jan 10 15:56:24 ccia1 pkg-static[67905]: openpgm-5.2.122_6 installed
Jan 10 15:56:24 ccia1 pkg-static[67905]: norm-1.5r6_1 installed
Jan 10 15:56:25 ccia1 pkg-static[67905]: groff-1.22.4_4 installed
Jan 10 15:56:26 ccia1 pkg-static[67905]: protobuf-3.17.3,1 installed
Jan 10 15:56:26 ccia1 pkg-static[67905]: webp-1.2.1 installed
Jan 10 15:56:26 ccia1 pkg-static[67905]: mkfontscale-1.2.1 installed
Jan 10 15:56:27 ccia1 pkg-static[67905]: fontconfig-2.13.94_1,1 installed
Jan 10 15:56:27 ccia1 pkg-static[67905]: libsodium-1.0.18 installed
Jan 10 15:56:28 ccia1 pkg-static[67905]: ndpi-4.0.d20210818,1 installed
Jan 10 15:56:28 ccia1 pkg-static[67905]: libzmq4-4.3.4 installed
Jan 10 15:56:28 ccia1 pkg-static[67905]: libmaxminddb-1.6.0 installed
Jan 10 15:56:29 ccia1 pkg-static[67905]: lua53-5.3.6 installed
Jan 10 15:56:30 ccia1 pkg-static[67905]: mysql57-client-5.7.36 installed
Jan 10 15:56:30 ccia1 pkg-static[67905]: zstd-1.5.0 installed
Jan 10 15:56:30 ccia1 pkg-static[67905]: libgd-2.3.1,1 installed
Jan 10 15:56:33 ccia1 pkg-static[67905]: webfonts-0.30_14 installed
Jan 10 15:56:35 ccia1 pkg-static[67905]: ntopng-5.0.d20210923,1 installed
Jan 10 15:56:36 ccia1 pkg-static[67905]: graphviz-2.44.1_17 installed
Jan 10 15:56:37 ccia1 pkg-static[67905]: redis-6.2.6 installed
Jan 10 15:56:37 ccia1 pkg-static[67905]: gdbm-1.22 installed
Jan 10 15:56:37 ccia1 php[68850]: /etc/rc.packages: Beginning package installation for ntopng .
Jan 10 15:56:38 ccia1 php[68850]: /etc/rc.packages: Configuration Change: (system): Intermediate config write during package install for ntopng.
Jan 10 15:56:38 ccia1 check_reload_status[397]: Syncing firewall
Jan 10 15:57:21 ccia1 php[68850]: /etc/rc.packages: Configuration Change: (system): Installed ntopng package.
Jan 10 15:57:21 ccia1 check_reload_status[397]: Syncing firewall
Jan 10 15:57:21 ccia1 php[68850]: /etc/rc.packages: Successfully installed package: ntopng.
Jan 10 15:57:22 ccia1 pkg-static[67905]: pfSense-pkg-ntopng-0.8.13_10 installed
Jan 10 15:57:23 ccia1 check_reload_status[397]: Reloading filter
Jan 10 15:57:23 ccia1 check_reload_status[397]: Starting packages
Jan 10 15:57:24 ccia1 php-fpm[368]: /rc.start_packages: Restarting/Starting all packages.
Jan 10 15:57:24 ccia1 php-fpm[368]: /rc.start_packages: The command '/usr/local/etc/rc.d/ntopng.sh stop' returned exit code '1', the output was 'No matching processes were found No matching processes were found'
Jan 10 15:57:27 ccia1 kernel: re0: promiscuous mode enabled
Jan 10 15:57:27 ccia1 kernel: re1: promiscuous mode enabled
Jan 10 15:57:45 ccia1 ntopng[62182]: [HTTPserver.cpp:1391] ERROR: [HTTP] set_ports_option: cannot bind to 3000s: Address already in use
Jan 10 15:57:45 ccia1 ntopng[62182]: [mongoose.c:4645] ERROR: set_ports_option: cannot bind to 3000s: No error: 0
Jan 10 15:57:45 ccia1 ntopng[62182]: [HTTPserver.cpp:1681] ERROR: Unable to start HTTP server (IPv4) on ports 3000s
Jan 10 15:57:45 ccia1 ntopng[62182]: [HTTPserver.cpp:1687] ERROR: Either port in use or another ntopng instance is running (using the same port)
Jan 10 16:19:09 ccia1 php-fpm[367]: /pkg_edit.php: Configuration Change:
Jan 10 16:19:09 ccia1 check_reload_status[397]: Syncing firewall
Jan 10 16:19:12 ccia1 kernel: re0: promiscuous mode disabled
Jan 10 16:19:12 ccia1 kernel: re1: promiscuous mode disabled
Jan 10 16:19:12 ccia1 php-fpm[367]: /pkg_edit.php: The command '/usr/local/etc/rc.d/ntopng.sh stop' returned exit code '1', the output was 'No matching processes were found No matching processes were found'
Jan 10 16:19:13 ccia1 kernel: re0: promiscuous mode enabled
Jan 10 16:19:13 ccia1 kernel: re1: promiscuous mode enabled
Jan 10 16:25:00 ccia1 sshguard[22836]: Exiting on signal.
Jan 10 16:25:00 ccia1 sshguard[38778]: Now monitoring attacks.
Jan 10 16:32:00 ccia1 sshguard[38778]: Exiting on signal.
Jan 10 16:32:00 ccia1 sshguard[58980]: Now monitoring attacks.
Jan 10 17:08:00 ccia1 sshguard[99400]: Now monitoring attacks.
Jan 10 17:19:28 ccia1 php-fpm[368]: /pkg_edit.php: Configuration Change:
Jan 10 17:19:28 ccia1 check_reload_status[397]: Syncing firewall
Jan 10 17:19:32 ccia1 kernel: re0: promiscuous mode disabled
Jan 10 17:19:32 ccia1 kernel: re1: promiscuous mode disabled
Jan 10 17:19:35 ccia1 php-fpm[368]: /pkg_edit.php: The command '/usr/local/etc/rc.d/ntopng.sh stop' returned exit code '1', the output was 'No matching processes were found No matching processes were found'
Jan 10 17:19:35 ccia1 kernel: re0: promiscuous mode enabled
Jan 10 17:19:35 ccia1 kernel: re1: promiscuous mode enabled
Jan 10 17:19:35 ccia1 kernel: re2: promiscuous mode enabled
Jan 10 17:36:46 ccia1 php-fpm[367]: /pkg_edit.php: Configuration Change:
Jan 10 17:36:46 ccia1 check_reload_status[397]: Syncing firewall
Jan 10 17:36:49 ccia1 kernel: re0: promiscuous mode disabled
Jan 10 17:36:49 ccia1 kernel: re1: promiscuous mode disabled
Jan 10 17:36:49 ccia1 kernel: re2: promiscuous mode disabled
Jan 10 17:36:52 ccia1 php-fpm[367]: /pkg_edit.php: The command '/usr/local/etc/rc.d/ntopng.sh stop' returned exit code '1', the output was 'No matching processes were found No matching processes were found'
Jan 10 17:36:53 ccia1 kernel: re0: promiscuous mode enabled
Jan 10 17:36:53 ccia1 kernel: re1: promiscuous mode enabled
Jan 10 17:36:53 ccia1 kernel: re2: promiscuous mode enabled
Jan 10 17:37:06 ccia1 php-fpm[368]: /pkg_edit.php: Configuration Change:
Jan 10 17:37:06 ccia1 check_reload_status[397]: Syncing firewall
Jan 10 17:37:09 ccia1 kernel: re0: promiscuous mode disabled
Jan 10 17:37:09 ccia1 kernel: re1: promiscuous mode disabled
Jan 10 17:37:09 ccia1 kernel: re2: promiscuous mode disabled
Jan 10 17:37:12 ccia1 php-fpm[368]: /pkg_edit.php: The command '/usr/local/etc/rc.d/ntopng.sh stop' returned exit code '1', the output was 'No matching processes were found No matching processes were found'
Jan 10 17:37:13 ccia1 kernel: re0: promiscuous mode enabled
Jan 10 17:37:13 ccia1 kernel: re2: promiscuous mode enabled
Jan 10 17:43:00 ccia1 sshguard[99400]: Exiting on signal.
Jan 10 17:43:00 ccia1 sshguard[88504]: Now monitoring attacks.
, client: 10.1.0.2, server: 0.0.0.0:48562
Jan 10 18:26:00 ccia1 sshguard[88504]: Exiting on signal.
Jan 10 18:26:00 ccia1 sshguard[14898]: Now monitoring attacks.
Jan 10 19:15:00 ccia1 sshguard[14898]: Exiting on signal.
Jan 10 19:15:00 ccia1 sshguard[45834]: Now monitoring attacks.
Jan 10 20:04:00 ccia1 sshguard[45834]: Exiting on signal.
Jan 10 20:04:00 ccia1 sshguard[87400]: Now monitoring attacks.
Jan 10 20:04:04 ccia1 nginx: 2023/01/10 20:04:04 [crit] 601#100529: *2472 SSL_write() failed (13: Permission denied) while processing HTTP/2 connection, client: 10.1.0.2, server: 0.0.0.0:48562
Jan 10 20:56:00 ccia1 sshguard[87400]: Exiting on signal.
Jan 10 20:56:00 ccia1 sshguard[16073]: Now monitoring attacks.
Jan 10 21:48:00 ccia1 sshguard[16073]: Exiting on signal.
Jan 10 21:48:00 ccia1 sshguard[47268]: Now monitoring attacks.
Jan 10 22:43:00 ccia1 sshguard[47268]: Exiting on signal.
Jan 10 22:43:00 ccia1 sshguard[82446]: Now monitoring attacks.
Jan 10 23:39:00 ccia1 sshguard[82446]: Exiting on signal.
Jan 10 23:39:00 ccia1 sshguard[18269]: Now monitoring attacks.
                1 Reply Last reply Reply Quote 0
                • stephenw10S
                  stephenw10 Netgate Administrator
                  last edited by

                  Hmm, so not really.

                  What packages do you have running there?

                  1 Reply Last reply Reply Quote 0
                  • hendiH
                    hendi
                    last edited by

                    ntopng net 0.8.13_10
                    openvpn-client-export security 1.6_9
                    pfBlockerNG-devel net 3.1.0_11

                    1 Reply Last reply Reply Quote 0
                    • stephenw10S
                      stephenw10 Netgate Administrator
                      last edited by

                      Hmm, you might try disabling ntop-ng as a test. It can use resources and cause problems. Though I don't see any direct evidence of that here.

                      hendiH 1 Reply Last reply Reply Quote 0
                      • hendiH
                        hendi @stephenw10
                        last edited by

                        @stephenw10 said in Monitoring Gateway:

                        Hmm, you might try disabling ntop-ng as a test. It can use resources and cause problems. Though I don't see any direct evidence of that here.

                        I tried this from the beginning but without result.

                        1 Reply Last reply Reply Quote 0
                        • First post
                          Last post
                        Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.