Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Monitoring Gateway

    Scheduled Pinned Locked Moved General pfSense Questions
    55 Posts 5 Posters 8.5k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • stephenw10S
      stephenw10 Netgate Administrator
      last edited by

      Clearly something in your install is starting that script incorrectly or the script is getting hung up on something trying to update it. I would have expected clearing the RRD data to remove whatever that might be though.
      I'd probably look at reinstalling at this point. At least that would narrow it to a config problem or something specific to your install.

      Steve

      johnpozJ 1 Reply Last reply Reply Quote 1
      • johnpozJ
        johnpoz LAYER 8 Global Moderator @stephenw10
        last edited by

        @stephenw10 I concur with the reinstall.. If there was something amiss with it in general you would think the forums would be a fire with people having issues with their monitoring. So it has to be something odd with the install or issue with something being monitored that is unique to the setup that is causing the issue.

        An intelligent man is sometimes forced to be drunk to spend time with his fools
        If you get confused: Listen to the Music Play
        Please don't Chat/PM me for help, unless mod related
        SG-4860 24.11 | Lab VMs 2.8, 24.11

        1 Reply Last reply Reply Quote 1
        • hendiH
          hendi
          last edited by

          I will reinstall pfsense at some point. After installation, monitoring worked until a certain point. Unfortunately, I don't have all the logs anymore, but I think it coincided with the moment when I added another network card for a separate network. Thanks

          1 Reply Last reply Reply Quote 0
          • hendiH
            hendi
            last edited by

            I have now installed ntopng. Monitoring has started to work.😍 Let's see for how long.

            1 Reply Last reply Reply Quote 0
            • stephenw10S
              stephenw10 Netgate Administrator
              last edited by

              Huh. That's.... interesting 🤔

              1 Reply Last reply Reply Quote 0
              • hendiH
                hendi
                last edited by

                He stopped again. I don't know what or how I should search in the logs.

                1 Reply Last reply Reply Quote 0
                • stephenw10S
                  stephenw10 Netgate Administrator
                  last edited by

                  It appears the update script is hitting something it's failing to process but it doesn't do it immediately. So what changes when it fails? Is there anything logged at all at that time?

                  1 Reply Last reply Reply Quote 0
                  • hendiH
                    hendi
                    last edited by 

                    [2.6.0-RELEASE][admin@ccia1.home.arpa]/root: ls -lt /var/db/rrd
total 402
-rw-r--r--  1 nobody  wheel  393168 Jan 10 17:05 ovpns1-packets.rrd
-rw-r--r--  1 nobody  wheel  393168 Jan 10 17:05 ovpns1-traffic.rrd
-rw-r--r--  1 nobody  wheel  393168 Jan 10 17:05 ipsec-packets.rrd
-rw-r--r--  1 nobody  wheel  393168 Jan 10 17:05 ipsec-traffic.rrd
-rw-r--r--  1 nobody  wheel  393168 Jan 10 17:05 opt1-packets.rrd
-rw-r--r--  1 nobody  wheel  393168 Jan 10 17:05 opt1-traffic.rrd
-rw-r--r--  1 nobody  wheel  393168 Jan 10 17:05 lan-packets.rrd
-rw-r--r--  1 nobody  wheel  393168 Jan 10 17:05 lan-traffic.rrd
-rw-r--r--  1 nobody  wheel  393168 Jan 10 17:05 wan-packets.rrd
-rw-r--r--  1 nobody  wheel  393168 Jan 10 17:05 wan-traffic.rrd
-rw-r--r--  1 root    wheel  147848 Jan 10 17:04 WANGW-quality.rrd
-rw-r--r--  1 nobody  wheel  882048 Jan 10 17:04 ntpd.rrd
-rw-r--r--  1 nobody  wheel  275648 Jan 10 17:04 system-sensors.rrd
-rw-r--r--  1 nobody  wheel  588592 Jan 10 17:04 system-mbuf.rrd
-rw-r--r--  1 nobody  wheel  735320 Jan 10 17:04 system-memory.rrd
-rw-r--r--  1 nobody  wheel  245976 Jan 10 17:04 system-processor.rrd
-rw-r--r--  1 nobody  wheel  245976 Jan 10 17:04 system-states.rrd
-rw-r--r--  1 nobody  wheel   49720 Jan 10 17:04 ovpns1-vpnusers.rrd
-rw-r--r--  1 root    wheel    9205 Jan 10 15:48 updaterrd.sh

                    [2.6.0-RELEASE][admin@ccia1.home.arpa]/root: grep "rrd" /var/log/system.log
Jan  8 11:35:05 ccia1 php[415]: rc.bootup: Creating rrd update script
Jan  8 16:46:33 ccia1 php[415]: rc.bootup: Creating rrd update script
Jan  8 17:29:28 ccia1 php-fpm[367]: /services_ntpd.php: Creating rrd update script
Jan  8 17:36:42 ccia1 php-fpm[368]: /status_monitoring.php: Creating rrd update script
Jan  8 17:37:29 ccia1 php-fpm[367]: /status_monitoring.php: Creating rrd update script
Jan  8 17:40:40 ccia1 php-fpm[367]: /status_monitoring.php: Creating rrd update script
Jan  8 17:42:29 ccia1 php-fpm[367]: /status_monitoring.php: Creating rrd update script
Jan  8 17:45:32 ccia1 php-fpm[4876]: /status_monitoring.php: Creating rrd update script
Jan  8 17:46:03 ccia1 php-fpm[368]: /status_monitoring.php: Creating rrd update script
Jan  8 18:00:17 ccia1 php-fpm[368]: /status_monitoring.php: Creating rrd update script
Jan  8 18:39:20 ccia1 php-fpm[367]: /status_monitoring.php: Creating rrd update script
Jan  8 19:02:27 ccia1 php-fpm[368]: /status_monitoring.php: Creating rrd update script
Jan  8 19:04:55 ccia1 php-fpm[367]: /status_monitoring.php: Creating rrd update script
Jan 10 15:48:02 ccia1 php-fpm[368]: /status_monitoring.php: Creating rrd update script

                    [2.6.0-RELEASE][admin@ccia1.home.arpa]/root: ps -auxwwd | grep rrd
root    21285   0.0  0.1  11212   2484  0  S+   19:57      0:00.00 |       `-- grep rrd
root    90514   0.0  0.1  11592   2860  -  IN   15:48      0:00.62 |-- /bin/sh /var/db/rrd/updaterrd.sh
root    56237   0.0  0.1  11592   2856  -  IN   17:05      0:00.00 | `-- /bin/sh /var/db/rrd/updaterrd.sh

                    Is the problem because updaterrd.sh is duplicated? It seems to coincide with the time when the logs stop. But why does it run twice?

                    1 Reply Last reply Reply Quote 0
                    • stephenw10S
                      stephenw10 Netgate Administrator
                      last edited by

                      Is there anything in the logs at 17.04/5 though? Not just rrd entries?

                      hendiH 1 Reply Last reply Reply Quote 0
                      • hendiH
                        hendi @stephenw10
                        last edited by stephenw10

                        @stephenw10

                        Jan 10 03:23:00 ccia1 sshguard[90295]: Exiting on signal.
Jan 10 03:23:00 ccia1 sshguard[3397]: Now monitoring attacks.
Jan 10 06:49:00 ccia1 sshguard[3397]: Exiting on signal.
Jan 10 06:49:00 ccia1 sshguard[13163]: Now monitoring attacks.
Jan 10 09:25:00 ccia1 sshguard[13163]: Exiting on signal.
Jan 10 09:25:00 ccia1 sshguard[94432]: Now monitoring attacks.
Jan 10 11:09:00 ccia1 sshguard[94432]: Exiting on signal.
Jan 10 11:09:00 ccia1 sshguard[51714]: Now monitoring attacks.
Jan 10 12:00:00 ccia1 sshguard[51714]: Exiting on signal.
Jan 10 12:00:00 ccia1 sshguard[78531]: Now monitoring attacks.
Jan 10 12:30:00 ccia1 php[94000]: rc.update_urltables: /etc/rc.update_urltables: Starting up.
Jan 10 12:30:00 ccia1 php[94000]: rc.update_urltables: /etc/rc.update_urltables: Sleeping for 58 seconds.
Jan 10 12:30:58 ccia1 php[94000]: rc.update_urltables: /etc/rc.update_urltables: Starting URL table alias updates
Jan 10 12:30:58 ccia1 php[94000]: rc.update_urltables: /etc/rc.update_urltables: pfB_Top_v4 does not need updating.
Jan 10 12:30:58 ccia1 php[94000]: rc.update_urltables: /etc/rc.update_urltables: pfB_Top_v6 does not need updating.
Jan 10 12:30:58 ccia1 php[94000]: rc.update_urltables: /etc/rc.update_urltables: pfB_PRI1_v4 does not need updating.
Jan 10 13:24:00 ccia1 sshguard[78531]: Exiting on signal.
Jan 10 13:24:00 ccia1 sshguard[22732]: Now monitoring attacks.
Jan 10 15:47:52 ccia1 php-fpm[367]: /status_logs_settings.php: Configuration Change: admin@::1 (Local Database): Changed system logging options.
Jan 10 15:47:52 ccia1 check_reload_status[397]: Syncing firewall
Jan 10 15:47:52 ccia1 syslogd: exiting on signal 15
Jan 10 15:47:52 ccia1 syslogd: kernel boot file is /boot/kernel/kernel
Jan 10 15:47:52 ccia1 nginx: 2023/01/10 15:47:52 [error] 776#100532: send() failed (54: Connection reset by peer)
Jan 10 15:48:02 ccia1 php-fpm[368]: /status_monitoring.php: Configuration Change: admin@::1 (Local Database): Status: Monitoring: RRD graphing has been enabled.
Jan 10 15:48:02 ccia1 check_reload_status[397]: Syncing firewall
Jan 10 15:48:02 ccia1 php-fpm[368]: /status_monitoring.php: Creating rrd update script
Jan 10 15:55:35 ccia1 sshguard[22836]: Now monitoring attacks.
Jan 10 15:55:44 ccia1 php-fpm[368]: /index.php: Successful login for user 'admin' from: ::1 (Local Database)
Jan 10 15:56:16 ccia1 php-fpm[368]: /pkg_mgr_install.php: Configuration Change: admin@::1 (Local Database): Creating restore point before package installation.
Jan 10 15:56:16 ccia1 check_reload_status[397]: Syncing firewall
Jan 10 15:56:22 ccia1 pkg-static[67905]: libpaper-1.1.28 installed
Jan 10 15:56:22 ccia1 pkg-static[67905]: jpeg-turbo-2.1.1_1 installed
Jan 10 15:56:22 ccia1 pkg-static[67905]: jbigkit-2.1_1 installed
Jan 10 15:56:22 ccia1 pkg-static[67905]: png-1.6.37_1 installed
Jan 10 15:56:22 ccia1 pkg-static[67905]: uchardet-0.0.7 installed
Jan 10 15:56:22 ccia1 pkg-static[67905]: psutils-1.17_5 installed
Jan 10 15:56:23 ccia1 pkg-static[67905]: tiff-4.3.0 installed
Jan 10 15:56:24 ccia1 pkg-static[67905]: giflib-5.2.1 installed
Jan 10 15:56:24 ccia1 pkg-static[67905]: libfontenc-1.1.4 installed
Jan 10 15:56:24 ccia1 pkg-static[67905]: freetype2-2.11.0 installed
Jan 10 15:56:24 ccia1 pkg-static[67905]: openpgm-5.2.122_6 installed
Jan 10 15:56:24 ccia1 pkg-static[67905]: norm-1.5r6_1 installed
Jan 10 15:56:25 ccia1 pkg-static[67905]: groff-1.22.4_4 installed
Jan 10 15:56:26 ccia1 pkg-static[67905]: protobuf-3.17.3,1 installed
Jan 10 15:56:26 ccia1 pkg-static[67905]: webp-1.2.1 installed
Jan 10 15:56:26 ccia1 pkg-static[67905]: mkfontscale-1.2.1 installed
Jan 10 15:56:27 ccia1 pkg-static[67905]: fontconfig-2.13.94_1,1 installed
Jan 10 15:56:27 ccia1 pkg-static[67905]: libsodium-1.0.18 installed
Jan 10 15:56:28 ccia1 pkg-static[67905]: ndpi-4.0.d20210818,1 installed
Jan 10 15:56:28 ccia1 pkg-static[67905]: libzmq4-4.3.4 installed
Jan 10 15:56:28 ccia1 pkg-static[67905]: libmaxminddb-1.6.0 installed
Jan 10 15:56:29 ccia1 pkg-static[67905]: lua53-5.3.6 installed
Jan 10 15:56:30 ccia1 pkg-static[67905]: mysql57-client-5.7.36 installed
Jan 10 15:56:30 ccia1 pkg-static[67905]: zstd-1.5.0 installed
Jan 10 15:56:30 ccia1 pkg-static[67905]: libgd-2.3.1,1 installed
Jan 10 15:56:33 ccia1 pkg-static[67905]: webfonts-0.30_14 installed
Jan 10 15:56:35 ccia1 pkg-static[67905]: ntopng-5.0.d20210923,1 installed
Jan 10 15:56:36 ccia1 pkg-static[67905]: graphviz-2.44.1_17 installed
Jan 10 15:56:37 ccia1 pkg-static[67905]: redis-6.2.6 installed
Jan 10 15:56:37 ccia1 pkg-static[67905]: gdbm-1.22 installed
Jan 10 15:56:37 ccia1 php[68850]: /etc/rc.packages: Beginning package installation for ntopng .
Jan 10 15:56:38 ccia1 php[68850]: /etc/rc.packages: Configuration Change: (system): Intermediate config write during package install for ntopng.
Jan 10 15:56:38 ccia1 check_reload_status[397]: Syncing firewall
Jan 10 15:57:21 ccia1 php[68850]: /etc/rc.packages: Configuration Change: (system): Installed ntopng package.
Jan 10 15:57:21 ccia1 check_reload_status[397]: Syncing firewall
Jan 10 15:57:21 ccia1 php[68850]: /etc/rc.packages: Successfully installed package: ntopng.
Jan 10 15:57:22 ccia1 pkg-static[67905]: pfSense-pkg-ntopng-0.8.13_10 installed
Jan 10 15:57:23 ccia1 check_reload_status[397]: Reloading filter
Jan 10 15:57:23 ccia1 check_reload_status[397]: Starting packages
Jan 10 15:57:24 ccia1 php-fpm[368]: /rc.start_packages: Restarting/Starting all packages.
Jan 10 15:57:24 ccia1 php-fpm[368]: /rc.start_packages: The command '/usr/local/etc/rc.d/ntopng.sh stop' returned exit code '1', the output was 'No matching processes were found No matching processes were found'
Jan 10 15:57:27 ccia1 kernel: re0: promiscuous mode enabled
Jan 10 15:57:27 ccia1 kernel: re1: promiscuous mode enabled
Jan 10 15:57:45 ccia1 ntopng[62182]: [HTTPserver.cpp:1391] ERROR: [HTTP] set_ports_option: cannot bind to 3000s: Address already in use
Jan 10 15:57:45 ccia1 ntopng[62182]: [mongoose.c:4645] ERROR: set_ports_option: cannot bind to 3000s: No error: 0
Jan 10 15:57:45 ccia1 ntopng[62182]: [HTTPserver.cpp:1681] ERROR: Unable to start HTTP server (IPv4) on ports 3000s
Jan 10 15:57:45 ccia1 ntopng[62182]: [HTTPserver.cpp:1687] ERROR: Either port in use or another ntopng instance is running (using the same port)
Jan 10 16:19:09 ccia1 php-fpm[367]: /pkg_edit.php: Configuration Change:
Jan 10 16:19:09 ccia1 check_reload_status[397]: Syncing firewall
Jan 10 16:19:12 ccia1 kernel: re0: promiscuous mode disabled
Jan 10 16:19:12 ccia1 kernel: re1: promiscuous mode disabled
Jan 10 16:19:12 ccia1 php-fpm[367]: /pkg_edit.php: The command '/usr/local/etc/rc.d/ntopng.sh stop' returned exit code '1', the output was 'No matching processes were found No matching processes were found'
Jan 10 16:19:13 ccia1 kernel: re0: promiscuous mode enabled
Jan 10 16:19:13 ccia1 kernel: re1: promiscuous mode enabled
Jan 10 16:25:00 ccia1 sshguard[22836]: Exiting on signal.
Jan 10 16:25:00 ccia1 sshguard[38778]: Now monitoring attacks.
Jan 10 16:32:00 ccia1 sshguard[38778]: Exiting on signal.
Jan 10 16:32:00 ccia1 sshguard[58980]: Now monitoring attacks.
Jan 10 17:08:00 ccia1 sshguard[99400]: Now monitoring attacks.
Jan 10 17:19:28 ccia1 php-fpm[368]: /pkg_edit.php: Configuration Change:
Jan 10 17:19:28 ccia1 check_reload_status[397]: Syncing firewall
Jan 10 17:19:32 ccia1 kernel: re0: promiscuous mode disabled
Jan 10 17:19:32 ccia1 kernel: re1: promiscuous mode disabled
Jan 10 17:19:35 ccia1 php-fpm[368]: /pkg_edit.php: The command '/usr/local/etc/rc.d/ntopng.sh stop' returned exit code '1', the output was 'No matching processes were found No matching processes were found'
Jan 10 17:19:35 ccia1 kernel: re0: promiscuous mode enabled
Jan 10 17:19:35 ccia1 kernel: re1: promiscuous mode enabled
Jan 10 17:19:35 ccia1 kernel: re2: promiscuous mode enabled
Jan 10 17:36:46 ccia1 php-fpm[367]: /pkg_edit.php: Configuration Change:
Jan 10 17:36:46 ccia1 check_reload_status[397]: Syncing firewall
Jan 10 17:36:49 ccia1 kernel: re0: promiscuous mode disabled
Jan 10 17:36:49 ccia1 kernel: re1: promiscuous mode disabled
Jan 10 17:36:49 ccia1 kernel: re2: promiscuous mode disabled
Jan 10 17:36:52 ccia1 php-fpm[367]: /pkg_edit.php: The command '/usr/local/etc/rc.d/ntopng.sh stop' returned exit code '1', the output was 'No matching processes were found No matching processes were found'
Jan 10 17:36:53 ccia1 kernel: re0: promiscuous mode enabled
Jan 10 17:36:53 ccia1 kernel: re1: promiscuous mode enabled
Jan 10 17:36:53 ccia1 kernel: re2: promiscuous mode enabled
Jan 10 17:37:06 ccia1 php-fpm[368]: /pkg_edit.php: Configuration Change:
Jan 10 17:37:06 ccia1 check_reload_status[397]: Syncing firewall
Jan 10 17:37:09 ccia1 kernel: re0: promiscuous mode disabled
Jan 10 17:37:09 ccia1 kernel: re1: promiscuous mode disabled
Jan 10 17:37:09 ccia1 kernel: re2: promiscuous mode disabled
Jan 10 17:37:12 ccia1 php-fpm[368]: /pkg_edit.php: The command '/usr/local/etc/rc.d/ntopng.sh stop' returned exit code '1', the output was 'No matching processes were found No matching processes were found'
Jan 10 17:37:13 ccia1 kernel: re0: promiscuous mode enabled
Jan 10 17:37:13 ccia1 kernel: re2: promiscuous mode enabled
Jan 10 17:43:00 ccia1 sshguard[99400]: Exiting on signal.
Jan 10 17:43:00 ccia1 sshguard[88504]: Now monitoring attacks.
, client: 10.1.0.2, server: 0.0.0.0:48562
Jan 10 18:26:00 ccia1 sshguard[88504]: Exiting on signal.
Jan 10 18:26:00 ccia1 sshguard[14898]: Now monitoring attacks.
Jan 10 19:15:00 ccia1 sshguard[14898]: Exiting on signal.
Jan 10 19:15:00 ccia1 sshguard[45834]: Now monitoring attacks.
Jan 10 20:04:00 ccia1 sshguard[45834]: Exiting on signal.
Jan 10 20:04:00 ccia1 sshguard[87400]: Now monitoring attacks.
Jan 10 20:04:04 ccia1 nginx: 2023/01/10 20:04:04 [crit] 601#100529: *2472 SSL_write() failed (13: Permission denied) while processing HTTP/2 connection, client: 10.1.0.2, server: 0.0.0.0:48562
Jan 10 20:56:00 ccia1 sshguard[87400]: Exiting on signal.
Jan 10 20:56:00 ccia1 sshguard[16073]: Now monitoring attacks.
Jan 10 21:48:00 ccia1 sshguard[16073]: Exiting on signal.
Jan 10 21:48:00 ccia1 sshguard[47268]: Now monitoring attacks.
Jan 10 22:43:00 ccia1 sshguard[47268]: Exiting on signal.
Jan 10 22:43:00 ccia1 sshguard[82446]: Now monitoring attacks.
Jan 10 23:39:00 ccia1 sshguard[82446]: Exiting on signal.
Jan 10 23:39:00 ccia1 sshguard[18269]: Now monitoring attacks.
                        1 Reply Last reply Reply Quote 0
                        • stephenw10S
                          stephenw10 Netgate Administrator
                          last edited by

                          Hmm, so not really.

                          What packages do you have running there?

                          1 Reply Last reply Reply Quote 0
                          • hendiH
                            hendi
                            last edited by

                            ntopng net 0.8.13_10
                            openvpn-client-export security 1.6_9
                            pfBlockerNG-devel net 3.1.0_11

                            1 Reply Last reply Reply Quote 0
                            • stephenw10S
                              stephenw10 Netgate Administrator
                              last edited by

                              Hmm, you might try disabling ntop-ng as a test. It can use resources and cause problems. Though I don't see any direct evidence of that here.

                              hendiH 1 Reply Last reply Reply Quote 0
                              • hendiH
                                hendi @stephenw10
                                last edited by

                                @stephenw10 said in Monitoring Gateway:

                                Hmm, you might try disabling ntop-ng as a test. It can use resources and cause problems. Though I don't see any direct evidence of that here.

                                I tried this from the beginning but without result.

                                1 Reply Last reply Reply Quote 0
                                • First post
                                  Last post
                                Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.