Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    pfsense 2.6.0 sshguard @ web gui bug/crash

    Scheduled Pinned Locked Moved General pfSense Questions
    108 Posts 4 Posters 25.6k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • VioletDragonV
      VioletDragon @stephenw10
      last edited by

      @stephenw10 503 errors is when I try and connect through Haproxy with my valid SSL cert but I know that is not the problem as It does the same when connecting using its IP instead of FQDN.

      1 Reply Last reply Reply Quote 0
      • stephenw10S
        stephenw10 Netgate Administrator
        last edited by

        Ok. Seeing 50X errors trying to access the GUI is usually an indication php isn't responding. SSH continues to function since it doesn't require php.

        I think those sshguard logs are a symptom not a cause here. 1m is probably the minimum log rotation time and that's why you're seeing them. It implies that something is absolutely hammering one of the logs when the WAN is connected. It should be pretty obvious which one of you look in /var/log

        Also try running top -aSH via ssh when the gui is down. What processes are being used?

        Steve

        VioletDragonV 1 Reply Last reply Reply Quote 0
        • VioletDragonV
          VioletDragon @stephenw10
          last edited by

          @stephenw10 Screenshot from 2022-04-14 13-48-15.png

          You can see that the CPU is not pegged, it is behaving itself now as it seems. SSHGuard is not reloading as frequent.

          1 Reply Last reply Reply Quote 0
          • stephenw10S
            stephenw10 Netgate Administrator
            last edited by

            Mmm, nothing unexpected shown there. If it's now behaving there wouldn't be though.

            I would check the log file though. I bet you will find one of the logs has numerous compressed backups from rotations. I could imagine it being the nginx log if the gui was unavailable.

            Steve

            VioletDragonV 1 Reply Last reply Reply Quote 0
            • VioletDragonV
              VioletDragon @stephenw10
              last edited by

              @stephenw10 Where would the location be of those files ? i was using the auth.log to check.

              Thanks.

              Jack.

              GertjanG 1 Reply Last reply Reply Quote 0
              • GertjanG
                Gertjan @VioletDragon
                last edited by

                @violetdragon

                /var/log/

                No "help me" PM's please. Use the forum, the community will thank you.
                Edit : and where are the logs ??

                VioletDragonV 1 Reply Last reply Reply Quote 0
                • VioletDragonV
                  VioletDragon @Gertjan
                  last edited by

                  @gertjan that is where I have been looking, I just took a look at /var/log/nginx/error.log but there the only errors there are is from 2020,

                  2020/07/31 12:48:14 [crit] 4493#100471: *102827 SSL_write() failed (13: Permission denied) while processing HTTP/2 connection, client: 192.168.1.9, server: 0.0.0.0:4430
                  2020/08/26 15:32:03 [crit] 8968#100454: *25803 SSL_write() failed (13: Permission denied) while processing HTTP/2 connection, client: 192.168.1.9, server: 0.0.0.0:10443
                  2020/09/08 04:19:59 [error] 4127#100429: *20842 upstream timed out (60: Operation timed out) while reading response header from upstream, client: 192.168.1.9, server: , request: "POST /acme/acme_certificates.php HTTP/2.0", upstream: "fastcgi://unix:/var/run/php-fpm.socket", host: "violetdragon.ddns.net:10443", referrer: "https://violetdragon.ddns.net:10443/acme/acme_certificates.php"
                  
                  GertjanG 1 Reply Last reply Reply Quote 0
                  • stephenw10S
                    stephenw10 Netgate Administrator
                    last edited by

                    For example on this box I'm doing some OpenVPN testing and have logging turned up and a bunch of clients failing to connect:

                    [22.05-DEVELOPMENT][admin@plusdev-2.stevew.lan]/root: ls -ls /var/log
                    total 2176
                    480 -rw-------  1 root  wheel  458507 Apr 14 14:19 auth.log
                     40 -rw-------  1 root  wheel   38536 Apr 14 10:00 auth.log.0.bz2
                     40 -rw-------  1 root  wheel   38625 Apr 14 04:52 auth.log.1.bz2
                     40 -rw-------  1 root  wheel   40401 Apr 13 23:46 auth.log.2.bz2
                     40 -rw-r--r--  1 root  wheel   37447 Apr 12 21:52 bsdinstall_log
                     16 -rw-------  1 root  wheel   12416 Apr 14 13:22 dhcpd.log
                     12 -rw-r--r--  1 root  wheel    9237 Apr 12 22:27 dmesg.boot
                    100 -rw-------  1 root  wheel   99080 Apr 14 12:42 filter.log
                     16 -rw-------  1 root  wheel   15266 Apr 13 13:14 filter.log.0.bz2
                     16 -rw-------  1 root  wheel   13850 Apr 13 03:54 filter.log.1.bz2
                      4 -rw-------  1 root  wheel    1862 Apr 12 22:27 gateways.log
                      0 -rw-------  1 root  wheel       0 Apr 12 22:27 ipsec.log
                      0 -rw-------  1 root  wheel       0 Apr 12 22:27 l2tps.log
                      0 -rw-r--r--  1 root  wheel       0 Apr 12 22:15 lastlog
                      4 drwxr-xr-x  2 root  wheel     512 Apr 12 22:09 nginx
                    108 -rw-------  1 root  wheel  108014 Apr 14 14:10 nginx.log
                      4 drwxr-xr-x  2 root  wheel     512 Apr 12 22:09 ntp
                    376 -rw-------  1 root  wheel  382407 Apr 14 12:42 ntpd.log
                    168 -rw-------  1 root  wheel  171541 Apr 14 14:19 openvpn.log
                     24 -rw-------  1 root  wheel   21875 Apr 14 14:15 openvpn.log.0.bz2
                     24 -rw-------  1 root  wheel   21089 Apr 14 14:01 openvpn.log.1.bz2
                     24 -rw-------  1 root  wheel   21162 Apr 14 13:48 openvpn.log.2.bz2
                     24 -rw-------  1 root  wheel   20837 Apr 14 13:35 openvpn.log.3.bz2
                     24 -rw-------  1 root  wheel   20897 Apr 14 13:22 openvpn.log.4.bz2
                     24 -rw-------  1 root  wheel   21290 Apr 14 13:09 openvpn.log.5.bz2
                     24 -rw-------  1 root  wheel   21258 Apr 14 12:56 openvpn.log.6.bz2
                      0 -rw-------  1 root  wheel       0 Apr 12 22:27 poes.log
                      0 -rw-------  1 root  wheel       0 Apr 12 22:27 portalauth.log
                      0 -rw-------  1 root  wheel       0 Apr 12 22:27 ppp.log
                      4 -rw-------  1 root  wheel    2932 Apr 14 09:32 resolver.log
                     44 -rw-------  1 root  wheel   41582 Apr 14 14:14 routing.log
                    480 -rw-------  1 root  wheel  429823 Apr 14 14:19 system.log
                      8 -rw-------  1 root  wheel    4663 Apr 13 14:55 userlog
                      4 -rw-r--r--  1 root  wheel     394 Apr 14 14:18 utx.lastlogin
                      4 -rw-------  1 root  wheel     284 Apr 14 14:18 utx.log
                      0 -rw-------  1 root  wheel       0 Apr 12 22:27 vpn.log
                      0 -rw-------  1 root  wheel       0 Apr 12 22:27 watchdogd.log
                      0 -rw-------  1 root  wheel       0 Apr 12 22:27 wireless.log
                    

                    It's rotating the logs every 15min which is quite rapid.
                    You can see the times there coincide exactly with this in the system log:

                    Apr 14 12:56:00 	sshguard 	8441 	Exiting on signal.
                    Apr 14 12:56:00 	sshguard 	8983 	Now monitoring attacks.
                    Apr 14 13:09:00 	sshguard 	8983 	Exiting on signal.
                    Apr 14 13:09:00 	sshguard 	54656 	Now monitoring attacks.
                    Apr 14 13:22:00 	sshguard 	54656 	Exiting on signal.
                    Apr 14 13:22:00 	sshguard 	84504 	Now monitoring attacks.
                    Apr 14 13:35:00 	sshguard 	84504 	Exiting on signal.
                    Apr 14 13:35:00 	sshguard 	98800 	Now monitoring attacks.
                    Apr 14 13:48:00 	sshguard 	98800 	Exiting on signal.
                    Apr 14 13:48:00 	sshguard 	60475 	Now monitoring attacks.
                    Apr 14 14:01:00 	sshguard 	60475 	Exiting on signal.
                    Apr 14 14:01:00 	sshguard 	517 	Now monitoring attacks.
                    Apr 14 14:15:00 	sshguard 	517 	Exiting on signal.
                    Apr 14 14:15:00 	sshguard 	53773 	Now monitoring attacks. 
                    

                    I suspect one of you logs files was rotating every minute and that should give you a good idea of what was happening.

                    Steve

                    VioletDragonV 2 Replies Last reply Reply Quote 0
                    • VioletDragonV
                      VioletDragon @stephenw10
                      last edited by

                      @stephenw10 Urmm strange one, it is behaving itself now, but there aren't any errors in the Nginx error.log recent anyway, i guess ill keep an eye on it but when it does start acting up again i will report back i will also look at top to see if it gives me any errors.

                      Something i did do though was run a Memtest on the Memory but it passed the tests so not a memory issue.

                      1 Reply Last reply Reply Quote 0
                      • VioletDragonV
                        VioletDragon @stephenw10
                        last edited by

                        @stephenw10 Not sure if could be related to the Gui or not but the NTP section in the Gui shows no peers then when you go to the NTP under Status it show's that is not running, Haproxy statistics also doing the same, so maybe there could be something borked with the Gui what do you think?

                        Screenshot from 2022-04-14 14-36-35.png Screenshot from 2022-04-14 14-36-55.png Screenshot from 2022-04-14 14-37-19.png

                        1 Reply Last reply Reply Quote 0
                        • stephenw10S
                          stephenw10 Netgate Administrator
                          last edited by

                          What does the ntp log show?

                          But do you have any log files that have rotated far faster than the others? That's what I'm showing you above. What's the output of: ls -ls /var/log

                          VioletDragonV 1 Reply Last reply Reply Quote 0
                          • VioletDragonV
                            VioletDragon @stephenw10
                            last edited by

                            @stephenw10 Heres the output of,

                            [2.6.0-RELEASE][jack@pfSense.violets.lan]/var/log: ls -ls
                            total 1970
                            89 -rw-------  1 root        wheel       476444 Apr 14 16:07 auth.log
                            33 -rw-------  1 root        wheel        28893 Apr 11 09:56 auth.log.0.bz2
                            29 -rw-------  1 root        wheel        28032 Apr  8 19:11 auth.log.1.bz2
                            33 -rw-------  1 root        wheel        28998 Apr  6 04:29 auth.log.2.bz2
                            33 -rw-------  1 root        wheel        29217 Apr  2 19:54 auth.log.3.bz2
                            29 -rw-------  1 root        wheel        28120 Mar 30 12:00 auth.log.4.bz2
                            29 -rw-------  1 root        wheel        27687 Mar 27 05:35 auth.log.5.bz2
                            29 -rw-------  1 root        wheel        27999 Mar 24 07:37 auth.log.6.bz2
                            13 -rw-r--r--  1 root        wheel        61784 Jul 11  2020 bsdinstall_log
                             9 -rw-------  1 root        wheel        42842 Apr 14 16:06 dhcpd.log
                            17 -rw-------  1 root        wheel        13920 Apr 14 14:47 dhcpd.log.0.bz2
                            17 -rw-------  1 root        wheel        14055 Apr 13 22:44 dhcpd.log.1.bz2
                            17 -rw-------  1 root        wheel        14856 Apr 13 06:49 dhcpd.log.2.bz2
                            17 -rw-------  1 root        wheel        13473 Apr 12 14:44 dhcpd.log.3.bz2
                            17 -rw-------  1 root        wheel        13258 Apr 11 22:33 dhcpd.log.4.bz2
                            17 -rw-------  1 root        wheel        13449 Apr 11 06:16 dhcpd.log.5.bz2
                            17 -rw-------  1 root        wheel        13433 Apr 10 14:03 dhcpd.log.6.bz2
                             9 -rw-r--r--  1 root        wheel        12159 Apr 13 06:38 dmesg.boot
                            33 -rw-------  1 root        wheel       149181 Apr 14 16:07 filter.log
                            33 -rw-------  1 root        wheel        29512 Apr 14 16:00 filter.log.0.bz2
                            29 -rw-------  1 root        wheel        28214 Apr 14 15:30 filter.log.1.bz2
                            33 -rw-------  1 root        wheel        29153 Apr 14 15:01 filter.log.2.bz2
                            29 -rw-------  1 root        wheel        28029 Apr 14 14:24 filter.log.3.bz2
                            33 -rw-------  1 root        wheel        29776 Apr 14 13:53 filter.log.4.bz2
                            33 -rw-------  1 root        wheel        29545 Apr 14 13:33 filter.log.5.bz2
                            33 -rw-------  1 root        wheel        28726 Apr 14 13:12 filter.log.6.bz2
                            41 -rw-------  1 root        wheel       465069 Apr 13 06:48 gateways.log
                             5 -rw-------  1 root        wheel         6059 Mar 15 01:13 gateways.log.0.bz2
                             9 -rw-------  1 root        wheel         8179 Mar 15 00:19 gateways.log.1.bz2
                             9 -rw-------  1 root        wheel         7461 Mar  4 11:51 gateways.log.2.bz2
                            13 -rw-------  1 root        wheel        10273 Mar  4 08:56 gateways.log.3.bz2
                            57 -rw-------  1 root        wheel       503298 Apr 14 16:07 haproxy.log
                            13 -rw-------  1 root        wheel        10667 Apr 13 12:23 haproxy.log.0.bz2
                            17 -rw-------  1 root        wheel        15846 Apr 13 07:18 haproxy.log.1.bz2
                            13 -rw-------  1 root        wheel        10079 Apr 12 05:10 haproxy.log.2.bz2
                            13 -rw-------  1 root        wheel        10086 Apr 11 23:49 haproxy.log.3.bz2
                            21 -rw-------  1 root        wheel        18933 Apr 11 13:55 haproxy.log.4.bz2
                            21 -rw-------  1 root        wheel        17074 Apr 10 01:19 haproxy.log.5.bz2
                            13 -rw-------  1 root        wheel        10329 Apr  9 01:13 haproxy.log.6.bz2
                             1 -rw-------  1 root        wheel            0 Apr 13 06:37 ipsec.log
                             1 -rw-------  1 root        wheel            0 Apr 13 06:37 l2tps.log
                             1 -rw-r--r--  1 root        wheel            0 Jul 11  2020 lastlog
                             1 drwx------  2 www         www              2 Jan 13  2020 lighttpd
                             1 drwxr-xr-x  2 root        wheel            3 Jul 11  2020 nginx
                            49 -rw-------  1 root        wheel       440586 Apr 14 14:37 nginx.log
                            13 -rw-------  1 root        wheel         9734 Apr 13 07:44 nginx.log.0.bz2
                            13 -rw-------  1 root        wheel        10221 Apr 13 07:11 nginx.log.1.bz2
                            13 -rw-------  1 root        wheel        11284 Apr 13 06:48 nginx.log.2.bz2
                            17 -rw-------  1 root        wheel        13970 Apr 13 05:07 nginx.log.3.bz2
                            13 -rw-------  1 root        wheel        11333 Apr  6 20:20 nginx.log.4.bz2
                            13 -rw-------  1 root        wheel         9552 Apr  4 18:50 nginx.log.5.bz2
                            13 -rw-------  1 root        wheel         9869 Apr  4 18:07 nginx.log.6.bz2
                             1 drwxr-xr-x  2 root        wheel            2 Jul 11  2020 ntp
                            17 -rw-------  1 root        wheel        83456 Apr 14 02:16 ntpd.log
                            29 -rw-------  1 root        wheel        26441 Apr  4 17:01 ntpd.log.0.bz2
                             1 drwxr-xr-x  2 uucp        uucp             2 Jun 24  2021 nut
                            17 -rw-------  1 root        wheel       127699 Apr 14 15:27 openvpn.log
                            21 -rw-------  1 root        wheel        19454 Apr  4 03:43 openvpn.log.0.bz2
                             9 -rw-------  1 root        wheel         7459 Feb 28 14:31 openvpn.log.1.bz2
                            17 -rw-------  1 root        wheel        14950 Feb 25 14:10 openvpn.log.2.bz2
                            21 -rw-------  1 root        wheel        19548 Feb 11 17:19 openvpn.log.3.bz2
                            21 -rw-------  1 root        wheel        19758 Jan 20 09:53 openvpn.log.4.bz2
                            21 -rw-------  1 root        wheel        16680 Dec 26 09:34 openvpn.log.5.bz2
                            21 -rw-------  1 root        wheel        17905 Oct  3  2021 openvpn.log.6.bz2
                             9 drwxr-xr-x  2 root        wheel           12 Apr 14 16:06 pfblockerng
                             1 -rw-------  1 root        wheel            0 Apr 13 06:37 poes.log
                            13 -rw-------  1 root        wheel        11227 Apr 13 06:37 portalauth.log
                            81 -rw-------  1 root        wheel       465398 Apr 13 06:48 ppp.log
                            25 -rw-------  1 root        wheel        22405 Mar  4 12:05 ppp.log.0.bz2
                            25 -rw-------  1 root        wheel        21618 Aug  8  2021 ppp.log.1.bz2
                             1 drwx------  4 root        wheel            4 Jun 23  2021 radacct
                             1 -rw-------  1 freeradius  freeradius       0 Jun 23  2021 radius.log
                             1 -rw-------  1 freeradius  freeradius       0 Jun 23  2021 radutmp
                             1 -rw-------  1 freeradius  freeradius       0 Jun 23  2021 radwtmp
                            13 -rw-------  1 root        wheel       511488 Jul 11  2020 relayd.log
                            69 -rw-------  1 root        wheel       460822 Apr 14 16:06 resolver.log
                            21 -rw-------  1 root        wheel        20372 Apr 13 07:50 resolver.log.0.bz2
                            25 -rw-------  1 root        wheel        20689 Apr 12 04:32 resolver.log.1.bz2
                            21 -rw-------  1 root        wheel        20227 Apr 10 18:05 resolver.log.2.bz2
                            25 -rw-------  1 root        wheel        20850 Apr  9 07:06 resolver.log.3.bz2
                            25 -rw-------  1 root        wheel        20983 Apr  7 22:32 resolver.log.4.bz2
                            25 -rw-------  1 root        wheel        21109 Apr  6 14:06 resolver.log.5.bz2
                            25 -rw-------  1 root        wheel        20578 Apr  5 08:11 resolver.log.6.bz2
                             1 -rw-------  1 root        wheel            0 Apr 13 06:37 routing.log
                             9 drwxr-xr-x  3 root        wheel            5 Feb 17  2021 snort
                             1 drwx------  2 root        wheel            3 Dec 17  2020 suricata
                            21 -rw-------  1 root        wheel       129542 Apr 14 16:07 system.log
                            41 -rw-------  1 root        wheel        40644 Apr 14 01:49 system.log.0.bz2
                            33 -rw-------  1 root        wheel        30645 Apr 11 20:35 system.log.1.bz2
                            33 -rw-------  1 root        wheel        30312 Apr  9 07:00 system.log.2.bz2
                            41 -rw-------  1 root        wheel        37309 Apr  6 22:58 system.log.3.bz2
                            41 -rw-------  1 root        wheel        39148 Apr  4 16:35 system.log.4.bz2
                            33 -rw-------  1 root        wheel        30902 Apr  2 08:49 system.log.5.bz2
                            33 -rw-------  1 root        wheel        30102 Mar 30 07:01 system.log.6.bz2
                            13 -rw-------  1 root        wheel        48962 Apr 14 14:29 userlog
                             5 -rw-r--r--  1 root        wheel          591 Apr 14 16:07 utx.lastlogin
                             5 -rw-------  1 root        wheel        11299 Apr 14 16:07 utx.log
                             1 -rw-------  1 root        wheel            0 Apr 13 06:37 vpn.log
                             1 -rw-------  1 root        wheel            0 Apr 13 06:37 wireless.log
                            
                            

                            NTP log

                            
                            Apr 13 04:55:19 pfSense ntpd[9625]: Command line: /usr/local/sbin/ntpd -g -c /var/etc/ntpd.conf -p /var/run/ntpd.pid
                            Apr 13 04:55:19 pfSense ntpd[9625]: ----------------------------------------------------
                            Apr 13 04:55:19 pfSense ntpd[9625]: ntp-4 is maintained by Network Time Foundation,
                            Apr 13 04:55:19 pfSense ntpd[9625]: Inc. (NTF), a non-profit 501(c)(3) public-benefit
                            Apr 13 04:55:19 pfSense ntpd[9625]: corporation.  Support and training for ntp-4 are
                            Apr 13 04:55:19 pfSense ntpd[9625]: available at https://www.nwtime.org/support
                            Apr 13 04:55:19 pfSense ntpd[9625]: ----------------------------------------------------
                            Apr 13 04:55:19 pfSense ntpd[9901]: proto: precision = 0.109 usec (-23)
                            Apr 13 04:55:19 pfSense ntpd[9901]: basedate set to 2021-12-31
                            Apr 13 04:55:19 pfSense ntpd[9901]: gps base set to 2022-01-02 (week 2191)
                            Apr 13 04:55:19 pfSense ntpd[9901]: Listen normally on 0 lo0 [::1]:123
                            Apr 13 04:55:19 pfSense ntpd[9901]: Listen normally on 1 lo0 [fe80::1%6]:123
                            Apr 13 04:55:19 pfSense ntpd[9901]: Listen normally on 2 lo0 127.0.0.1:123
                            Apr 13 04:55:19 pfSense ntpd[9901]: Listen normally on 3 lagg0 [fe80::225:90ff:fe26:f40%9]:123
                            Apr 13 04:55:19 pfSense ntpd[9901]: Listen normally on 4 lagg0 192.168.1.254:123
                            Apr 13 04:55:19 pfSense ntpd[9901]: Listen normally on 5 lagg0 10.10.10.1:123
                            Apr 13 04:55:19 pfSense ntpd[9901]: Listen normally on 6 lagg0.50 [fe80::225:90ff:fe26:f40%10]:123
                            Apr 13 04:55:19 pfSense ntpd[9901]: Listen normally on 7 lagg0.50 192.168.50.254:123
                            Apr 13 04:55:19 pfSense ntpd[9901]: Listen normally on 8 lagg0.60 [fe80::225:90ff:fe26:f40%11]:123
                            Apr 13 04:55:19 pfSense ntpd[9901]: Listen normally on 9 lagg0.60 192.168.60.254:123
                            Apr 13 04:55:19 pfSense ntpd[9901]: Listen normally on 10 lagg0.70 [fe80::225:90ff:fe26:f40%12]:123
                            Apr 13 04:55:19 pfSense ntpd[9901]: Listen normally on 11 lagg0.70 192.168.70.254:123
                            Apr 13 04:55:19 pfSense ntpd[9901]: Listen normally on 12 lagg0.80 [fe80::225:90ff:fe26:f40%13]:123
                            Apr 13 04:55:19 pfSense ntpd[9901]: Listen normally on 13 lagg0.80 192.168.80.254:123
                            Apr 13 04:55:19 pfSense ntpd[9901]: Listen normally on 14 lagg0.90 [fe80::225:90ff:fe26:f40%14]:123
                            Apr 13 04:55:19 pfSense ntpd[9901]: Listen normally on 15 lagg0.90 192.168.90.254:123
                            Apr 13 04:55:19 pfSense ntpd[9901]: Listen normally on 16 lagg0.100 [fe80::225:90ff:fe26:f40%15]:123
                            Apr 13 04:55:19 pfSense ntpd[9901]: Listen normally on 17 lagg0.100 192.168.100.254:123
                            Apr 13 04:55:19 pfSense ntpd[9901]: Listen normally on 18 lagg0.101 [fe80::225:90ff:fe26:f40%16]:123
                            Apr 13 04:55:19 pfSense ntpd[9901]: Listen normally on 19 lagg0.101 192.168.101.254:123
                            Apr 13 04:55:19 pfSense ntpd[9901]: Listen normally on 20 lagg0.102 [fe80::225:90ff:fe26:f40%17]:123
                            Apr 13 04:55:19 pfSense ntpd[9901]: Listen normally on 21 lagg0.102 192.168.102.254:123
                            Apr 13 04:55:19 pfSense ntpd[9901]: Listen normally on 22 lagg0.102 192.168.102.10:123
                            Apr 13 04:55:19 pfSense ntpd[9901]: Listen normally on 23 lagg0.102 192.168.102.11:123
                            Apr 13 04:55:19 pfSense ntpd[9901]: Listen normally on 24 lagg0.102 192.168.102.12:123
                            Apr 13 04:55:19 pfSense ntpd[9901]: Listen normally on 25 lagg0.104 [fe80::225:90ff:fe26:f40%18]:123
                            Apr 13 04:55:19 pfSense ntpd[9901]: Listen normally on 26 lagg0.104 192.168.104.254:123
                            Apr 13 04:55:19 pfSense ntpd[9901]: Listen normally on 27 lagg0.105 [fe80::225:90ff:fe26:f40%19]:123
                            Apr 13 04:55:19 pfSense ntpd[9901]: Listen normally on 28 lagg0.105 192.168.105.254:123
                            Apr 13 04:55:19 pfSense ntpd[9901]: Listen normally on 29 lagg0.106 [fe80::225:90ff:fe26:f40%20]:123
                            Apr 13 04:55:19 pfSense ntpd[9901]: Listen normally on 30 lagg0.106 192.168.106.254:123
                            Apr 13 04:55:19 pfSense ntpd[9901]: Listen normally on 31 lagg0.103 [fe80::225:90ff:fe26:f40%22]:123
                            Apr 13 04:55:19 pfSense ntpd[9901]: Listen normally on 32 lagg0.103 192.168.103.254:123
                            Apr 13 04:55:19 pfSense ntpd[9901]: Listen normally on 33 ovpns1 [fe80::225:90ff:fe26:f40%24]:123
                            Apr 13 04:55:19 pfSense ntpd[9901]: Listen normally on 34 ovpns1 192.168.200.1:123
                            Apr 13 04:55:19 pfSense ntpd[9901]: Listening on routing socket on fd #55 for interface updates
                            Apr 13 04:55:19 pfSense ntpd[9901]: kernel reports TIME_ERROR: 0x41: Clock Unsynchronized
                            Apr 13 04:55:19 pfSense ntpd[9901]: 0.0.0.0 c01d 0d kern kernel time sync enabled
                            Apr 13 04:55:19 pfSense ntpd[9901]: kernel reports TIME_ERROR: 0x41: Clock Unsynchronized
                            Apr 13 04:55:19 pfSense ntpd[9901]: 0.0.0.0 c012 02 freq_set kernel 36.409 PPM
                            Apr 13 04:55:19 pfSense ntpd[9901]: 0.0.0.0 c016 06 restart
                            Apr 13 04:55:20 pfSense ntpd[9901]: Soliciting pool server 81.21.65.169
                            Apr 13 04:55:21 pfSense ntpd[9901]: Soliciting pool server 85.199.214.101
                            Apr 13 04:55:21 pfSense ntpd[9901]: Soliciting pool server 185.53.93.157
                            Apr 13 04:55:22 pfSense ntpd[9901]: Soliciting pool server 129.250.35.250
                            Apr 13 04:55:22 pfSense ntpd[9901]: Soliciting pool server 162.159.200.1
                            Apr 13 04:55:22 pfSense ntpd[9901]: Soliciting pool server 134.0.16.1
                            Apr 13 04:55:23 pfSense ntpd[9901]: Soliciting pool server 81.128.218.110
                            Apr 13 04:55:23 pfSense ntpd[9901]: Soliciting pool server 217.114.59.3
                            Apr 13 04:55:23 pfSense ntpd[9901]: Soliciting pool server 2606:4700:f1::123
                            Apr 13 04:55:28 pfSense ntpd[9901]: 0.0.0.0 c615 05 clock_sync
                            Apr 13 05:07:39 pfSense ntpd[9901]: 81.21.65.169 local addr 10.10.10.1 -> <null>
                            Apr 13 05:19:22 pfSense ntpd[9901]: 217.114.59.3 local addr 10.10.10.1 -> <null>
                            Apr 13 05:19:22 pfSense ntpd[9901]: 81.128.218.110 local addr 10.10.10.1 -> <null>
                            Apr 13 05:19:22 pfSense ntpd[9901]: 134.0.16.1 local addr 10.10.10.1 -> <null>
                            Apr 13 05:19:22 pfSense ntpd[9901]: 162.159.200.1 local addr 10.10.10.1 -> <null>
                            Apr 13 05:19:22 pfSense ntpd[9901]: 185.53.93.157 local addr 10.10.10.1 -> <null>
                            Apr 13 05:19:22 pfSense ntpd[9901]: 85.199.214.101 local addr 10.10.10.1 -> <null>
                            Apr 13 05:20:13 pfSense ntpd[9901]: 0.0.0.0 0618 08 no_sys_peer
                            Apr 13 05:20:13 pfSense ntpd[9901]: 217.114.59.3 local addr 10.10.10.1 -> <null>
                            Apr 13 05:20:26 pfSense ntpd[9901]: Soliciting pool server 81.21.65.169
                            Apr 13 05:30:34 pfSense ntpd[9901]: 81.21.65.169 local addr 10.10.10.1 -> <null>
                            Apr 13 06:38:12 pfSense ntpd[17974]: ntpd 4.2.8p15@1.3728-o Wed Jan 12 15:39:52 UTC 2022 (1): Starting
                            Apr 13 06:38:12 pfSense ntpd[17974]: Command line: /usr/local/sbin/ntpd -g -q -c /dev/null 216.239.35.0 216.239.35.4 216.239.35.8 216.239.35.12
                            Apr 13 06:38:12 pfSense ntpd[17974]: ----------------------------------------------------
                            Apr 13 06:38:12 pfSense ntpd[17974]: ntp-4 is maintained by Network Time Foundation,
                            Apr 13 06:38:12 pfSense ntpd[17974]: Inc. (NTF), a non-profit 501(c)(3) public-benefit
                            Apr 13 06:38:12 pfSense ntpd[17974]: corporation.  Support and training for ntp-4 are
                            Apr 13 06:38:12 pfSense ntpd[17974]: available at https://www.nwtime.org/support
                            Apr 13 06:38:12 pfSense ntpd[17974]: ----------------------------------------------------
                            Apr 13 06:38:12 pfSense ntpd[17974]: proto: precision = 0.188 usec (-22)
                            Apr 13 06:38:12 pfSense ntpd[17974]: line 0 column 0 syntax error, unexpected $end
                            Apr 13 06:38:12 pfSense ntpd[17974]: basedate set to 2021-12-31
                            Apr 13 06:38:12 pfSense ntpd[17974]: gps base set to 2022-01-02 (week 2191)
                            Apr 13 06:38:12 pfSense ntpd[17974]: Listen and drop on 0 v6wildcard [::]:123
                            Apr 13 06:38:12 pfSense ntpd[17974]: Listen and drop on 1 v4wildcard 0.0.0.0:123
                            Apr 13 06:38:12 pfSense ntpd[17974]: Listen normally on 2 lo0 [::1]:123
                            Apr 13 06:38:12 pfSense ntpd[17974]: Listen normally on 3 lo0 [fe80::1%6]:123
                            Apr 13 06:38:12 pfSense ntpd[17974]: Listen normally on 4 lo0 127.0.0.1:123
                            Apr 13 06:38:12 pfSense ntpd[17974]: Listen normally on 5 lagg0 [fe80::225:90ff:fe26:f40%9]:123
                            Apr 13 06:38:12 pfSense ntpd[17974]: Listen normally on 6 lagg0 192.168.1.254:123
                            Apr 13 06:38:12 pfSense ntpd[17974]: Listen normally on 7 lagg0 10.10.10.1:123
                            Apr 13 06:38:12 pfSense ntpd[17974]: Listen normally on 8 lagg0.50 [fe80::225:90ff:fe26:f40%10]:123
                            Apr 13 06:38:12 pfSense ntpd[17974]: Listen normally on 9 lagg0.50 192.168.50.254:123
                            Apr 13 06:38:12 pfSense ntpd[17974]: Listen normally on 10 lagg0.60 [fe80::225:90ff:fe26:f40%11]:123
                            Apr 13 06:38:12 pfSense ntpd[17974]: Listen normally on 11 lagg0.60 192.168.60.254:123
                            Apr 13 06:38:12 pfSense ntpd[17974]: Listen normally on 12 lagg0.70 [fe80::225:90ff:fe26:f40%12]:123
                            Apr 13 06:38:12 pfSense ntpd[17974]: Listen normally on 13 lagg0.70 192.168.70.254:123
                            Apr 13 06:38:12 pfSense ntpd[17974]: Listen normally on 14 lagg0.80 [fe80::225:90ff:fe26:f40%13]:123
                            Apr 13 06:38:12 pfSense ntpd[17974]: Listen normally on 15 lagg0.80 192.168.80.254:123
                            Apr 13 06:38:12 pfSense ntpd[17974]: Listen normally on 16 lagg0.90 [fe80::225:90ff:fe26:f40%14]:123
                            Apr 13 06:38:12 pfSense ntpd[17974]: Listen normally on 17 lagg0.90 192.168.90.254:123
                            Apr 13 06:38:12 pfSense ntpd[17974]: Listen normally on 18 lagg0.100 [fe80::225:90ff:fe26:f40%15]:123
                            Apr 13 06:38:12 pfSense ntpd[17974]: Listen normally on 19 lagg0.100 192.168.100.254:123
                            Apr 13 06:38:12 pfSense ntpd[17974]: Listen normally on 20 lagg0.101 [fe80::225:90ff:fe26:f40%16]:123
                            Apr 13 06:38:12 pfSense ntpd[17974]: Listen normally on 21 lagg0.101 192.168.101.254:123
                            Apr 13 06:38:12 pfSense ntpd[17974]: Listen normally on 22 lagg0.102 [fe80::225:90ff:fe26:f40%17]:123
                            Apr 13 06:38:12 pfSense ntpd[17974]: Listen normally on 23 lagg0.102 192.168.102.254:123
                            Apr 13 06:38:12 pfSense ntpd[17974]: Listen normally on 24 lagg0.102 192.168.102.10:123
                            Apr 13 06:38:12 pfSense ntpd[17974]: Listen normally on 25 lagg0.102 192.168.102.11:123
                            Apr 13 06:38:12 pfSense ntpd[17974]: Listen normally on 26 lagg0.102 192.168.102.12:123
                            Apr 13 06:38:12 pfSense ntpd[17974]: Listen normally on 27 lagg0.104 [fe80::225:90ff:fe26:f40%18]:123
                            Apr 13 06:38:12 pfSense ntpd[17974]: Listen normally on 28 lagg0.104 192.168.104.254:123
                            Apr 13 06:38:12 pfSense ntpd[17974]: Listen normally on 29 lagg0.105 [fe80::225:90ff:fe26:f40%19]:123
                            Apr 13 06:38:12 pfSense ntpd[17974]: Listen normally on 30 lagg0.105 192.168.105.254:123
                            Apr 13 06:38:12 pfSense ntpd[17974]: Listen normally on 31 lagg0.106 [fe80::225:90ff:fe26:f40%20]:123
                            Apr 13 06:38:12 pfSense ntpd[17974]: Listen normally on 32 lagg0.106 192.168.106.254:123
                            Apr 13 06:38:12 pfSense ntpd[17974]: Listen normally on 33 lagg0.40 [fe80::225:90ff:fe26:f40%21]:123
                            Apr 13 06:38:12 pfSense ntpd[17974]: Listen normally on 34 lagg0.103 [fe80::225:90ff:fe26:f40%22]:123
                            Apr 13 06:38:12 pfSense ntpd[17974]: Listen normally on 35 lagg0.103 192.168.103.254:123
                            Apr 13 06:38:12 pfSense ntpd[17974]: Listen normally on 36 ovpns1 [fe80::225:90ff:fe26:f40%24]:123
                            Apr 13 06:38:12 pfSense ntpd[17974]: Listen normally on 37 ovpns1 192.168.200.1:123
                            Apr 13 06:38:12 pfSense ntpd[17974]: Listening on routing socket on fd #58 for interface updates
                            Apr 13 06:38:42 pfSense ntpd[17974]: ntpd exiting on signal 15 (Terminated)
                            Apr 13 06:38:42 pfSense ntpd[21121]: ntpd 4.2.8p15@1.3728-o Wed Jan 12 15:39:52 UTC 2022 (1): Starting
                            Apr 13 06:38:42 pfSense ntpd[21121]: Command line: /usr/local/sbin/ntpd -g -c /var/etc/ntpd.conf -p /var/run/ntpd.pid
                            Apr 13 06:38:42 pfSense ntpd[21121]: ----------------------------------------------------
                            Apr 13 06:38:42 pfSense ntpd[21121]: ntp-4 is maintained by Network Time Foundation,
                            Apr 13 06:38:42 pfSense ntpd[21121]: Inc. (NTF), a non-profit 501(c)(3) public-benefit
                            Apr 13 06:38:42 pfSense ntpd[21121]: corporation.  Support and training for ntp-4 are
                            Apr 13 06:38:42 pfSense ntpd[21121]: available at https://www.nwtime.org/support
                            Apr 13 06:38:42 pfSense ntpd[21121]: ----------------------------------------------------
                            Apr 13 06:38:42 pfSense ntpd[21260]: proto: precision = 0.113 usec (-23)
                            Apr 13 06:38:42 pfSense ntpd[21260]: basedate set to 2021-12-31
                            Apr 13 06:38:42 pfSense ntpd[21260]: gps base set to 2022-01-02 (week 2191)
                            Apr 13 06:38:42 pfSense ntpd[21260]: Listen normally on 0 lo0 [::1]:123
                            Apr 13 06:38:42 pfSense ntpd[21260]: Listen normally on 1 lo0 [fe80::1%6]:123
                            Apr 13 06:38:42 pfSense ntpd[21260]: Listen normally on 2 lo0 127.0.0.1:123
                            Apr 13 06:38:42 pfSense ntpd[21260]: Listen normally on 3 lagg0 [fe80::225:90ff:fe26:f40%9]:123
                            Apr 13 06:38:42 pfSense ntpd[21260]: Listen normally on 4 lagg0 192.168.1.254:123
                            Apr 13 06:38:42 pfSense ntpd[21260]: Listen normally on 5 lagg0 10.10.10.1:123
                            Apr 13 06:38:42 pfSense ntpd[21260]: Listen normally on 6 lagg0.50 [fe80::225:90ff:fe26:f40%10]:123
                            Apr 13 06:38:42 pfSense ntpd[21260]: Listen normally on 7 lagg0.50 192.168.50.254:123
                            Apr 13 06:38:42 pfSense ntpd[21260]: Listen normally on 8 lagg0.60 [fe80::225:90ff:fe26:f40%11]:123
                            Apr 13 06:38:42 pfSense ntpd[21260]: Listen normally on 9 lagg0.60 192.168.60.254:123
                            Apr 13 06:38:42 pfSense ntpd[21260]: Listen normally on 10 lagg0.70 [fe80::225:90ff:fe26:f40%12]:123
                            Apr 13 06:38:42 pfSense ntpd[21260]: Listen normally on 11 lagg0.70 192.168.70.254:123
                            Apr 13 06:38:42 pfSense ntpd[21260]: Listen normally on 12 lagg0.80 [fe80::225:90ff:fe26:f40%13]:123
                            Apr 13 06:38:42 pfSense ntpd[21260]: Listen normally on 13 lagg0.80 192.168.80.254:123
                            Apr 13 06:38:42 pfSense ntpd[21260]: Listen normally on 14 lagg0.90 [fe80::225:90ff:fe26:f40%14]:123
                            Apr 13 06:38:42 pfSense ntpd[21260]: Listen normally on 15 lagg0.90 192.168.90.254:123
                            Apr 13 06:38:42 pfSense ntpd[21260]: Listen normally on 16 lagg0.100 [fe80::225:90ff:fe26:f40%15]:123
                            Apr 13 06:38:42 pfSense ntpd[21260]: Listen normally on 17 lagg0.100 192.168.100.254:123
                            Apr 13 06:38:42 pfSense ntpd[21260]: Listen normally on 18 lagg0.101 [fe80::225:90ff:fe26:f40%16]:123
                            Apr 13 06:38:42 pfSense ntpd[21260]: Listen normally on 19 lagg0.101 192.168.101.254:123
                            Apr 13 06:38:42 pfSense ntpd[21260]: Listen normally on 20 lagg0.102 [fe80::225:90ff:fe26:f40%17]:123
                            Apr 13 06:38:42 pfSense ntpd[21260]: Listen normally on 21 lagg0.102 192.168.102.254:123
                            Apr 13 06:38:42 pfSense ntpd[21260]: Listen normally on 22 lagg0.102 192.168.102.10:123
                            Apr 13 06:38:42 pfSense ntpd[21260]: Listen normally on 23 lagg0.102 192.168.102.11:123
                            Apr 13 06:38:42 pfSense ntpd[21260]: Listen normally on 24 lagg0.102 192.168.102.12:123
                            Apr 13 06:38:42 pfSense ntpd[21260]: Listen normally on 25 lagg0.104 [fe80::225:90ff:fe26:f40%18]:123
                            Apr 13 06:38:42 pfSense ntpd[21260]: Listen normally on 26 lagg0.104 192.168.104.254:123
                            Apr 13 06:38:42 pfSense ntpd[21260]: Listen normally on 27 lagg0.105 [fe80::225:90ff:fe26:f40%19]:123
                            Apr 13 06:38:42 pfSense ntpd[21260]: Listen normally on 28 lagg0.105 192.168.105.254:123
                            Apr 13 06:38:42 pfSense ntpd[21260]: Listen normally on 29 lagg0.106 [fe80::225:90ff:fe26:f40%20]:123
                            Apr 13 06:38:42 pfSense ntpd[21260]: Listen normally on 30 lagg0.106 192.168.106.254:123
                            Apr 13 06:38:42 pfSense ntpd[21260]: Listen normally on 31 lagg0.103 [fe80::225:90ff:fe26:f40%22]:123
                            Apr 13 06:38:42 pfSense ntpd[21260]: Listen normally on 32 lagg0.103 192.168.103.254:123
                            Apr 13 06:38:42 pfSense ntpd[21260]: Listen normally on 33 ovpns1 [fe80::225:90ff:fe26:f40%24]:123
                            Apr 13 06:38:42 pfSense ntpd[21260]: Listen normally on 34 ovpns1 192.168.200.1:123
                            Apr 13 06:38:42 pfSense ntpd[21260]: Listening on routing socket on fd #55 for interface updates
                            Apr 13 06:38:42 pfSense ntpd[21260]: kernel reports TIME_ERROR: 0x41: Clock Unsynchronized
                            Apr 13 06:38:42 pfSense ntpd[21260]: 0.0.0.0 c01d 0d kern kernel time sync enabled
                            Apr 13 06:38:42 pfSense ntpd[21260]: kernel reports TIME_ERROR: 0x41: Clock Unsynchronized
                            Apr 13 06:38:42 pfSense ntpd[21260]: 0.0.0.0 c012 02 freq_set kernel 36.409 PPM
                            Apr 13 06:38:42 pfSense ntpd[21260]: 0.0.0.0 c016 06 restart
                            Apr 13 06:38:43 pfSense ntpd[21260]: error resolving pool 0.pfsense.pool.ntp.org: Name does not resolve (8)
                            Apr 13 06:38:50 pfSense ntpd[21260]: error resolving pool 0.uk.pool.ntp.org: Name does not resolve (8)
                            Apr 13 06:38:50 pfSense ntpd[21260]: error resolving pool 1.uk.pool.ntp.org: Name does not resolve (8)
                            Apr 13 06:38:50 pfSense ntpd[21260]: error resolving pool 2.uk.pool.ntp.org: Name does not resolve (8)
                            Apr 13 06:39:47 pfSense ntpd[21260]: error resolving pool 0.pfsense.pool.ntp.org: Name does not resolve (8)
                            Apr 13 06:39:49 pfSense ntpd[21260]: error resolving pool 0.uk.pool.ntp.org: Name does not resolve (8)
                            Apr 13 06:39:51 pfSense ntpd[21260]: error resolving pool 1.uk.pool.ntp.org: Name does not resolve (8)
                            Apr 13 06:39:52 pfSense ntpd[21260]: error resolving pool 2.uk.pool.ntp.org: Name does not resolve (8)
                            Apr 13 06:40:52 pfSense ntpd[21260]: error resolving pool 0.pfsense.pool.ntp.org: Name does not resolve (8)
                            Apr 13 06:40:55 pfSense ntpd[21260]: error resolving pool 1.uk.pool.ntp.org: Name does not resolve (8)
                            Apr 13 06:40:56 pfSense ntpd[21260]: error resolving pool 0.uk.pool.ntp.org: Name does not resolve (8)
                            Apr 13 06:40:59 pfSense ntpd[21260]: error resolving pool 2.uk.pool.ntp.org: Name does not resolve (8)
                            Apr 13 06:41:56 pfSense ntpd[21260]: error resolving pool 0.pfsense.pool.ntp.org: Name does not resolve (8)
                            Apr 13 06:42:00 pfSense ntpd[21260]: error resolving pool 1.uk.pool.ntp.org: Name does not resolve (8)
                            Apr 13 06:42:00 pfSense ntpd[21260]: error resolving pool 0.uk.pool.ntp.org: Name does not resolve (8)
                            Apr 13 06:42:06 pfSense ntpd[21260]: error resolving pool 2.uk.pool.ntp.org: Name does not resolve (8)
                            Apr 13 06:43:02 pfSense ntpd[21260]: error resolving pool 0.pfsense.pool.ntp.org: Name does not resolve (8)
                            Apr 13 06:43:05 pfSense ntpd[21260]: error resolving pool 1.uk.pool.ntp.org: Name does not resolve (8)
                            Apr 13 06:43:06 pfSense ntpd[21260]: error resolving pool 0.uk.pool.ntp.org: Name does not resolve (8)
                            Apr 13 06:43:13 pfSense ntpd[21260]: error resolving pool 2.uk.pool.ntp.org: Name does not resolve (8)
                            Apr 13 06:44:08 pfSense ntpd[21260]: error resolving pool 0.pfsense.pool.ntp.org: Name does not resolve (8)
                            Apr 13 06:44:10 pfSense ntpd[21260]: error resolving pool 1.uk.pool.ntp.org: Name does not resolve (8)
                            Apr 13 06:44:10 pfSense ntpd[21260]: error resolving pool 0.uk.pool.ntp.org: Name does not resolve (8)
                            Apr 13 06:44:18 pfSense ntpd[21260]: error resolving pool 2.uk.pool.ntp.org: Name does not resolve (8)
                            Apr 13 06:45:13 pfSense ntpd[21260]: error resolving pool 0.pfsense.pool.ntp.org: Name does not resolve (8)
                            Apr 13 06:45:14 pfSense ntpd[21260]: error resolving pool 0.uk.pool.ntp.org: Name does not resolve (8)
                            Apr 13 06:45:17 pfSense ntpd[21260]: error resolving pool 1.uk.pool.ntp.org: Name does not resolve (8)
                            Apr 13 06:45:23 pfSense ntpd[21260]: error resolving pool 2.uk.pool.ntp.org: Name does not resolve (8)
                            Apr 13 06:46:20 pfSense ntpd[21260]: error resolving pool 0.uk.pool.ntp.org: Name does not resolve (8)
                            Apr 13 06:46:20 pfSense ntpd[21260]: error resolving pool 0.pfsense.pool.ntp.org: Name does not resolve (8)
                            Apr 13 06:46:23 pfSense ntpd[21260]: error resolving pool 1.uk.pool.ntp.org: Name does not resolve (8)
                            Apr 13 06:46:28 pfSense ntpd[21260]: error resolving pool 2.uk.pool.ntp.org: Name does not resolve (8)
                            Apr 13 06:47:25 pfSense ntpd[21260]: error resolving pool 0.uk.pool.ntp.org: Name does not resolve (8)
                            Apr 13 06:47:25 pfSense ntpd[21260]: error resolving pool 0.pfsense.pool.ntp.org: Name does not resolve (8)
                            Apr 13 06:47:30 pfSense ntpd[21260]: error resolving pool 1.uk.pool.ntp.org: Name does not resolve (8)
                            Apr 13 06:47:35 pfSense ntpd[21260]: error resolving pool 2.uk.pool.ntp.org: Name does not resolve (8)
                            Apr 13 06:48:30 pfSense ntpd[21260]: Soliciting pool server 81.187.240.205
                            Apr 13 06:48:30 pfSense ntpd[21260]: Soliciting pool server 139.162.219.252
                            Apr 13 06:48:31 pfSense ntpd[21260]: Soliciting pool server 193.47.147.20
                            Apr 13 06:48:32 pfSense ntpd[21260]: Soliciting pool server 83.151.207.133
                            Apr 13 06:48:32 pfSense ntpd[21260]: Soliciting pool server 162.159.200.123
                            Apr 13 06:48:33 pfSense ntpd[21260]: Soliciting pool server 85.199.214.100
                            Apr 13 06:48:34 pfSense ntpd[21260]: Soliciting pool server 80.82.244.120
                            Apr 13 06:48:35 pfSense ntpd[21260]: Soliciting pool server 80.93.163.202
                            Apr 13 06:48:42 pfSense ntpd[21260]: 0.0.0.0 061c 0c clock_step -0.494278 s
                            Apr 13 06:48:42 pfSense ntpd[21260]: 0.0.0.0 0615 05 clock_sync
                            Apr 13 06:48:42 pfSense ntpd[21260]: receive: Unexpected origin timestamp 0xe600e2ba.e27741ad does not match aorg 0000000000.00000000 from server@193.47.147.20 xmt 0xe600e2ba.67a7e2eb
                            Apr 13 06:48:43 pfSense ntpd[21260]: 0.0.0.0 c618 08 no_sys_peer
                            Apr 13 06:58:19 pfSense ntpd[21260]: 80.82.244.120 local addr 10.10.10.1 -> <null>
                            Apr 13 07:13:25 pfSense ntpd[21260]: 81.187.240.205 local addr 10.10.10.1 -> <null>
                            Apr 13 07:14:54 pfSense ntpd[21260]: 80.93.163.202 local addr 10.10.10.1 -> <null>
                            Apr 13 07:31:36 pfSense ntpd[21260]: 139.162.219.252 local addr 10.10.10.1 -> <null>
                            Apr 14 02:04:13 pfSense ntpd[21260]: 162.159.200.123 local addr 10.10.10.1 -> <null>
                            Apr 14 02:04:24 pfSense ntpd[21260]: 193.47.147.20 local addr 10.10.10.1 -> <null>
                            Apr 14 02:04:57 pfSense ntpd[21260]: Soliciting pool server 46.227.207.91
                            Apr 14 02:04:58 pfSense ntpd[21260]: Soliciting pool server 85.199.214.98
                            Apr 14 02:04:59 pfSense ntpd[21260]: Soliciting pool server 5.101.146.245
                            Apr 14 02:05:00 pfSense ntpd[21260]: Soliciting pool server 85.199.214.102
                            Apr 14 02:05:01 pfSense ntpd[21260]: Soliciting pool server 178.79.155.116
                            Apr 14 02:05:02 pfSense ntpd[21260]: Soliciting pool server 90.255.244.219
                            Apr 14 02:15:08 pfSense ntpd[21260]: 90.255.244.219 local addr 10.10.10.1 -> <null>
                            Apr 14 02:15:09 pfSense ntpd[21260]: 5.101.146.245 local addr 10.10.10.1 -> <null>
                            Apr 14 02:15:16 pfSense ntpd[21260]: 178.79.155.116 local addr 10.10.10.1 -> <null>
                            Apr 14 02:16:14 pfSense ntpd[21260]: 46.227.207.91 local addr 10.10.10.1 -> <null>
                            
                            GertjanG 1 Reply Last reply Reply Quote 0
                            • stephenw10S
                              stephenw10 Netgate Administrator
                              last edited by stephenw10

                              Hmm, well the only log it could have been us the filter log. That is now overwritten so we can't see for sure it was rotating every minute during the issue.
                              So potentially you were under some sort of attack? The monitoring graphs should show a massively increased rate of blocks on WAN at that time if it was.

                              Nothing outstanding in the ntp logs other than a failure to resole the pool IPs for 10mins yesterday. That would not be causing an issue now.
                              Do you see anything from : ntpq -pn

                              VioletDragonV 1 Reply Last reply Reply Quote 0
                              • VioletDragonV
                                VioletDragon @stephenw10
                                last edited by VioletDragon

                                @stephenw10 So it looks like to be working, i am guessing it is something to do with the Gui that could be broken,

                                     remote           refid      st t when poll reach   delay   offset  jitter
                                ==============================================================================
                                 0.pfsense.pool. .POOL.          16 p    -   64    0    0.000   +0.000   0.000
                                 0.uk.pool.ntp.o .POOL.          16 p    -   64    0    0.000   +0.000   0.000
                                 1.uk.pool.ntp.o .POOL.          16 p    -   64    0    0.000   +0.000   0.000
                                 2.uk.pool.ntp.o .POOL.          16 p    -   64    0    0.000   +0.000   0.000
                                #185.132.43.60   85.199.214.98    2 u   24   64  377   21.345   +0.133   0.158
                                +81.128.218.110  .GPS.            1 u    9   64  377   22.794   +0.182   0.233
                                *85.199.214.101  .GPS.            1 u   15   64  377   18.637   -0.185   0.253
                                -195.171.43.12   .PPS.            1 u   18   64  377   23.762   +0.416   0.458
                                -185.83.169.27   .GPS.            1 u   17   64  377   25.767   +0.702   0.461
                                +85.199.214.222  85.199.214.99    2 u   19   64  377   18.945   -0.178   0.356
                                #162.159.200.1   10.20.14.167     3 u   84   64    7   22.647   +0.184   0.149
                                
                                

                                I don't believe the UK NTP Servers are working

                                1 Reply Last reply Reply Quote 0
                                • stephenw10S
                                  stephenw10 Netgate Administrator
                                  last edited by

                                  I would think they are. You always see an entry for each pool you have configured like that.

                                  185.83.169.27 is in 2.uk.pool.ntp.org

                                  [22.05-DEVELOPMENT][admin@plusdev-2.stevew.lan]/root: host 2.uk.pool.ntp.org
                                  2.uk.pool.ntp.org has address 185.103.117.60
                                  2.uk.pool.ntp.org has address 185.83.169.27
                                  2.uk.pool.ntp.org has address 85.199.214.101
                                  2.uk.pool.ntp.org has address 103.214.44.30
                                  2.uk.pool.ntp.org has IPv6 address 2001:8b0:df52:914d::123
                                  2.uk.pool.ntp.org has IPv6 address 2a00:da00:1800:7f::1
                                  2.uk.pool.ntp.org has IPv6 address 2a00:2381:19c6::100
                                  2.uk.pool.ntp.org has IPv6 address 2a0b:9b00:463::123
                                  

                                  Steve

                                  VioletDragonV 1 Reply Last reply Reply Quote 0
                                  • VioletDragonV
                                    VioletDragon @stephenw10
                                    last edited by

                                    @stephenw10 urmm interesting. when i do host host 2.uk.pool.ntp.org it shows different here,

                                    2.uk.pool.ntp.org has address 103.214.44.30
                                    2.uk.pool.ntp.org has address 45.63.100.187
                                    2.uk.pool.ntp.org has address 139.143.5.30
                                    2.uk.pool.ntp.org has address 134.0.16.1
                                    2.uk.pool.ntp.org has IPv6 address 2606:4700:f1::1
                                    2.uk.pool.ntp.org has IPv6 address 2a0b:9b00:463::123
                                    2.uk.pool.ntp.org has IPv6 address 2a03:b980:123:2::a
                                    2.uk.pool.ntp.org has IPv6 address 2a01:7e00::f03c:91ff:fe73:fd27
                                    
                                    1 Reply Last reply Reply Quote 0
                                    • stephenw10S
                                      stephenw10 Netgate Administrator
                                      last edited by

                                      Probably more things in the pool. I've never really looked into it that deeply. Local caching involved in the resolving.

                                      Steve

                                      VioletDragonV 1 Reply Last reply Reply Quote 0
                                      • VioletDragonV
                                        VioletDragon @stephenw10
                                        last edited by VioletDragon

                                        @stephenw10 Hi i'm back, the problem has started again Web Gui is not working and comes up with ERR_CONNECTION_CLOSED, top is not showing much any ideas? I believe either php or nginx is crashing and not restarting.

                                        Screenshot from 2022-04-17 00-48-36.png

                                        VioletDragonV 1 Reply Last reply Reply Quote 0
                                        • VioletDragonV
                                          VioletDragon @VioletDragon
                                          last edited by

                                          @violetdragon I have noticed that PHP is using some RAM and some CPU when the Gui works.

                                          23165 www           1  20    0    27M    16M kqread   1   0:59   0.11% haproxy
                                          71979 root          1  20    0    13M  3644K CPU1     1   0:00   0.10% top
                                          85706 root          1  27    0    60M    43M nanslp   1   0:31   0.09% php
                                          23709 root          1  20    0    30M  9612K kqread   0   0:00   0.06% nginx
                                          62037 root          3  41   20   538M   471M bpf      2   0:07   0.03% snort
                                          68274 root          1  20    0    20M  9396K select   3   0:00   0.02% sshd
                                          54051 dhcpd         1  20    0    25M    13M select   1   0:29   0.02% dhcpd
                                          39229 root          5  52    0    11M  2644K uwait    0   0:36   0.01% dpinger
                                          29141 root          1  20    0    21M  8408K select   1   0:04   0.01% mpd5
                                          65272 root          1  20    0    11M  2200K select   3   0:53   0.01% powerd
                                          95019 avahi         1  20    0    12M  3568K select   1   0:34   0.01% avahi-daemon
                                          38649 root          5  52    0    11M  2644K uwait    1   0:42   0.01% dpinger
                                           2502 root          1  20    0    11M  2700K select   2   0:29   0.01% syslogd
                                          31450 root          1  20    0    19M  7216K select   1   0:14   0.01% ntpd
                                          77306 root          1  20    0    11M  2212K kqread   0   0:13   0.01% tail_pfb
                                          85029 root          1  20    0    17M  7900K kqread   2   1:35   0.00% lighttpd_pfb
                                            372 root          1  20    0   101M    27M kqread   1   0:07   0.00% php-fpm
                                          60982 root          1  20    0    12M  3024K bpf      3   0:48   0.00% filterlog
                                          77510 root          1  21    0    77M    59M piperd   0   4:36   0.00% php_pfb
                                           5480 root          2  20    0    19M  7788K select   2   1:54   0.00% openvpn
                                          12965 uucp          1  20    0    12M  2864K select   1   0:37   0.00% usbhid-ups
                                          57324 root          1  52    0   134M    52M accept   3   0:19   0.00% php-fpm
                                          58571 root          1  52    0   134M    52M accept   1   0:19   0.00% php-fpm
                                          47138 root          1  20    0   132M    50M piperd   2   0:17   0.00% php-fpm
                                          85596 root          1  52    0   132M    50M accept   0   0:16   0.00% php-fpm
                                           8628 root          1  52    0   134M    51M accept   0   0:15   0.00% php-fpm
                                          
                                          VioletDragonV 1 Reply Last reply Reply Quote 0
                                          • VioletDragonV
                                            VioletDragon @VioletDragon
                                            last edited by VioletDragon

                                            @violetdragon Just thought i'd mention this, after having another look it's something on the home page that is causing the issue, when the home page does not load if i go to any of the tab it loads them but not the home page.

                                            (Edit)

                                            The problem is the Disks Widget, when the Disk widget is on the home page the problem appears home page does not load when removed off the home page problem disappears

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.