Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Syslog service in pfSense v2.8.1 often stop itself

    Scheduled Pinned Locked Moved General pfSense Questions
    59 Posts 16 Posters 8.7k Views 14 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • S Offline
      slu @sokeada
      last edited by

      @sokeada said in Syslog service in pfSense v2.8.1 often stop itself:

      This is make sense, I enabled remote logging as well.

      That explain also why my other boxes without remote logging doesn't hit this issue.

      pfSense Gold subscription

      sokeadaS 1 Reply Last reply Reply Quote 0
      • sokeadaS Offline
        sokeada @slu
        last edited by

        @slu said in Syslog service in pfSense v2.8.1 often stop itself:

        @sokeada said in Syslog service in pfSense v2.8.1 often stop itself:

        This is make sense, I enabled remote logging as well.

        That explain also why my other boxes without remote logging doesn't hit this issue.

        yeah.

        1 Reply Last reply Reply Quote 0
        • M Offline
          myke30
          last edited by

          Hello Everyone,
          I have exactly the same issue only with pfsense 2.8.1 ( 2.8.0 is okay) with the same logs :

          "sendto: Connection refused " on the syslogd process and the service stopped after that ( i think)

          The remote logging is enabled.

          I hope we'll be a patch :-)

          Best Regards.

          1 Reply Last reply Reply Quote 0
          • w0wW Away
            w0w
            last edited by w0w

            https://redmine.pfsense.org/issues/16362

            sendto: Connection refused
            

            Do you have any idea why it can be refused? I see this only when booting and network not initiated yet or whatever, never happened when everything is up and running.

            1 Reply Last reply Reply Quote 0
            • stephenw10S Offline
              stephenw10 Netgate Administrator
              last edited by

              Typically it seems to be when the syslog server is cycling in some way. Archiving perhaps? Seems like an issue for a syslog server to periodically stop accepting logs! But enough people are seeing it that I guess it's a thing.
              This looks to be fixed by changes that went into syslogd recently. I can't replicate it in 25.11.

              T S 2 Replies Last reply Reply Quote 0
              • stephenw10S stephenw10 referenced this topic on
              • T Offline
                tsmalmbe @stephenw10
                last edited by

                @stephenw10 It is not an "issue" for syslogservers to stop accepting logs for the duration of a restart. It is a common fact that this may happen, all syslog receivers are not (or cannot be) clusters. And while this is all UDP, i am (still) a bit baffled why the pfSense syslog would actually even care - it should just spit UDP in that direction and let the receiver worry about ...well, the receiving part.

                Security Consultant at Mint Security Ltd - www.mintsecurity.fi

                1 Reply Last reply Reply Quote 0
                • S Offline
                  slu @stephenw10
                  last edited by

                  @stephenw10 said in Syslog service in pfSense v2.8.1 often stop itself:

                  Typically it seems to be when the syslog server is cycling in some way. Archiving perhaps?

                  In our case we saw this multiple times after reboot and pfSense is ready before all the KVM guests are online for example the syslog server...

                  Not sure this also happens in 2.8.0, maybe possible.

                  pfSense Gold subscription

                  stephenw10S 1 Reply Last reply Reply Quote 0
                  • stephenw10S Offline
                    stephenw10 Netgate Administrator
                    last edited by

                    Oh it absolutely shouldn't. It's a bug and it's now fixed. https://reviews.freebsd.org/D51995

                    1 Reply Last reply Reply Quote 3
                    • stephenw10S Offline
                      stephenw10 Netgate Administrator @slu
                      last edited by

                      @slu said in Syslog service in pfSense v2.8.1 often stop itself:

                      Not sure this also happens in 2.8.0, maybe possible.

                      Yes, it's in 2.8.0 and 2.8.1 and 25.07. Unfortunately.

                      S T 2 Replies Last reply Reply Quote 0
                      • S Offline
                        slu @stephenw10
                        last edited by

                        @stephenw10
                        mhm, I try the "Service Watchdog" at the moment, maybe a workaround?
                        Will see..

                        pfSense Gold subscription

                        S 1 Reply Last reply Reply Quote 0
                        • T Offline
                          tsmalmbe
                          last edited by

                          ChatGPT overview of the change, seems legit (of course caveat emptor for LLM...)

                          After D51995, all of the still-fatal cases are local to the pfSense box (syslogd/process/kernel socket). None of the remaining fatal errors are caused by the remote syslog host; the remote/network-state errors were reclassified as transient and no longer make the destination “dead.”

                          What can still make syslogd drop the destination (and why)

                          Local to the pfSense box (syslogd / socket / config):

                          • EBADF – invalid/closed descriptor used for sendmsg(). Programming/state issue on the sender.
                            man.freebsd.org

                          • EACCES – permission denied (e.g., trying to send to a broadcast address without SO_BROADCAST, or lacking permission on a UNIX-domain socket path). Sender-side socket option or filesystem perms.

                          • ENOTSOCK – fd is not a socket. Sender bug/misconfiguration.

                          • EFAULT – bad user-space buffer/pointer given to sendmsg(). Sender bug.

                          • EMSGSIZE – message too large for the socket/protocol to send atomically (e.g., oversize UDP/UNIX-dgram). Sender data/MTU limits at the local stack boundary—not the remote host.

                          • Any other unexpected errno not on the new whitelist (e.g., EINVAL, EAFNOSUPPORT, EDESTADDRREQ, ENOTCONN)—all indicate a local misuse/state problem.

                          Dependent on the remote syslog host or wider network?

                          None of the still-fatal ones. Host/network conditions like refused connection, no route, host down/unreachable, address not available, buffer pressure, or EAGAIN were explicitly moved to the “transient, keep retrying” bucket and no longer cause F_UNUSED.

                          Security Consultant at Mint Security Ltd - www.mintsecurity.fi

                          1 Reply Last reply Reply Quote 0
                          • T Offline
                            tsmalmbe @stephenw10
                            last edited by

                            @stephenw10 said in Syslog service in pfSense v2.8.1 often stop itself:

                            @slu said in Syslog service in pfSense v2.8.1 often stop itself:

                            Not sure this also happens in 2.8.0, maybe possible.

                            Yes, it's in 2.8.0 and 2.8.1 and 25.07. Unfortunately.

                            I was going to look into this and maybe a cronjob "just in case every 3 hours". Let us know the results of your investigation!

                            Security Consultant at Mint Security Ltd - www.mintsecurity.fi

                            1 Reply Last reply Reply Quote 0
                            • stephenw10S Offline
                              stephenw10 Netgate Administrator
                              last edited by

                              That patch is in the new 25.11-dev snapshots if you're able to test that. No CE snaps yet.

                              J 1 Reply Last reply Reply Quote 1
                              • dennypageD Offline
                                dennypage
                                last edited by

                                This issue may warrant a 25.07.2 release.

                                1 Reply Last reply Reply Quote 2
                                • J Offline
                                  jrey @stephenw10
                                  last edited by

                                  @stephenw10

                                  Depends - what else is in the snapshot ?

                                  Not really interested in alpha development builds this early in the cycle just to try the one failing item. The risk of the unknown outweighs the current workaround.

                                  -- honestly, too bad one can not just get the updated build of the syslogd code.

                                  KOMK 1 Reply Last reply Reply Quote 0
                                  • KOMK Offline
                                    KOM @jrey
                                    last edited by

                                    @jrey Why would you need a whole new snapshot for something that could be fixed via System Patches? And, for the record, I also had this issue because I had an old remote syslog defined that no longer exists. Removing the remote fixed the crashing.

                                    dennypageD 1 Reply Last reply Reply Quote 0
                                    • dennypageD Offline
                                      dennypage @KOM
                                      last edited by

                                      @KOM said in Syslog service in pfSense v2.8.1 often stop itself:

                                      Why would you need a whole new snapshot for something that could be fixed via System Patches?

                                      Binaries cannot be updated via System Patches.

                                      KOMK 1 Reply Last reply Reply Quote 2
                                      • stephenw10S Offline
                                        stephenw10 Netgate Administrator
                                        last edited by

                                        That's a compile time fix, it can't be applied via System Patches.

                                        J 1 Reply Last reply Reply Quote 0
                                        • J Offline
                                          jrey @stephenw10
                                          last edited by

                                          @stephenw10

                                          wasn't suggesting a "patch" in the current "pf"sense of a patch

                                          On the other hand it is just a binary file, that could be provided and copied into place.

                                          For those not willing to play with "Alpha" builds the release of 25.07.2 would be a great alternative, rather then having to wait for beta or even final release of 25.11

                                          Maybe the "patch system" should have the ability to deliver a hot fix for certain binaries in the future?
                                          somewhat surprising that since BSD at the core a patch could be created with bsdiff and bspatch both of which are available and actually installed as part of the package.

                                          Annoying issues (and not that there are that many, this is one) could likely be fixed by providing this ability to either install a new file or run a binary patch, without having to wait for a full drop of the next version.
                                          (the problem with patching vs copying, is as we have seen in that past, the files where different in the same "release" from one day to the next. Refresh my memory when was that 23.xx, I'd have to look it up)

                                          point is when there is a will there is a way..

                                          S 1 Reply Last reply Reply Quote 0
                                          • stephenw10S Offline
                                            stephenw10 Netgate Administrator
                                            last edited by

                                            Yes we are looking at options.

                                            1 Reply Last reply Reply Quote 4
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.