Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    System Logs Format (rsyslog)

    General pfSense Questions
    4
    17
    1.6k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • arrmoA
      arrmo
      last edited by

      Hi,

      I'm trying to get pfSense to send my system logs in RSYSLOG_SyslogProtocol23Format (specific format handled well by system log parsing software).

      Is there a known / easy way to do this?

      Thanks!

      1 Reply Last reply Reply Quote 0
      • stephenw10S
        stephenw10 Netgate Administrator
        last edited by

        There is no way of doing that I'm aware of. Certainly not within normal pfSense config.

        Do you need to do that on pfSense itself? Or can you export the logs via syslog to, maybe, rsyslog on something else and convert it there?

        Steve

        1 Reply Last reply Reply Quote 0
        • arrmoA
          arrmo
          last edited by

          That makes sense. I did a bit of digging, and in BSD v12 syslog does support the official (RFC 5424 format). But not in v11.x.

          Not sure I understand your rsyslog comment - can you clarify? rsyslog doesn't exist on pfSense, does it?

          Thanks!

          1 Reply Last reply Reply Quote 0
          • stephenw10S
            stephenw10 Netgate Administrator
            last edited by

            Indeed it doesn't. I was suggesting exporting it to something else and converting there before sending it to the log analyser.

            I've never tried that myself.

            Steve

            1 Reply Last reply Reply Quote 0
            • arrmoA
              arrmo
              last edited by

              Ahh, OK - NP. Thanks for the idea! Will dig more.

              To be honest, if I go to v2.5 of pfSense the issue goes away (i.e. RFC5424 support is there, directly in syslog). Just not sure how stable v2.5 is.

              Thanks again.

              stephenw10S 1 Reply Last reply Reply Quote 0
              • stephenw10S
                stephenw10 Netgate Administrator @arrmo
                last edited by

                It's quite stable on x86-64. I've been running it for months on numerous boxes with no issues. Is is still in dev though so the normal precautions apply etc... 😉

                Steve

                1 Reply Last reply Reply Quote 1
                • arrmoA
                  arrmo
                  last edited by

                  No worries, understand the caveats, legal-ize, etc. ... LOL.

                  Thanks! I may go this way - then of course need to see if I can tweak the output format (i.e. need to modify the syslogd options a bit, to output the needed format).

                  Thanks again.

                  1 Reply Last reply Reply Quote 0
                  • arrmoA
                    arrmo
                    last edited by

                    OK, shifted to v2.5, seem to have the new and improved version of syslogd ... :-). Meaning, the -O format option exists. Perfect!

                    Now, how to modify the execution script to have this added to the command? I just need to find that.

                    1 Reply Last reply Reply Quote 0
                    • arrmoA
                      arrmo
                      last edited by

                      Checked the output, working great now - thanks for all the help!

                      Need to figure out the next step - would be nice to have this as a (GUI) option ... it's pretty simple. Just need to figure out how / where to suggest it.

                      Thanks again.

                      1 Reply Last reply Reply Quote 0
                      • kiokomanK
                        kiokoman LAYER 8
                        last edited by

                        i still see syslogd on my 2.5.0
                        you can place additional configuration files in /var/etc/syslog.d
                        best place to ask for new features is https://redmine.pfsense.org/

                        ̿' ̿'\̵͇̿̿\з=(◕_◕)=ε/̵͇̿̿/'̿'̿ ̿
                        Please do not use chat/PM to ask for help
                        we must focus on silencing this @guest character. we must make up lies and alter the copyrights !
                        Don't forget to Upvote with the 👍 button for any post you find to be helpful.

                        1 Reply Last reply Reply Quote 0
                        • stephenw10S
                          stephenw10 Netgate Administrator
                          last edited by

                          Yup or pull-requests directly in github: https://github.com/pfsense

                          Steve

                          1 Reply Last reply Reply Quote 0
                          • jimpJ
                            jimp Rebel Alliance Developer Netgate
                            last edited by

                            I made an issue for it here: https://redmine.pfsense.org/issues/9808

                            Should be simple enough to code, I'll get to it before long, assuming someone doesn't send in a PR first.

                            Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                            Need help fast? Netgate Global Support!

                            Do not Chat/PM for help!

                            1 Reply Last reply Reply Quote 2
                            • arrmoA
                              arrmo
                              last edited by

                              Thanks! I was going to do that - just hadn't had a chance to yet.

                              1 Reply Last reply Reply Quote 0
                              • kiokomanK
                                kiokoman LAYER 8
                                last edited by kiokoman

                                yes ... it was easy to add the gui fuction

                                Immagine.jpg

                                the problem is that if i set rfc5424
                                remote syslog still work

                                [2.5.0-DEVELOPMENT][root@pfSense.localdomain]/usr/local/www: ps aux | grep syslogd
root  76833   0.0  0.1  11376   2836  -  Ss   23:18       0:00.03 /usr/sbin/syslogd -s -c -c -l /var/dhcpd/var/run/log -P /var/run/syslog.pid -f /etc/syslog.conf -O rfc5424
root  62262   0.0  0.1  11144   2636  0  S+   23:19       0:00.00 grep syslogd
[2.5.0-DEVELOPMENT][root@pfSense.localdomain]/usr/local/www: ps aux | grep syslogd
root  74853   0.0  0.1  11376   2836  -  Ss   23:20       0:00.07 /usr/sbin/syslogd -s -c -c -l /var/dhcpd/var/run/log -P /var/run/syslog.pid -f /etc/syslog.conf -O rfc3164
root   3527   0.0  0.1  11144   2636  0  S+   23:30       0:00.00 grep syslogd

                                this are just example
                                on my rsyslog server, there is only the hostname instead of the ip, it is able to filter the incoming log

                                Oct  3 23:19:39 pfSense.localdomain radvd[28029] resuming normal operation
Oct  3 23:19:55 pfSense.localdomain radvd[28029] IPv6 forwarding on interface seems to be disabled, but continuing anyway
Oct  3 23:19:55 pfSense.localdomain radvd[28029] message repeated 2 times: [IPv6 forwarding on interface seems to be disabled, but continuing anyway]
Oct  3 23:20:09 172.17.0.254 radvd[28029]: attempting to reread config file
Oct  3 23:20:09 172.17.0.254 radvd[28029]: IPv6 forwarding on interface seems to be disabled, but continuing anyway
Oct  3 23:20:09 172.17.0.254 radvd[28029]: message repeated 5 times: [ IPv6 forwarding on interface seems to be disabled, but continuing anyway]

                                anyway this is what is written inside pfsense

                                <190>1 2019-10-03T23:19:39.586931+02:00 pfSense.localdomain dhcpd 57488 - - Listening on Socket/6/ix0/2001:470:26:5dc::/64
<190>1 2019-10-03T23:19:39.586942+02:00 pfSense.localdomain dhcpd 57488 - - Sending on   Socket/6/ix0/2001:470:26:5dc::/64
<190>1 2019-10-03T23:19:39.586942+02:00 pfSense.localdomain dhcpd 57488 - - Sending on   Socket/6/ix0/2001:470:26:5dc::/64
<190>1 2019-10-03T23:19:39.587172+02:00 pfSense.localdomain dhcpd 57488 - - Server starting service.
<190>1 2019-10-03T23:19:39.587172+02:00 pfSense.localdomain dhcpd 57488 - - Server starting service.
Oct  3 23:20:08 pfSense dhcpd[85579]: Internet Systems Consortium DHCP Server 4.4.1
Oct  3 23:20:08 pfSense dhcpd[85579]: Internet Systems Consortium DHCP Server 4.4.1
Oct  3 23:20:08 pfSense dhcpd[85579]: Copyright 2004-2018 Internet Systems Consortium.
Oct  3 23:20:08 pfSense dhcpd[85579]: Copyright 2004-2018 Internet Systems Consortium.
Oct  3 23:20:08 pfSense dhcpd[85579]: All rights reserved.
Oct  3 23:20:08 pfSense dhcpd[85579]: All rights reserved.

                                but from the gui i'm unable to see any log (i see only rfc3164) , i think that log filters also need to be adjusted based on rfc selected. and ... well ... that it's not easy for me 😂

                                ̿' ̿'\̵͇̿̿\з=(◕_◕)=ε/̵͇̿̿/'̿'̿ ̿
                                Please do not use chat/PM to ask for help
                                we must focus on silencing this @guest character. we must make up lies and alter the copyrights !
                                Don't forget to Upvote with the 👍 button for any post you find to be helpful.

                                1 Reply Last reply Reply Quote 0
                                • jimpJ
                                  jimp Rebel Alliance Developer Netgate
                                  last edited by

                                  Finally had time to pivot back to this one. I pushed a fix that seems to do a decent job of parsing dynamically, even when the logs contain a mix of entries in different formats.

                                  It should show up in snapshots soon.

                                  https://redmine.pfsense.org/issues/9808
                                  https://github.com/pfsense/pfsense/commit/b16c3a12c61c117e9c8140b115efc7f9acea96c5

                                  Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                                  Need help fast? Netgate Global Support!

                                  Do not Chat/PM for help!

                                  arrmoA 2 Replies Last reply Reply Quote 4
                                  • arrmoA
                                    arrmo @jimp
                                    last edited by

                                    @jimp Awesome, thanks! Will give it a try once it's in a snapshot. Much appreciated!

                                    1 Reply Last reply Reply Quote 0
                                    • arrmoA
                                      arrmo @jimp
                                      last edited by

                                      @jimp Seems to be working - thanks so much! Will keep an eye on it, let you know if I come across any issues. Much appreciated!

                                      1 Reply Last reply Reply Quote 0
                                      • First post
                                        Last post