Remote logging is intermittent (SOLVED - Unrelated to pfSense)

  • I'm using a remote logging server to monitor my logs for my servers. The logging is working over all but dozens of times a day it will stop receiving logs from pfSense and there is a gap of nothing that last for a few minutes up to hours looking at the logs I'm now noticing a pattern. At 48 Minutes past the hour my logging stops. Then at 17 minutes past the hour the logging starts again. It's been that way for the past several hours. I'm Trying to see if the time is as correlation to when the system was booted up. I don't believe it's a problem with the logging server because it is still receiving logs from other systems. I know pfSense is logging things because I can see the local logs getting new hits. This is on the 2.3 beta and has been happening before the update to 2.3 beta. If there is any information I can provide please let me know.

    Thank you

  • When you write "remote logging", are you referring to the built-in Syslog and forwarding it to an external/internal custom solution, remote/local Syslog server or are you using something else for sending/collecting logs/events?

  • I'm referring to the built in section on the web interface title "Remote Logging Options' under the menu Status > System Logs > Settings. I have filled out all that information to forward my logs to a Splunk server using the default port and protocol of 514/UDP.

  • Packet capture filtered on port 514 on the interface facing the syslog server. Do you see traffic leaving the system when you have gaps on the syslog server?

  • This is whats happening on a full packet capture. This is the packet minus the payload. They look the same when packets are being received successfully too so it appears to not be pfSense. Unless I'm missing something. If any one has any ideas I'm happy to hear them.

    22:07:40.316341 00:25:90:XX:XX:XX > 00:0c:29:XX:XX:XX, ethertype IPv4 (0x0800), length 167: (tos 0x0, ttl 64, id 7838, offset 0, flags [none], proto UDP (17), length 153)
        10.X.X.X.514 > 10.X.X.X.514: [bad udp cksum 0x1520 -> 0xf1b7!] SYSLOG, length: 125
    	Facility daemon (3), Severity info (6)
    	Msg: Mar 10 22:07:40 charon: 02[NET] <con3000|10> received packet: from 162.XXX.XXX.XXX[500] to 24.XXX.XXX.XXX[500] (92 bytes)</con3000|10>

  • Looks like it's likely normal. The bad checksum is normal with checksum offloading enabled. Do the same capture on the syslog server to further confirm. It indeed is sending it, so likely it'll show up at the syslog server too, and Splunk is either logging it somewhere you're not expecting or dropping it.

  • Thanks for the help!

    I'll dig more into it on that side.