Old log files and logging, how does it work?

[madivad]

I'm trying to work out how the log files work in pfSense while I'm trying to diagnose a connection issue.

In the webGui I can see back to 9th April. A few hours ago I could see back to the 7th.

I SSH'd into the box to try and look further and did a head and tail of the ppp.log file to find they were only a few seconds apart. It really made no sense to me (initially):

[2.3.3-RELEASE][admin@gateway.dav3.net]/var/log: head ppp.log 
ink0] PPPoE: Connecting to 'Internode'
Jun 10 08:55:32 gateway ppp: [wan_link0] PPPoE connection timeout after 9 seconds
Jun 10 08:55:32 gateway ppp: [wan_link0] Link: DOWN event
Jun 10 08:55:32 gateway ppp: [wan_link0] LCP: Down event
Jun 10 08:55:32 gateway ppp: [wan_link0] Link: reconnection attempt 79 in 4 seconds
Jun 10 08:55:36 gateway ppp: [wan_link0] Link: reconnection attempt 79
Jun 10 08:55:36 gateway ppp: [wan_link0] PPPoE: Connecting to 'Internode'
Jun 10 08:55:45 gateway ppp: [wan_link0] PPPoE connection timeout after 9 seconds
Jun 10 08:55:45 gateway ppp: [wan_link0] Link: DOWN event
Jun 10 08:55:45 gateway ppp: [wan_link0] LCP: Down event
[2.3.3-RELEASE][admin@gateway.dav3.net]/var/log: tail ppp.log 
Jun 10 08:55:10 gateway ppp: [wan_link0] LCP: Down event
Jun 10 08:55:10 gateway ppp: [wan_link0] Link: reconnection attempt 77 in 2 seconds
Jun 10 08:55:12 gateway ppp: [wan_link0] Link: reconnection attempt 77
Jun 10 08:55:12 gateway ppp: [wan_link0] PPPoE: Connecting to 'Internode'
Jun 10 08:55:21 gateway ppp: [wan_link0] PPPoE connection timeout after 9 seconds
Jun 10 08:55:21 gateway ppp: [wan_link0] Link: DOWN event
Jun 10 08:55:21 gateway ppp: [wan_link0] LCP: Down event
Jun 10 08:55:21 gateway ppp: [wan_link0] Link: reconnection attempt 78 in 2 seconds
Jun 10 08:55:23 gateway ppp: [wan_link0] Link: reconnection attempt 78
Jun 10 08:55:23 gateway ppp: [wan_lCLOG
�q��[2.3.3-RELEASE][admin@gateway.dav3.net]/var/log:

Q1At first I didn't notice the clipping at the extreme head and tail. It took me some time to work out (I think) that the log file rotates within itself? Would that be correct?

The log file is ~500kb and as can be seen from the head and tail above I've had an extensive outage for a few hours this morning, as a result the log file has almost wiped out all the history.

Q2. Are any backups or archives made of the logs?

I was expecting that at 500kb the file would be archived and a new log started. If that's not the case, is there an option to make it so? I know pfSense is designed with flexibility in mind and most/some users will be using it on limited platforms, but I'm running mine on a stand-alone, minicomputer with a 128GB drive. I have the space to burn for archives and/or larger files.

Q3. In the interim I have set the log file size to 1GB. and saved the setting.However a lot of these files are now filled with nulls causing 'head' and 'tail' to not process. Is there an easy way to 'touch' these to fix that??

[biggsy]

This might help.

[madivad]

@biggsy:

This might help.

OH EXCELLENT!

I'd never heard of them but makes perfect sense! Thanks for the reference.

Considering i have the space, are there any issues with me having large log files? I initially set it to 1G, but relented and made them 100MB :D

Cheers!

[biggsy]

@madivad:

@biggsy:

This might help.

Considering i have the space, are there any issues with me having large log files?
Cheers!

Probably not but it might be better, if you have another system sitting around, to set up a syslog server and forward the logs to that.  It opens up a whole bunch of options for analysis.  I use nxLog to capture the logs and Splunk (free) for analysis - both running in a Windows VM.  I used the free version of Kiwi syslog for about 15 years but its performance is very limited and it's passed its prime - a bit like me :)