Logs (mostly) stopped working one day.
-
On may 11 my pfsense firewall logs stopped working with the exception of lighttpd.error.log.
Firewall rule updates, system bounces, service restarts, dhcp updates, ntp changes, etc log nothing.
Further more I cant seem to display what is in the existing log which is approximately 512K for system.log alone.
I have tried bouncing syslog both with a reboot and after the fact with no change in the state.
I have done absolutely nothing on the system with regard to changes other than I have tinkered with snort rules and whitelists(which btw dont seem to work).
What am I missing?
-
So what little I have read this doesnt seem to look right and I have zero clue why it would be wrong if it is. Do these not need fully qualified paths and files associated with them?
[2.0.1-RELEASE][root@ares.local]/var/log(5): cat /etc/syslog.conf
!ntpdate,!ntpd
!ppp
!pptps
!poes
!l2tps
!racoon
!openvpn
!apinger
!relayd
!hostapd
!-ntpd,racoon,openvpn,pptps,poes,l2tps,relayd,hostapd -
Ok so clog is creating the files and specifying a size for a binary file. I still dont see why it would out of the blue for no given reason simply stop logging.
BTW, if I add a line to push items to /var/log/system.log it "works" but its not longer the binary format. After an hour+ or searching Ive still found no information on this setup. Anyhow help would be greatly appreciated.
-
When I would attempt a clog -f system.log I would get the following:
"clog: ERROR: could not write output (Bad address)"
I just did a full reinstall of packages under the Backup/Restore tool.
I no longer the above error but still nothing that is using the binary clog format is logging.
Can anyone explain how the configuration for the syslog+clog works? /etc/syslog.conf doesnt appear to have everything required (ie: where to log) and I can find squat online regarding this in any helpful level of detail and there are no man pages. Basically Im shooting in the dark trial and error like hoping I hit something.
Also after the reinstall (and upgrade) now snort wont start. Of course I have zero method of debugging it because nothing logs. =/
update: So I just tossed . /var/log/wtf into the syslog.conf. So at least I can now get logs. clog is obviously still busted beyond all working.
-
When I would attempt a clog -f system.log I would get the following:
"clog: ERROR: could not write output (Bad address)"
Always fun to get error messages where the only result on Google for the log is your own post. I've never seen that, and obviously no one else has as well. Maybe try installing the 2.0.1 update on top of 2.0.1, that will reinstall the clog binary and all the others which may fix it. Given no one has ever gotten that log before, and nothing touches the clog binary, I'm not so sure that will do anything but it's the first thing I would try.
-
@cmb:
When I would attempt a clog -f system.log I would get the following:
"clog: ERROR: could not write output (Bad address)"
Always fun to get error messages where the only result on Google for the log is your own post. I've never seen that, and obviously no one else has as well.
Yah, I was more than perturbed when I saw posts on "unclogging" ink jet printer heads. Tried numerous search engines as well. Maybe my search F00 just sucks, dunno.
btw:
250613067 6632 /usr/sbin/clog
815765781 40036 /usr/sbin/syslogdAnyhow, are you saying to in effect reinstall pfsense from media or did I miss some reinstall "big red button" like the package one under backup/restore?
-
Just upload the update file for the version you're on under System>Firmware.
-
Sorry to come in late on this:
@Amarth:When I would attempt a clog -f system.log I would get the following:
"clog: ERROR: could not write output (Bad address)"
I have seen that error message. I suspect it means the pointer to the start of the log information in the circular log file is corrupt (e.g. points outside the valid file area) and therefore clog won't write anything from the file. I suspect the pointer CAN be corrupted when the system has a "dirty" shutdown (e.g. panic or sudden power failure).
PERHAPS the only way to recover is to reinitialise the log file and reboot (or backup config, reinstall, restore config).
-
The cksums are the same as those on a clean download and Ive had no unclean reboots.
The clog binary files appear to be recreated from scratch each reboot so Im guessing thats not the problem either.
Can anyone explain how clog creates these files? ie: Where is the config and the method of initializing the files. Does it some how run through syslogd as an option and if so how does it know what and where to write?
editWell I just looked and something is destroying the syslogd.conf file everytime the service starts. It sure as hell wasnt me but if I copy it from the FULL firmware download:
local0.* %/var/log/filter.log
local3.* %/var/log/vpn.log
local4.* %/var/log/portalauth.log
local7.* %/var/log/dhcpd.log
local7.none %/var/log/system.log
kern.debug;lpr.info;mail.crit; %/var/log/system.log
news.err;local3.none;local4.none; %/var/log/system.log
.notice; %/var/log/system.log
local0.none;daemon.info %/var/log/system.log
daemon.info;security. %/var/log/ipsec.log
auth.info;authpriv.info %/var/log/system.log
auth.info;authpriv.info |exec /usr/local/sbin/sshlockout_pfIt seems to be logging now. So, what would have recreated syslogd.conf as near the top of this original post and continued to do so under reboots and service restarts?
-
Right as I hit post for this I just found what seems to do it.
"Disable writing log files to the local RAM disk" If that is checked then clog no longer works and the syslogd.conf file is turned into that listed at the beginning of the post and hence nothing will log to any place any longer. I recommend that setting be renamed to "Disables all logging."