SYSLOG gone crazy since 2.0.3 upgrade.
-
I did an upgrade to 2.0.3. This "breaks" syslog output as it breaks the output into 2 lines as per http://redmine.pfsense.org/issues/1938.
I put the fix in as per http://redmine.pfsense.org/issues/1938 which worked fine previously, however 2 days later and my log files which usually are 7-8MB is around 70MB. Today is already at 72MB and climbing.
The problem is that I'm seeing duplicate lines, random numbers of them between 4 and 10.
These are not sequential - ie I dont see lines 1,1,1,1,1,1,2,2,2,2,3,3,3,3,3,3,3,3,3,3,3,3 but instead I'll see 1,2,3,4,5,6,1,2,3,1,2,3,4,1,2,2,3,4,2,3,4,5,6,1,2,3,4,5,6.At this rate I've going to run out of disk space ~10x quicker and I don't need the additional lines.
I've checked the syslog server and it definitely receiving the duplicate lines.Anyone got any ideas?
-
Log file is at 134MB and I still have time left for today.
I'm going to have to roll back to an earlier version if this carries on
-
This has gotten completely out of hand now - each line is being sent 18 times!!!
My log file for today is 260MB
-
The filter output is the same on ALL 2.0.x installs. That is not new for 2.0.3. It would not be the source of this issue.
The only confirmed issue we've seen with syslog on 2.0.3 was this: http://forum.pfsense.org/index.php/topic,61186.15.html (fix is in the thread on the second page)
Please post the contents of /var/etc/syslog.conf
And the output from:
ps uxawww | grep syslogd -
Hi,
I rolled back the change I made to stop the lines splitting and they are still massive files ~150Mb
Here is the /var/etc/syslog.conf (I sanitised the IP)
!ntp,ntpd,ntpdate
. %/var/log/ntpd.log
!ppp
. %/var/log/ppp.log
!pptps
. %/var/log/pptps.log
!poes
. %/var/log/poes.log
!l2tps
. %/var/log/l2tps.log
!racoon
. %/var/log/ipsec.log
!openvpn
. %/var/log/openvpn.log
!apinger
. %/var/log/apinger.log
!relayd
. %/var/log/relayd.log
!hostapd
. %/var/log/wireless.log
!-ntpd,racoon,openvpn,pptps,poes,l2tps,relayd,hostapd
local0.* %/var/log/filter.log
local3.* %/var/log/vpn.log
local4.* %/var/log/portalauth.log
local7.* %/var/log/dhcpd.log
.notice;kern.debug;lpr.info;mail.crit; %/var/log/system.log
news.err;local0.none;local3.none;local4.none; %/var/log/system.log
local7.none %/var/log/system.log
security. %/var/log/system.log
auth.info;authpriv.info;daemon.info %/var/log/system.log
auth.info;authpriv.info |exec /usr/local/sbin/sshlockout_pf 15
.emerg *
!
. @nnn.nnn.nnn.nnnps uxawww | grep syslogd
root 10501 0.0 0.1 14848 2852 ?? Ss 29Apr13 15:53.97 /usr/sbin/syslogd -s -c -c -l /var/dhcpd/var/run/log -f /var/etc/syslog.conf
root 36241 0.0 0.0 9040 1460 0 S+ 2:52PM 0:00.00 grep syslogd -
Does this mean anything to anyone?? I have to do something to fix this pretty fast - my log hit 774Mb yesterday and I'm running short of disk space fast!
-
I've got no choice but to pull Pfsense from the firewall now.
Last 5 days logs files have been over 1GB each, 140x the average of the log files prior to the upgrade and I dont see this resolving itself. -
OK, as a last ditch attempt to get this sorted (the Syslog volume ran out of space so I had to do something) I set the development snapshots and upgraded and so far so good!
Syslog appears to be only sending 1 line per log line now and the files are growing at approximately the correct rate.
I'll continue to monitor.
-
1 month on and it's back to 10 duplicated lines and 1GB log files….. not happy.
-
1 month on and it's back to 10 duplicated lines and 1GB log files….. not happy.
Well, not sure, if you've read the thread jimp has proposed. Exactly this answer in the thread has solved the syslog problem for me.
Please let me know, if your problems still exist after you've applied the proposed changes to /etc/nsswitch.conf.
Peter
-
Not sure how that issue is related to mine. I got the NSSwitch problem immediately after upgrading, but it disappeared just as quick.
My issue was the Pfsense kept sending multiple copies of the same lineAlso, I upgraded to the latest Beta as of 25th June and it's solved the problem and I'm back to 8MB log files again, so whatever the problem was has been fixed…...either that or maybe the old fix for repairing the broken Syslog where is breaks lines in two no longer works correctly as i've not applied it.
-
Not sure how that issue is related to mine. I got the NSSwitch problem immediately after upgrading, but it disappeared just as quick.
My issue was the Pfsense kept sending multiple copies of the same lineAlso, I upgraded to the latest Beta as of 25th June and it's solved the problem and I'm back to 8MB log files again, so whatever the problem was has been fixed…...either that or maybe the old fix for repairing the broken Syslog where is breaks lines in two no longer works correctly as i've not applied it.
I've probably not got what the "multiple copies of the same lines" really mean in your syslog as this is particularly true for the "NSSWTICH" entries. Maybe your initial description has been a bit to abstract at least to me - a few lines from your syslog could have clarified it :).
Nevertheless, it seems unclear why your syslog problems have gone with 2.1 beta. If you're still interested in getting it solved, you'll probably have to send some representive parts of your multiple syslog entries.
Peter
