Firewall logs entries only display the last minute
-
I am struggling to come up with an answer and it must be something obvious.
I have a pair of PFSense with CARP, and after getting everything up and running the logs for the firewall are timing out entries after 60 seconds. So most the time when I go into the Firewall logs, I have a few lines, or nothing. Each entry disappears from the Web GUI front end after 60 seconds. It appears that it is a mixed bag. System shows items older than 60 seconds, and so does open VPN, but load balancer does not.
I have set all the setting back to default that I know of, but I do not see anything based on time. clog -f filter.log show entries much older than 60 seconds.
Anyone have any ideas on this one?
Thanks,
-
RTFM https://doc.pfsense.org/index.php/Log_Settings
-
Come on. Really. And can you point to the part of that Manual that fixes this issue? No, because it is not there. There is no logging setting that is for the am mount of time an entry is displayed in the interface. Which is why this is so confusing.
If I know how to use clog to look at the raw filter files, I think I know where the manual is and have checked it and spent several hours reading forum posts trying to avoid people like you pointing me RIGHT back to what I have been already looking at.
If you have something actually helpful, feel free to point the details. If some how your posting of the log settings which I have been over many times has the answer and I am just missing it, prove me wrong. But in this case, I do not think you can do that.
Anyone else want to be helpful?
-
So are you saying your logs of full and its only from the last minute.. So your logs are getting slammed? You mention a few entries.. What exact log are you looking at - please post picture which is worth 1,000 words..
More context to your actual problem would be helpful in helping you.
What do you have the gui set for entries… Defaults to like 50 or 100 entries I think... I have mine set to 2000.. And log file bumped to 2MB vs the default 500KB...
-
Thanks for the helpful response.
I would provide more context, and on other issues, I can think of alot more context, but this one I am just not sure what is relevant to post.
This is version 2.4.2_1
The log file is not full. It is not getting slammed. The current log file is default yes, at 500kb, but from clog shows entries going back 14 minutes right now. Displaying 250 entries.
I had the logs set to a higher amount, but reset everything to default when this issue started to make sure something was not messed up.So the clog show s 15 minutes of data, but the gui show 0 entries.
If I initiate something to get a blocked packets, the entry will show up in the GUI and then disappear exactly 60 seconds after it hits the logs. For example, if a packet from a foreign source his the filter at 20:02:16, it will disappear from the GUI at 20:03:16 and will still be in the clog until about 20:16:00 or so, depending on the amount of traffic.
I have two other PFSense units on this exactly same version, and either are doing this, so it is unique to this installation.
Thanks,
-
I don't use the load balancer.. I could try and replicate your problem but since you say it doesn't happen on other systems it highly unlikely I would be able to replicate.
You don't have any filter set in gui when looking at the load sharing log?
I would suggest you flush the log.. Which log file are you looking at with clog, the filter.log? I am almost positive that the load bal log file would be relayd.log
My guess is maybe something is being put in that doesn't display in the gui correctly… The load balancer log not really going to have a lot of stuff in it.. Is it??
"Typically this only includes messages about startup events, server availability, and status changes. For example if a server becomes unavailable, or if it recovers. "
Maybe yours is being flooded with stuff that is not displaying in the gui and messing with your gui output? I would suggest you use clog to view the correct log, which woudl not be filter.log for sure - that is the firewall log not load balancing that uses relayd
Suggest you post a screenshot of what your seeing in the log via gui, and via clog and then when your stuff goes away what does clog of relayd.log show?
edit: Ok I fired up a test load bal setup.. See attached gui log and clog... I will check it in a few to see if still there, etc..
edit2: Ok a few min later, still seeing the startup it did add a table test entry..
-
Johnpoz, thank you again for helping out with this.
One point to clarify. It is not the load balancer log that I really care about, it is the firewall one that I care about. I was just suggesting that the issue may not be isolated to the fitler.log and firewall entries.
Looking deeper into this, this looks to be a bigger issue. The log is set to display 250 entries, and even though more entries exist in the log file, they are filtered out of the GUI display because they are not relevant. It was my understanding it would display the last 250 entires that should be shown. But instead the log file is filling up with entries that are not shown in the GUI and thus filtering out the relevant ones.
I have a larger issue. I am seeing the CARP advertise broadcasts from the firewall to the network.
Mar 5 11:50:28 fw filterlog: 48,,,1000000201,em2,match,block,in,4,0x10,,255,0,0,DF,112,carp,56,192.168.99.2,224.0.0.18,advertise,255,1,2,0,1
Mar 5 11:50:28 fw filterlog: 48,,,1000000201,em1,match,block,in,4,0x10,,255,0,0,DF,112,carp,56,192.168.100.252,224.0.0.18,advertise,255,4,2,0,1
Mar 5 11:50:28 fw filterlog: 48,,,1000000201,em0,match,block,in,4,0x10,,255,0,0,DF,112,carp,56,216.7.132.172,224.0.0.18,advertise,255,2,2,0,1This is a virtualized PFSense on VMWare, and to get CARP to work right, you have to have serveral VMWare settings set to get it to work right, basically to rewrite the MAC addresses of the outbound packets. It looks like that is causing a layer 2 loop as well and it is receiving it's own packets back on the network. Well at least the advertisements. That is filling up the log with blocked packets.
Well at least I have an answer for why the logs are not showing what I am looking for.
I will dig deeper on this specific issue. Looks like other people have experienced this layer 2 loop as well, so I will start searching in that area.
Thanks again for the help
-
Layer 2 loop could for sure flood the shit out of your logs..
I have ran pfsense on vmware for years - back on vmware server 1 and 2, etc…
Post up your setup - why would you need to rewrite the mac?
-
Well I started with the filter issue, and moved on to other issues, and now figured it out.
Looks like there was a small configuration issue on VMWare.
https://doc.pfsense.org/index.php/CARP_Configuration_TroubleshootingNeeded to make sure the Net.ReversePathFwdCheckPromisc was changed.
The VMWare Hosts all have multiple trunk ports to the switch, so that was causing a layer 2 loop for the CARP advertisement traffic.
After changing that setting and bouncing the promiscuous mode on each vswitch, all is well.
Thank you for the help, and if anyone else is seeing the same, ther is a trail, from missing log filter entries to the actual root cause.
And a reminder for others, sometime we do read the manual and just need a little help from our fellow gurus on the web.
