Firewall filter LOG, GUI search and Circular logging

keyser

Hi.

I just noticed something in my 22.05 install today

I was trying to find some firewall log entries that I know should be there (logged traffic I did myself), but the entries are not visible when using the GUI with a filter applied (searched for source IP - well know for the generated traffic).

It seems that only the current active filter.log is visible/filterable in the GUI. The 19 others (older) i have as I'm doing circular logging, cannot be searched from the GUI.
I validated this by searching an older filter.log.2 from CLI because that was from the timeframe with the expected traffic. Indeed the traffic is present and logged in that file. Its just not searchable from GUI.

Is this by design, a bug, or something I need to disable, so it searched all logfiles?

jimp

Current versions do not do circular logging, they rotate the logs after a certain point.

The GUI does search all rotated log files, it's possible you are not searching the correct fields/terms to match it. Hard to say.

The files it uses to search automatically are based off the chosen compression type for the logs, but it does gather all of the relevant logs when attempting to locate entries.

It does also only search back so many lines, and some of those may be discarded, so it's possible that it's held back by other limits.

keyser

@jimp Yeah, poor choise of words, I meant log rotation.

But my boxes does definitively not include results from the filter.log.1 -> 19 in my search/filters. 100% certain.
It only presents results from the current active filter.log file. I can see that as simple as creating a filter for a destination port that hits a rule I have every once in a while. There is only some 30 results (of a 1000 possible) in the view right now. Those are the same ones I can grep from the current filter.log

But I have hundreds of results in the 1 -> 19 files as well on the same grep. No matter what I do, I can not get results from those files in the filter view.

jimp

It only goes back 10000 lines max. If your main filter log file has more than 10000 lines then it wouldn't go into the other files. If the files are smaller it will look at them all.

See https://redmine.pfsense.org/issues/11666

keyser

@jimp Ahh, so I’m hitting the 10.000 lines maximum…

But that makes the GUI useless for log searching, because who does not have much more than 10.000 linies in their rotated filter logs?

That figure needs at least two more 00’s before its of any use.
Is there no way to increase that, or get a proper GUI firewall log tool without resorting to overkill with external syslog and greylog/splunk/elasticsearch like tools?

jimp

It's partially a holdover from the days when clog could rarely have more than a few thousand lines and partially because parsing that many filter log lines can be a burden on the firewall so it is limited so it doesn't put undue strain on the system.

You can increase it on your own by editing the line at https://github.com/pfsense/pfsense/blob/master/src/etc/inc/syslog.inc#L705

We haven't come to a decision on what the best course of action is there overall.

keyser

@jimp said in Firewall filter LOG, GUI search and Circular logging:

It's partially a holdover from the days when clog could rarely have more than a few thousand lines and partially because parsing that many filter log lines can be a burden on the firewall so it is limited so it doesn't put undue strain on the system.

You can increase it on your own by editing the line at https://github.com/pfsense/pfsense/blob/master/src/etc/inc/syslog.inc#L705

We haven't come to a decision on what the best course of action is there overall.

Okay - that is very good to know. I think you should make a GUI setting for limiting how far it goes back - with a memory use estimator explaining the consequenses of increasing it (Just like you do where you can edit the current log file size in diskspace terms).

Alternatively there should be developed a dedicated log processing and analytical package people can install if they have CPU/memory enough.