How to analyse a problem using multiple logfiles?
-
Hello,
I have a problem on pfSense which I would like to analyse using the logs. Problem is that there is not one big logfile in a standard format, but a whole lot of logs (in /var/logs).
I could probably define forwarding alarms to a syslog server (which cost performance) or write a tool which import all the logs in a DB, but I hope that there is a more elegant/simple option.
Ideal would be if I could perform the following steps:
- copy /var/log (the logfiles) to a pc (not a problem, I can do that with e.g. Bitvise)
- start some simple tool which load all logfiles into a DB (source filename, time, program, message data etc in separate DB-fields)
- a gui which allows me to query the data
Doing this would allow me to see if an event/message in logfile "A" is perhaps related to events in other logfiles.
-
@louis2 said in How to analyse a problem using multiple logfiles?:
I could probably define forwarding alarms to a syslog server (which cost performance)
True.
But it will be far less as doing your own copy / parse / match / etc on pfSense.I forward 'everything' to a remote syslog 'server', a classic Debian device (on my LAN), and over there I can go wild with all the scanning, matching etc etc etc.
I presume GUI etc are possible.
I'm using fail2ban myself, who can do more then just 'ban', it can also 'mail' when an event happens.I wouldn't store these logs in a DB, which would need far more resources an the syslog server.
-
Yup, I would export them to a syslog server and analyse it there:
https://docs.netgate.com/pfsense/en/latest/monitoring/logs/remote.htmlWhat's the problem you are trying to analyse?
Steve
-
I noticed that my IPV6-gateway is now and then stopping for unkown reason. Someone is suggesting that I should look at multiple logs for things which might be related.
For that reason and to see a longer history than you can see in the GUI, I like the idea. Next to that analyzing other logs e.g. related to dns-lookups might be interesting as well (for example, I try to block traffic from the red-zone as much as possible)
So, thats the reason I was interested in this issue, where an DB option would e.g. allow me to see which websites are queered by my redzone server (so that I can decide to pass or block them).
Further on note that
- I intent to do this kind of analyses "if there is a reason" only
- and do not have a syslog or greylog server running (it would be extra effort to tale care of that)
-
You can use the syslog-ng package on pfSense itself. It just logs though, you would still need to export it to analyse it.
Steve
-
I did install GrayLog in a Jail on my TrueNas system. Lets see how that ^works^
