A few questions about logging and reporting tools

[templateunheard]

I'm making a pfSense to fit a purpose and it's going to have a few features such as multi-wan load balancing/failover, IPS solutions and a couple of other things, but I've yet to find a way to enable logging of, searching of and reporting of a few things that I need. Below are the questions I currently have:

1. In load balancing, is there a way to monitor the uptime of connections (how long it's been up, if it's currently up etc and maybe historical uptime eg WAN1 went down 2 weeks ago for 5mins or anything similar to that)
2. Monitoring of each device on the network for uptime, performance, outages etc - so far I'm leaning towards using darkstat, would that fulfill all the above needs and is there a native pfSense alternative? And is there a way to get historical data for all the above as session by session data is ok but I'd prefer something more long-term.
3. Monitoring of network performance - Is there any way of logging data across multiple restarts? I need to measure downtime over an extended period of time, things like WAN outages, connection issues and anything else would apply
4. Logging of all network traffic - is there a way to save these files so they're not gone upon restart? I want to be able to export them into other network analyser tools

I've looked into a syslog server but I'd prefer to keep in within pfSense if possible. I greatly appreciate any help given, I'd be happy to clarify any points above in later comments. Thanks all.

[templateunheard]

@templateunheard bump, still in need of answers if anyone's able

[mcury]

@templateunheard I personally like nfsen+nfdump.
You would need a server to install nfsen+nfdump..

Then, you install softflowd in pfsense and export data to this server..

There were some limitations, but it was good to check all the flows you would need.

[stephenw10]

Yes exporting syslog and netflow data is the way to go for that.

Long term data is not intended to be held in pfSense directly.