Data Retention

[jonnytabpni]

Hi Everyone,

Is anyone aware of any method on how I could use my pfsense box for data retention purposes, as per EU law? EU law states that some businesses must store IP header information.

Ideally, pfSense could be used to export the data to a "storage server" or some sort, similar to how syslog works

Any ideas would be appreciated

Cheers

[jonnytabpni]

I guess what I could do is turn on logging for my allow rules, then set pfSense to send the data to a remote syslog server.

Is this wise, given that this will be running 24/7?

Thanks

[jimp]

Remote syslog is definitely the way to go for long-term retention.

[jonnytabpni]

Thanks jimp, but my main concern is the sheer amount of logging that will be done by pfsense. Do you think it will be able to handle the load?

At the minute, our connection rate is about 10Mbps, however I expect this to rise sharply (to around 30Mbps) in the next 12 months or so as we sign up more customers.

The pfsense box is a LGA775-based supermicro box. The currently CPU is an Intel Celeron with 1GB of RAM, however this can be upgraded if you think I need to.

I guess I'm debating between doing it the above way or by using a dedicated box with a NIC in promiscuous mode

Thanks

[jimp]

Logging doesn't take that much, even if you put a log entry on each rule, that only logs each connection, not each packet. If your syslog server is local, it shouldn't have any problems keeping up, even on a smaller device.

Of course the only real way to know is to try.

[jonnytabpni]

Thanks jimp, that's good to know. Yes, our syslog server will be local (on an interface of its own probably) and any remote replication done will be at the backup storage level.

How does pfsense handle logging of connectionless protocols (e.g. UDP)?

Cheers

[jimp]

If there is no connection state when the first packet is seen, it logs the first packet. It won't log again unless the state expires and a new state is created.