Store pfSense (+ packages logs) on external (or internal) USB drive / memory card

provels

@Sergei_Shablovsky FWIW, I just setup the free version Kiwi Syslog in about 10 minutes and have real-time live logs going to it, and I'm definitely not the sharpest knife in the drawer. It can send the logs to whatever server/storage device I want for analysis from USB flash to multi-terabyte RAID, no scripting or cron jobs required. Just point pfSense to the syslog server's IP and choose the logs you want sent. That sounds like a little better solution to me. If an attacker is already crawling all over your network changing archived syslog entries, you have very big problems.

jimp

If a device is compromised, you can't trust logs from that compromised device. It doesn't matter what level of access you believe they obtained, you have no idea if they escalated from there and covered their tracks.

Unless they compromised the syslog server and/or intermediate devices (e.g. took out the network), the logs sent over the network to a syslog server will always be more reliable than logs on the device itself. And in cases of local storage failure, the only way to find that out would be logs sent over the network. The best solution is always local short term logs + long term remote logs. Any logs that happen in the event of a network failure can be obtained from the device itself, but normal logs are viewed/processed centrally.

If you want to be pedantic, the only real secure log would be one put on write-only/WORM type storage which isn't really a thing these days. In the old days with low volume logs they might have even been printed line-by-line on a dot matrix style printer (which has its own downsides). But no matter which method you choose, a clever attacker could likely find a way to compromise it or at least render it ineffectual.

If you want to be lazy and not setup a central log server, just own up to that. Plenty of people are in the same situation. I'm sure there are others who would like to see this request implemented. But you just aren't going to convince anyone who disagrees with this idea in principle to do the work of adding this feature for you. If someone comes along and submits a PR to allow setting a custom log storage/rotation directory on a future version, we'd happily review it.

Sergei_Shablovsky

@provels said in Store pfSense (+ packages logs) on external (or internal) USB drive / memory card:

@Sergei_Shablovsky FWIW, I just setup the free version Kiwi Syslog in about 10 minutes and have real-time live logs going to it, and I'm definitely not the sharpest knife in the drawer. It can send the logs to whatever server/storage device I want for analysis from USB flash to multi-terabyte RAID, no scripting or cron jobs required. Just point pfSense to the syslog server's IP and choose the logs you want sent. That sounds like a little better solution to me. If an attacker is already crawling all over your network changing archived syslog entries, you have very big problems.

There are some CONS about Your solution:

Logs = important part of security, only one way to know "what exactly happened BEFORE the problem". More additional software or hardware You using = more points of potential failure You have. From this point of view extra Kiwi Syslog mean "another one bunch of code, another one point in update schedule, another one source of new bugs, another one point of possible misscompability with bunch of other software You have". This is not about Kiwi exactly, this is about ANY extra software you add to Your system.
Why using extra software from 3rd party instead of function that are standard for each serious fw/router software? This is out common sense. Ok, if You make experiments or advanced setup for Your home router, but even in NetGate hardware list only first 2 models intended for small speeds, other hardware are much more powerful and intended to be using in professional environment, where security are much more important than in home/campus setup.

Sergei_Shablovsky

@jimp said in Store pfSense (+ packages logs) on external (or internal) USB drive / memory card:

If a device is compromised, you can't trust logs from that compromised device. It doesn't matter what level of access you believe they obtained, you have no idea if they escalated from there and covered their tracks.

Absolutely agree with this!

Unless they compromised the syslog server and/or intermediate devices (e.g. took out the network), the logs sent over the network to a syslog server will always be more reliable than logs on the device itself. And in cases of local storage failure, the only way to find that out would be logs sent over the network. The best solution is always local short term logs + long term remote logs. Any logs that happen in the event of a network failure can be obtained from the device itself, but normal logs are viewed/processed centrally.

Exactly about this I am writing: ability to make scheduled encrypted archives + external syslog. Two different procedures for two different purposes: local copy for improving security and in case device main HD/SDD crashed (primary for home users, small campuses, etc...) and remote more powerful log aggregator and analyser (like Splunk) for constantly monitoring and alerting tech stuff.
Local scheduled copying is not for replace remote log aggregator/analyzer, but more like additional security layer for part of users ordinary level.

If you want to be pedantic, the only real secure log would be one put on write-only/WORM type storage which isn't really a thing these days. In the old days with low volume logs they might have even been printed line-by-line on a dot matrix style printer (which has its own downsides). But no matter which method you choose, a clever attacker could likely find a way to compromise it or at least render it ineffectual.

No one of us need to be so mad. :) But I remember that times, not so long time ago... :)
Agree.

If you want to be lazy and not setup a central log server, just own up to that. Plenty of people are in the same situation. I'm sure there are others who would like to see this request implemented. But you just aren't going to convince anyone who disagrees with this idea in principle to do the work of adding this feature for you. If someone comes along and submits a PR to allow setting a custom log storage/rotation directory on a future version, we'd happily review it.

It's not about "i have no time and passion to install syslog server, but have a time and passion to flooding forum". This tread about "making scheduled creating logs archive on a local USB-drive as a part of standard pfSense feature".
As for me personally - I am "totally on Splunk side". All what I am asking in this thread - making local copy archived logs as standard feature of pfSense. :)

From ordinary user side all looks like "Apple magic": they unpack NetGate device, connect cables, go thru the Wizard (or install pfSense on own appliance) -> insert USB-flash -> all working. And all updates, patches makes remotely, not need all time to be physically near appliance.
I am sure that not small part from 600.000+ pfSense users are happy to have a local copy of archived logs.

P.S. Sorry for nub question, what is "submits a PR". ? I start to writing script, with cron work but ned more time for this...

Sergei_Shablovsky

@jimp said in Store pfSense (+ packages logs) on external (or internal) USB drive / memory card:

The best solution is always local short term logs + long term remote logs. Any logs that happen in the event of a network failure can be obtained from the device itself, but normal logs are viewed/processed centrally.

Need to add that using Automatic Configuration Backup (ACB) with conjunction with remote log aggregator/analyzer is the "remote backup", for purpose of backup archived logs on a local removable media better to use separate feature in pfSense.

And of course "ACB with Gold subscription" = paid remote backup option, when "backup archived logs on a local removable media" = free local option only for logs. (for manual local Configuration Backup already exist separate package where possible to adding any paths/files).

stephenw10

It is possible to do this already using the syslog-ng package. You can configure that to store it's logs in any location including some other drive like a memory card. Then just configure the main logs to export to it as well.

The difficulty is that pfSense has no facility for managing additional drives. They are not auto-mounted etc. You can just add new devices to the fstab but what happens if you pull the memory card?

There have been a few scripts written to address this. When NanoBSD was a thing local logs were RAM only so several people wrote stuff to use a separate drive for logging only.

Steve

Sergei_Shablovsky

@stephenw10 said in Store pfSense (+ packages logs) on external (or internal) USB drive / memory card:

It is possible to do this already using the syslog-ng package. You can configure that to store it's logs in any location including some other drive like a memory card. Then just configure the main logs to export to it as well.

Please take attention “storing encrypted archives of logs on a local media”. This is different that “just copy logs to another drive”. ;)

The difficulty is that pfSense has no facility for managing additional drives. They are not auto-mounted etc. You can just add new devices to the fstab but what happens if you pull the memory card?

There have been a few scripts written to address this. When NanoBSD was a thing local logs were RAM only so several people wrote stuff to use a separate drive for logging only.

Steve
Please wait, I just starting writing scripts. Cron, daemon, etc...

stephenw10

Mmm, reading through the syslog-ng manuals I think your need the Premium Edition to store logs encrypted.

Steve

Sergei_Shablovsky

@stephenw10 said in Store pfSense (+ packages logs) on external (or internal) USB drive / memory card:

Mmm, reading through the syslog-ng manuals I think your need the Premium Edition to store logs encrypted.

Steve
Sorry, what You mean ?

I told about storing encrypted archive with logs LOCALLY. This is the first level for advanced users or home users, that not need store logs in a remote place for various reasons.

stephenw10

Indeed, I thought you might be able to do that with syslog-ng since they do have that feature. But it looks to be for PE only so not in the FreeBSD port.

Steve

JonathanLee

use the syslog package, I am using it to send logs from my AP to pfSense however you can also forward syslogs out of pfsense

JonathanLee

I was thinking…

What about you mount a drive like a second SSD or a mpcie to m.2 and have a NVMe drive that you mount to something like /root/logs use gpart to make a partition on that other drive and mount to it with fstab automatically, after use it for snort logs and squid or any other package that lets you pick what location you log to… what would cut down a lot. I mean you can copy to usb drive with fat32 partition why not just do the same thing and make it a log partition ? Wouldn’t that work?

I got this monster Optane drive and I am using a small part as an emergency swap location so it has 255gb I could add a new partition to it with gpart and rock that for a log directory. Any thoughts ?

stephenw10

Yup that can work. The problem is if you have to re-install or at upgrade it may get overwritten. And then how does the package behave if its unable to reach it's log location.

JonathanLee

@stephenw10 does it have to be fat32? I can't get it to mount the zfs manually

Shell Output - mount /dev/nda0p2 /root/LOGS_Optane
mount: /dev/nda0p2: Integrity check failed

Shell Output - gpart show nda0
=>       40  500118112  nda0  GPT  (238G)
         40       2008        - free -  (1.0M)
       2048   16777216     1  freebsd-swap  (8.0G)
   16779264  482344960     2  freebsd-zfs  (230G)
  499124224     993928        - free -  (485M)

I researched it and found how to do it it needs efi for GPT I have learned that way you can make a fat32 that pfSense can allow use of. I think it is restricted to only fat32 for external stuff correct me if I am wrong

It needs to have this done

gpart add -t efi  -s 230GB -l LOG nda0
newfs_msdos -F 32 /dev/nda0p2
mount_msdosfs /dev/nda0p2 /root/LOGS_Optane

Shell Output - gpart show nda0
=>       40  500118112  nda0  GPT  (238G)
         40       2008        - free -  (1.0M)
       2048   16777216     1  freebsd-swap  (8.0G)
   16779264  482344960     2  efi  (230G)
  499124224     993928        - free -  (485M)

It works I can mount it send files to it after this you create a cron to mount it

Warning this can break stuff if you do not know what your doing here for others I am doing a new partition inside a second drive that I am also using as swap so be careful to not wipe out your drives

JonathanLee

Updated my unofficial guide if anyone else wants to try this here is a short guide for you.

https://forum.netgate.com/topic/195843/unofficial-guide-have-package-logs-record-to-a-secondary-ssd-drive-snort-syslog-squid-and-or-squid-cache-system