Logmail - sending syslogs by mail
-
Hello,
I've been working on a script hat sends the syslogs to a mail address. Similar functionality can be found on some/many hardware firewalls.
Current state of development is:
- I've got a working php script, that reads in the clogs and sends them using a direct smtp connection to any mail account.
- It writes down the information were it left off to /var/log/{$logfile}.logtailmail and therefore it just sends the information it has not already sent.
What needs to be done:
-
Currently there is no interface-> Extend interface as functionality grows -
Also the whole package round up is not done yet . -
function that add/removes the cronjob ..-> all done in 0.5 .. -
add smtp auth options -> PLEASE TEST! Try logmail-0.3.1 or newer
Feature wishes (for later inclusion):
-
"Alarming" and "Ignore" filtering lists: The logs matching the alarming list will be listed at the top, those of the ignore list will be not listed at all.
-
Automatically setting the cronjob intervall smaller in case logs were missed. (<- still no guarantee) -> the cron jobs runs every minute but it only sends out mails when there are more than 5 (todo: make configureable) new lines were found.-> it's all configurable now (release: 0.5) :) -
Running as a daemon (?)will not happen -
Encryption in some way: compressed zip attachment, gpg, s/mime, alternatively: mcrypt (proposed by dsh)
-
SSL/TLS for mail delivery
The first thing what I'd like to do is to share what I have up to now with you:
-
The logtailmail class: http://pastebin.ca/454361-> check pre-releases below.. -
The SMTP Class is taken from: http://www.phpclasses.org/browse/package/14.html-> now part of the patch
My questions are:
-
Should I go on and try to make a package out of it or should I try to make a patch for the core system?
-
I need to store the information were the script left of the last time, right now its: /var/log/{$logfile}.logtailmail is that ok, or should I place it somewere else?
Thanks dexcs, dsh, summat for your help :)
-
This should go in the base system I think.
Please continue developing the script. The critical parts now is the auto installation of cron and a simple webConfigurator screen.
-
Thanks for your reply sullrich,
it looks like I'm half way through .. :)
I've modified:
/usr/local/www/diag_logs_settings.php : added inputs
/usr/inc/system.inc : added system_logmail_setup() -> sets up cronjob ..I've added:
/usr/inc/smtp.inc : The above mentioned SMTP class
/usr/local/sbin/logmail.php : script that is called by cron ..It works so far :)
The patch attached, should create all required files ..
Note that this is still NOT production ready! But all testers are very welcomed! :) Thanks dexcs for being the first one :)Please check my first post for a updated todo/wishlist..
Update:
Bugs in logmail-0.1:-
remote smtp host is not used, instead the host is automatically queried using dns/mx records..(fixed in 0.3)
fix: change $smtp->direct_delivery = 1; to $smtp->direct_delivery = 0; in /usr/local/sbin/logmail.php:154
todo: the automatic selection of the smtp host will eventually become a build in feature, but later .. -
due to the fact that the /etc/inc/pfsense-utils.inc:add_text_to_file(); does add same lines multiple times (though the description states that it doesnt -> bug?),
upon enabling and later changing the settings on diag_logs_settings.php will cause the cronjob to be added more than once in the crontab.
Note that removing all cronjobs works with remove_text_from_file();
BTW: Note the namechange .. logtailmail -> logmail ..
-
-
I have a feature request for this: ;)
Could you implement some kind of encryption? Maybe like compressing the log as a zip with userconfigurable password so you can't uncopmpress it without knowing the password? Logs are not meant for everybodies eyes. -
Good idea. We can include a zip binary if needed for this.
-
I have a feature request for this: ;)
Could you implement some kind of encryption? Maybe like compressing the log as a zip with userconfigurable password so you can't uncopmpress it without knowing the password? Logs are not meant for everybodies eyes.Nice idea, though after some thoughts about it: Wouldn't it be nicer to encrypt the mail using the webgui's private key? Found a php based gpg class for this as well as an article on zend dev zone about this. All that's needed is a gpg executable.
What are your thoughts about this? -
I'd like to share logmail-0.3:
CHANGES:
logmail-0.3 - 2007-05-02 23:30
added auth mechs: plain, pop3 before smtp
specify the minumum account of logs before sending
moved all the configs to $config['syslog']['logmail']I've basically added some auth options which NEEDS to be tested!
I've supplied the added/modified scripts a long with a patch. I hope supplying the complete files makes it easier for you to have a look at my work, or maybe you'd even give it a try :)
The complete archive is available here:
http://alexanderschaber.de/ablage/logmail-0.3.tar.gzCheck below for newer version!Have fun, and I'm looking forward for your reply :)
-
Looking good! When will you be ready to commit these to -HEAD and RELENG_1? RELENG_1_2 is frozen so we cannot consider adding it here.
-
Looking good! When will you be ready to commit these to -HEAD and RELENG_1? RELENG_1_2 is frozen so we cannot consider adding it here.
Wow thanks for the quick answer :)
I still need to do some testing on the auth mechs (or: I need someone to test it) .. Other than that, it works great for me :)
I'll have my pfSense box in production next week, then I'll be able to do some real testing (currently just a vm) .. So I'd say that in a week or two the initial version should be ready. -
Sounds good. I'll issue you a commit bit when you are ready.
-
Releasing bugfix version:
logmail-0.3.1 - 2007-05-03 01:00 CEST
fixed a blocker bug: no cron job added
fixed a pfsense bug: add_text_to_file() did not filter
duplicates (patch sent to pfsense
devs and included in releng_1/head)
some minor cleanup..http://alexanderschaber.de/ablage/logmail-0.3.1.tar.gz
Hope this works out better .. :(
-
Maybe I'm jumping in a bit late here, but maybe it would make more sense to have a logging alerts tab or something, and just make the email alerts say that there's a new log alert, with a URL to the webGUI of the pfsense box it came from? Then it's as secure as your webGUI, and avoids the mess of encrypting emails.
If email encryption is the preferred way to go, personally I much prefer S/MIME. Have a field where I can put in my public key on pfsense, and then my MUA would automatically decrypt it since it has my private key.
-
@cmb:
Maybe I'm jumping in a bit late here, but maybe it would make more sense to have a logging alerts tab or something, and just make the email alerts say that there's a new log alert, with a URL to the webGUI of the pfsense box it came from? Then it's as secure as your webGUI, and avoids the mess of encrypting emails.
Nice idea, I might add that as well later .. Though my initial goal was it to have the actual logs on my mail account, so I hope you don't mind, if I go for that functionality first.
@cmb:If email encryption is the preferred way to go, personally I much prefer S/MIME. Have a field where I can put in my public key on pfsense, and then my MUA would automatically decrypt it since it has my private key.
Ok, that's another way to do it, though since pfSense has it's own private key for https, I guess I'll rather just use that, instead of adding another key, but I'm open to this as well if there are more people who would prefer this. Or then I might even implement both..
-
Release: logmail-0.4:
CHANGELOG: logmail 0.4 - 2007-05-04 13:40 CEST fixed bug: that offsets of last leftoff are written, although mail was not sent since the amount of collected logs is less than the specified minimum log count. fixed problems: with cronjob, which caused the script to stop without notice and without ever actually sending mail -> surpressing output of the script (piping to /dev/null) changed input type: password input is now actually an input type password changed default: the minimum count for logs is now 50 .. Note: this is the first version that actually works like a charm on our production pfSense box :)Though still need somebody to test auth mechanisms ..
Download: http://alexanderschaber.de/ablage/logmail-0.4.tar.bz2
-
Release: logmail-0.5:
CHANGELOG: logmail 0.5 - 2007-05-05 13:40 CEST added enhancement: select timeinterval for cron job added checkbox: now possible to select wether logmail should wait for a minimum number of new logs before sending or just send without counting .. added some cosmetics: button onclick: show auth options, hide/show of minlogcnt with checkbox onenable - logmail - after a js:confirm() popup some nice recomended values are loaded changed cron handling:cronjob is now added to $config['cron']['item'] and automatically added to crontab with configure_cron(); this should make the cronjob survive a reboot :) also removing a cronjob adjusted to fit the above mentioned changes. changed clog exec: clog is now called with complete path to make sure it works eventhough PATH might not be set correctly Note: auth mech: pop3 before smtp has been tested and confirmed as working by summat, thanks :)Some embedded or low-profile hardware around to test the cpu/memory load?
Summat tested on p3-500 and 192mb ram .. the first run was quiet a ressource hog (2 mb mail) but later ones didn't create any problem ..
GeekGod: I guess I'll be done at the end of next week .. Check the todo list in the first post ..
Download: http://alexanderschaber.de/ablage/logmail-0.5.tar.bz2
-
@cmb:
Maybe I'm jumping in a bit late here, but maybe it would make more sense to have a logging alerts tab or something, and just make the email alerts say that there's a new log alert, with a URL to the webGUI of the pfsense box it came from? Then it's as secure as your webGUI, and avoids the mess of encrypting emails.
Nice idea, I might add that as well later .. Though my initial goal was it to have the actual logs on my mail account, so I hope you don't mind, if I go for that functionality first.
You're the one doing the work, do it however suits you. :)
@cmb:
If email encryption is the preferred way to go, personally I much prefer S/MIME. Have a field where I can put in my public key on pfsense, and then my MUA would automatically decrypt it since it has my private key.
Ok, that's another way to do it, though since pfSense has it's own private key for https, I guess I'll rather just use that, instead of adding another key, but I'm open to this as well if there are more people who would prefer this. Or then I might even implement both..
I think both would be the way to go (eventually). Not a lot of people use S/MIME, though I think a large number of the people who want this would configure it just for this purpose because it's so convenient. All the emails would show in your MUA as normal messages, but be secure in transit.