Mailscanner + spamassassin + clamav package

alessandroa

@alessandroa:

I've just reinstalled my pfsense (2.0.1-RELEASE (i386)
built on Mon Dec 12 17:53:52 EST 2011
FreeBSD 8.1-RELEASE-p6), no firmware upgrade.
I've reisntalled postfix forwarder (2.8.7,1 pkg v.2.3.4_1 ) and I not find any clamv file and freschclam command…

I've upgraded pfsense firmware and no clamv file or process are present. After I've installed mailscanner package and /var/log/clamav folder is present and /usr/local/etc/clamd.conf is present. User in conf file is clamv, now I change the user in "postfix" and I try.

Thank you

Ok, now is working right but I had to change permission in two folder, /var/run/clamav and /var/db/clamav

The question now is: how I can view the log of SPAM messages? The problem is the fine tuning of the system and I need to view the log.

Thank you.

marcelloc

@alessandroa:

The question now is: how I can view the log of SPAM messages? The problem is the fine tuning of the system and I need to view the log.

enable postfix log and do a tail -f /var/log/maillog | grep-i spam

alessandroa

@marcelloc:

@alessandroa:

The question now is: how I can view the log of SPAM messages? The problem is the fine tuning of the system and I need to view the log.

enable postfix log and do a tail -f /var/log/maillog | grep-i spam

Thank you marcelloc but this isn't a good solution for me. In my organization people want know if a message was arrived or not. The classic question is: "I'm waiting an e-mail from xxxx@yy.com, is it arrived?" then I check the log and I can change the antispam rules if was blocked.

Maybe the best solution is bouce spam in a mailbox with auto-deletion in 7 days.

Thank you marcelloc!!

marcelloc

@alessandroa:

In my organization people want know if a message was arrived or not. The classic question is: "I'm waiting an e-mail from xxxx@yy.com, is it arrived?" then I check the log and I can change the antispam rules if was blocked.

Just enable the database logs on package gui and search mails on search mail tab

You can also create users on pfsense with just search mail permissions.

alessandroa

@marcelloc:

Just enable the database logs on package gui and search mails on search mail tab

You can also create users on pfsense with just search mail permissions.

Thank you marcelloc, in postfix configuration I've set log destination in /var/log/maillog and update sqllite every minute, Search mail is already ok.
Now I receive a domain with only 10-15 mail at day and no spam so I can't check earnest the packages. I see only relay denied but no spam reject. At the end of month I'll point the MX of my principal domain, about 5500 spam messages at day.

Thank you.

marcelloc

run saupdate on console to be sure spamassassin rules are fine/up-to-date.

In one server I forgot to check it and got no messages marked as spam.

FiSHswe

Hi Marcelloc!
i've never been able to make Mailscanner start :(
This is what i keep gettin' in log, any ideas?

php: /pkg_edit.php: The command '/usr/local/etc/rc.d/mailscanner start' returned exit code '1', the output was 'Starting mailscanner. eval: /usr/local/sbin/mailscanner: not found /usr/local/etc/rc.d/mailscanner: WARNING: failed to start mailscanner'

marcelloc

@FiSHswe:

i've never been able to make Mailscanner start :(

Did you tried to check and save all config tabs on it's gui?

FiSHswe

Yep, Done that!
I have tried to uninstall and reinstall several times and still the same error in log.

When i try to run sa-update i get "command not found"
freshclam gives me "ERROR: Can't create temporary directory /var/db/clamav/clamav-3294ff160111fb6d1625622676d2d9c9
Hint: The database directory must be writable for UID 106 or GID 106"

ps ax | grep -i mailscanner
259 0 S+ 0:00.00 grep -i mailscanner

running spamassassin –lint gives me "spamassassin: Command not found

Dont recall that i ever uninstalled any old version of mailscanner.
Whats the proper way to do a total manual cleanup of mailscanner?

marcelloc

@FiSHswe:

When i try to run sa-update i get "command not found"

weird

@FiSHswe:

freshclam gives me "ERROR: Can't create temporary directory /var/db/clamav/clamav-3294ff160111fb6d1625622676d2d9c9
Hint: The database directory must be writable for UID 106 or GID 106"

try to fix this missing dir/permission

fabiana

Hi

First I want to say thanks for this cool Package. The only problem we have is, that I think the Option Antivirus -> "Allow Pssword-Protected archives" is ignored. If I set this in the config File it works, but in the GUI it's ignored.

Thanks.
Fabian

marcelloc

@fabiana:

I think the Option Antivirus -> "Allow Pssword-Protected archives" is ignored. If I set this in the config File it works, but in the GUI it's ignored.

Thanks for your feedback fabiana,

I've pushed a fix for it.

reinstall the package in 15 minutes or manually fix /usr/local/pkg/mailscanner_antivirus.xml following this

https://github.com/bsdperimeter/pfsense-packages/commit/2d121b3b6fe79e301daf57b6a1060675a8886023

att,
Marcello Coutinho

fabiana

Hi

Thanks, I just reinstalled the package. But now I get in /var/log/system.log:

Jan 22 09:52:48 legatech root: /usr/local/etc/rc.d/clamav-clamd: WARNING: failed to start clamav_clamd

If I try to start "/usr/local/etc/rc.d/clamav-clamd start" I get:

Starting clamav_clamd.
ERROR: Can't open /var/log/clamav/clamd.log in append mode (check permissions!).
ERROR: Can't initialize the internal logger
/usr/local/etc/rc.d/clamav-clamd: WARNING: failed to start clamav_clamd

Then I "touch /var/log/clamav/clamd.log", "chown postfix:clamav clamd.log", "chmod 777 clamd.log" (could be more restrictive I guess) and try to start I get:


Starting clamav_clamd.

ERROR: LOCAL: Socket file /var/run/clamav/clamd.sock could not be bound: Permission denied
/usr/local/etc/rc.d/clamav-clamd: WARNING: failed to start clamav_clamd

chmod 777 /var/run/clamav fixed the problem. But also here, maybe it should be more restrictive.

And after reinstall, I miss "/usr/local/etc/rc.d/mailscanner", in the "services" Tab, I can't start the mailscanner…HELP! :)

Ok, I extracted the file from the .tbz and mailscanner finally works. postfix and mailscanner services don't start automatically after reboot, but I can start them manually in Status -> Services, then everything is fine.

Regards,
Fabian

jeppunen

I have managed to get spamassasin to work with my Pfsense. I had to do manually some chown changes but now spam gets blocked as supposed. Great packake.

But I have a problem with delay. The delay is about 30-50 seconds for every message. Which settings I should check first to get delay to 1-15 seconds?

Cheers!

marcelloc

It depends, what hardware are you using for mail scanner?

jeppunen

@marcelloc:

It depends, what hardware are you using for mail scanner?

The hardware is HP Proliant DL360 G4 Server, with 4G RAM, RAID1 74Gb HDD, 2 x XEON so there is guite a powerfull hardware. I found one mail that went throught the process from the maillog:

Mar 20 16:48:27 pfsense postfix/smtpd[45773]: connect from newsletter.nebula.fi[217.149.57.249]
Mar 20 16:48:27 pfsense postfix/smtpd[45773]: warning: connect to private/anvil: No such file or directory
Mar 20 16:48:28 pfsense postfix/smtpd[45773]: warning: connect to private/anvil: No such file or directory
Mar 20 16:48:28 pfsense postfix/smtpd[45773]: warning: problem talking to server private/anvil: No such file or directory
Mar 20 16:48:28 pfsense postfix/smtpd[45773]: 6AB8262CEED: client=newsletter.nebula.fi[217.149.57.249]
Mar 20 16:48:28 pfsense postfix/cleanup[45923]: 6AB8262CEED: hold: header From: Jerkka Tuote Oy jerkkatuote@jerkkatuote.fifrom newsletter.nebula.fi[217.149.57.249]; from= errors@mailerrs.hosting.fito=<posti@<em>ocxxxxxxxxx.fi> proto=ESMTP helo= <newsletter.nebula.fi>Mar 20 16:48:28 pfsense postfix/cleanup[45923]: 6AB8262CEED: message-id=20130320144826.E824280104@newsletter.nebula.fi
Mar 20 16:48:28 pfsense postfix/smtpd[45773]: warning: connect to private/anvil: No such file or directory
Mar 20 16:48:29 pfsense MailScanner[56797]: New Batch: Scanning 1 messages, 290323 bytes
Mar 20 16:48:29 pfsense postfix/smtpd[45773]: warning: connect to private/anvil: No such file or directory
Mar 20 16:48:29 pfsense postfix/smtpd[45773]: warning: problem talking to server private/anvil: No such file or directory
Mar 20 16:48:29 pfsense postfix/smtpd[45773]: disconnect from newsletter.nebula.fi[217.149.57.249]
Mar 20 16:48:29 pfsense MailScanner[56797]: Virus and Content Scanning: Starting
Mar 20 16:49:03 pfsense MailScanner[56797]: Spam Checks: Starting Mar 20 16:49:03 pfsense MailScanner[56797]: Expired 4 records from the SpamAssassin cache
Mar 20 16:49:03 pfsense MailScanner[56797]: Message 6AB8262CEED.A4789 from 217.149.57.249 (errors@mailerrs.hosting.fi) to ocxxxxxxxxx.fi is too big for spam checks (290323 > 200000 bytes)
Mar 20 16:49:03 pfsense MailScanner[56797]: Requeue: 6AB8262CEED.A4789 to 94AE862CEFE
Mar 20 16:49:03 pfsense MailScanner[56797]: Uninfected: Delivered 1 messages
Mar 20 16:49:03 pfsense postfix/qmgr[30137]: 94AE862CEFE: from=errors@mailerrs.hosting.fi, size=289619, nrcpt=1 (queue active)
Mar 20 16:49:03 pfsense MailScanner[56797]: Deleted 1 messages from processing-database
Mar 20 16:49:03 pfsense postfix/smtp[19548]: 94AE862CEFE: to=<posti@<em>ocxxxxxxxxx.fi .fi>, relay=62.216.102.245[62.216.102.245]:25, delay=36, delays=36/0.01/0.04/0.05, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as D349B7FE54)
Mar 20 16:49:03 pfsense postfix/qmgr[30137]: 94AE862CEFE: removed

So there is 30 second delay from virus scanning to spam check:

Is this an artificial delay controlled via some config or is it a problem?

Thanx for your answer Marcello, you have done a great job with this package :)</posti@<em>/errors@mailerrs.hosting.fi</newsletter.nebula.fi></posti@<em>/errors@mailerrs.hosting.fi /jerkkatuote@jerkkatuote.fi

jeppunen

@jeppunen:

Mar 20 16:48:29 pfsense MailScanner[56797]: Virus and Content Scanning: Starting
Mar 20 16:49:03 pfsense MailScanner[56797]: Spam Checks: Starting

Here is the more detailed version (with debug on) of the similar message log:

Mar 21 07:32:16 pfsense MailScanner[60965]: New Batch: Scanning 1 messages, 4334 bytes
Mar 21 07:32:16 pfsense MailScanner[60965]: Created attachment dirs for 1 messages
Mar 21 07:32:16 pfsense MailScanner[60965]: Completed checking by /usr/bin/file
Mar 21 07:32:16 pfsense MailScanner[60965]: Virus and Content Scanning: Starting
Mar 21 07:32:16 pfsense MailScanner[60965]: Commencing scanning by clamav…
Mar 21 07:32:44 pfsense MailScanner[60965]: Completed scanning by clamav
Mar 21 07:32:44 pfsense MailScanner[60965]: Spam Checks: Starting
Mar 21 07:32:44 pfsense MailScanner[60965]: Expired 2 records from the SpamAssassin cache
Mar 21 07:32:52 pfsense MailScanner[60965]: SpamAssassin returned 0
Mar 21 07:32:52 pfsense MailScanner[60965]: Requeue: 099E562CFE2.A127A to 0A1B662CFE4
Mar 21 07:32:52 pfsense MailScanner[60965]: About to deliver 1 messages
Mar 21 07:32:52 pfsense MailScanner[60965]: Uninfected: Delivered 1 messages

It seems that clamav takes quite a lot time to complete, but why is that? The hardware should be powerful enough to handle the scanning in couple of seconds. What could be wrong?

jeppunen

Ok, to resolve my slow scanning problen, I'm trying to get to the starting point and I have removed all packages (Postfix, Mailscanner, HAVP) and deleted all the files I could find that are related to mailscanner or postfix and PERL. Then I have reinstalled postfix and mailscanner (nothing else packages).

Commands I executed and came out clear:

ps ax | grep -i mailscanner:
11207 ?? S 0:01.13 MailScanner: waiting for messages (perl5.12.4)
14283 ?? S 0:00.75 MailScanner: waiting for messages (perl5.12.4)
14518 ?? S 0:00.77 MailScanner: waiting for messages (perl5.12.4)

/usr/local/bin/sa-update
/usr/local/bin/spamassassin –lint

But:
/usr/local/bin/freshclam gives error:

ERROR: Parse error at line 20: Unknown option MilterSocket
ERROR: Can't open/parse the config file /usr/local/etc/freshclam.conf

Mail is flowing through the scanner but in the maillog there is error:

Mar 22 12:24:25 pfsense MailScanner[50143]: Virus and Content Scanning: Starting
Mar 22 12:24:25 pfsense MailScanner[50143]: ERROR: Can't open file or directory
Mar 22 12:24:25 pfsense MailScanner[50143]: Spam Checks: Starting

Are these errors related and how can I resolve the MilterSocket error?

Cheers!

jeppunen

@jeppunen:

Mar 22 12:24:25 pfsense MailScanner[50143]: Virus and Content Scanning: Starting
Mar 22 12:24:25 pfsense MailScanner[50143]: ERROR: Can't open file or directory
Mar 22 12:24:25 pfsense MailScanner[50143]: Spam Checks: Starting

Are these errors related and how can I resolve the MilterSocket error?

Yes, they were! Virus Scanner did not complete, because it did not found any database. I fixed the MilterSocker error just by uncommenting the row from the /usr/local/etc/freshclam.conf (plus few other lines gave me an error) and I had to add line "DatabaseMirror db.fi.clamav.net" to the freshclam.conf.

After this freshclam downloaded latest database and now I see nice results, only 10 seconds of scanning time and this is fine by me:

Mar 22 13:38:56 pfsense MailScanner[14670]: Virus and Content Scanning: Starting
Mar 22 13:39:06 pfsense MailScanner[14670]: Spam Checks: Starting

So problem solved, hope my few past posts helps someone else also to debug their slow scanning problem.

But one question still remains; is the clamav better to use or clamd? Am I correct if I say that clamd is a service and clamav has to start each time to do the scanning? At least in my enviroment clamav takes about 15-20 seconds to pass the message and with clamd it is only about 5 seconds. So I'm currently going with the clamd.

cymesa

I have a fresh install of pfSense 2.0.3 with lasted Postfix 2.10.0, after spending a good time setting up the whole system and Postfix, I installed the Mailscanner 4.84.5_3 and then Squid 3.1.20. But Mailscanner not start, this is the error:
############################################################################
php: /pkg_edit.php: The command '/usr/local/etc/rc.d/mailscanner start' returned exit code '1', the output was 'Starting mailscanner. Can't load '/usr/local/lib/perl5/site_perl/5.12.4/mach/auto/Filesys/Df/Df.so' for module Filesys::Df: /usr/local/lib/perl5/site_perl/5.12.4/mach/auto/Filesys/Df/Df.so: Undefined symbol "PL_stack_max" at /usr/local/lib/perl5/5.12.4/mach/DynaLoader.pm line 200. at /usr/local/sbin/mailscanner line 91 Compilation failed in require at /usr/local/sbin/mailscanner line 91. BEGIN failed–compilation aborted at /usr/local/sbin/mailscanner line 91. /usr/local/etc/rc.d/mailscanner: WARNING: failed to start mailscanner'
#############################################################################
I removed the Mailscanner and I installed again with the same result.
My pkg_info | grep perl show:
p5-DBI-1.616_1
p5-Error-0.17016
p5-MIME-Tools-5.502,2
perl-5.12.4_3
perl-threaded-5.12.4_4
What should I do? And thank you very much for your excellent work.
Cy