Postfix - antispam and relay package

marcelloc · Apr 5, 2017, 7:46 PM

Do you have any non word character on server name like antispam-01 ?

The stats on widget are based on destination recipients. Is this what mailwatch does?

A message sid can have more then one destination recipient.

The spam message on widget is specific for spamassassin message that has the alert on logs with 'is spam'.

Bismarck · Apr 6, 2017, 12:00 PM

@marcelloc:

Do you have any non word character on server name like antispam-01 ?

No its just pfsense

The stats on widget are based on destination recipients. Is this what mailwatch does?

Yes, but it looks like the widget just counts high scored spam and not all, same when searching for spam in postfix_search.php it just finds high scored spam and the numbers of high scored spam are are always the same in the widget and MailMatch.

The spam message on widget is specific for spamassassin message that has the alert on logs with 'is spam'.

Yes, but for some odd reason, only the high scored spam gets logged to sqlite.

In the first unofficial version, the logging to sqlite was working fine, nothing changed since with my setup, except the new Postfix package.

If you like I can send you my maillog?

/edit

Searching for spam in postfix_search.php before the update in a sqlite db it shows all spam messages, the low and high scored.

marcelloc · Apr 6, 2017, 12:52 PM

@Bismarck:

If you like I can send you my maillog?

Send me the link on private message.

marcelloc · Apr 6, 2017, 4:28 PM

@Bismarck:

Yes, but it looks like the widget just counts high scored spam and not all, same when searching for spam in postfix_search.php it just finds high scored spam and the numbers of high scored spam are are always the same in the widget and MailMatch.

The point is that database gets the last status from message. 'Normal spam' on your configuration are sent to user, this way, last state is sent, instead of spam.

@Bismarck:

The spam message on widget is specific for spamassassin message that has the alert on logs with 'is spam'.

There was some logic error on widget I've fixed too.

There was also an index on database that suposed to be unique but wasn't. Fixed this too.

https://github.com/marcelloc/Unofficial-pfSense-packages/commit/5cea314818b93122582be87bdf63c0a1eedad475
Thanks for your feedback. :)

ps: Your mailscanner and spamassassin looks working really great. Did you had to change a lot of default gui config or rules? Are you using other spamassassin rules?

Bismarck · Apr 7, 2017, 9:27 AM

@marcelloc:

https://github.com/marcelloc/Unofficial-pfSense-packages/commit/5cea314818b93122582be87bdf63c0a1eedad475
Thanks for your feedback. :)

I've applied the patch and it looks very good so far! :)

ps: Your mailscanner and spamassassin looks working really great. Did you had to change a lot of default gui config or rules? Are you using other spamassassin rules?

I've changed a lot of small things and yes I use custom scripts, rule channels and clamav sigs etc. but I think a good bayes training is the key.

If you like a can pack it all together for you?

Muito obrigado novamente! ;)

marcelloc · Apr 7, 2017, 12:05 PM

@Bismarck:

If you like a can pack it all together for you?

Sure. Is it something I can apply on mailscanner package installation?

Bismarck · Apr 7, 2017, 12:34 PM

@marcelloc:

@Bismarck:

If you like a can pack it all together for you?

Sure. Is it something I can apply on mailscanner package installation?

Sure why not. I'll make you a list of the things and files and where to get them.

pf-disa · Apr 7, 2017, 4:36 PM

Hello marcelloc,

thank you for your work and this package… Is there any documentation on how to enable postfix as smarthost and to relay via gmail smtp?

I'm trying to configure it as the old package (in pfsense 2.2x) that was working as smarthost for the internal lan, with relay via my gmail account: the only instruction I've found are http://ghanima.net/doku.php?id=wiki:pfsense:postfixmailrelay but I can't make it work (different paths) and various errors in /var/log/system.log

Apr 7 17:42:38 pfSense0 php-fpm[64394]: /pkg_edit.php: Writing out configuration
Apr 7 17:42:40 pfSense0 php-fpm[64394]: /pkg_edit.php: Writing rc_file
Apr 7 17:42:41 pfSense0 php-fpm[64394]: /pkg_edit.php: Stopping postfix
Apr 7 17:42:41 pfSense0 postfix/postfix-script[73787]: fatal: the Postfix mail system is not running
Apr 7 17:42:42 pfSense0 php-fpm[64394]: /pkg_edit.php: The command stop' returned exit code '1', the output was ''
…
Apr 7 17:45:43 pfSense0 php-fpm[67433]: /pkg_edit.php: Writing out configuration
Apr 7 17:45:45 pfSense0 php-fpm[67433]: /pkg_edit.php: Writing rc_file
Apr 7 17:45:46 pfSense0 php-fpm[67433]: /pkg_edit.php: Reloading/starting postfix
Apr 7 17:45:46 pfSense0 postfix/postfix-script[98918]: fatal: the Postfix mail system is not running
Apr 7 17:45:47 pfSense0 php-fpm[67433]: /pkg_edit.php: Postfix setup completed
Apr 7 17:45:58 pfSense0 postfix/smtp[70787]: fatal: SASL library initialization
Apr 7 17:46:59 pfSense0 postfix/smtp[3068]: fatal: SASL library initialization
Apr 7 17:48:00 pfSense0 postfix/smtp[24940]: fatal: SASL library initialization

Executing via ssh the command "/usr/local/etc/rc.d/postfix.sh start" gives the following output:

kern.ipc.nmbclusters: 379268
sysctl: kern.ipc.nmbclusters=65536: Invalid argument
kern.ipc.somaxconn: 16384 -> 16384
kern.maxfiles: 131072 -> 131072
kern.maxfilesperproc: 104856 -> 104856
kern.threads.max_threads_per_proc: 4096 -> 4096
postfix: Postfix is running with backwards-compatible default settings
postfix: See http://www.postfix.org/COMPATIBILITY_README.html for details
postfix: To disable backwards compatibility use "postconf compatibility_level=2" and "postfix reload"
postfix/postfix-script: fatal: the Postfix mail system is already running

What can I check? Thank you for your time…
Riccardo.

marcelloc · Apr 7, 2017, 11:07 PM

New pkg version includes postwhite integration 8)

Thanks for Postwhite, spf-tools projects and Bismarck

Bismarck · Apr 8, 2017, 8:47 AM

@pf-disa - you could try (untested but would make sense)

cd /root

fetch https://raw.githubusercontent.com/marcelloc/Unofficial-pfSense-packages/master/pkg-postfix/files/install_postfix_23.sh

nano install_postfix_23.sh

go down to line 81 and edit like this:

Install postfix package

pkg install postfix**-sasl** libspf2 opendkim libmilter py27-postfix-policyd-spf-python p5-perl-ldap

and then

sh ./install_postfix_23.sh

to get rid of the compatibility warning add

smtputf8_enable=yes
compatibility_level=2

to the custom main.cf options

@marcelloc

This package is getting in a really beautiful shape, its so sad that its not be in the official repo. :'(

Anyway excellent work! :)

marcelloc · Apr 8, 2017, 10:37 AM

I'll include the sasl option in the install. The idea of this package is to give sysadmins an excellent smtp layer 7 filtering /proxy.

pf-disa · Apr 9, 2017, 10:55 AM

Hello,

@Bismark thanks for the info now everything works as desired! 8)

In /var/log/system.log remain the fatal error, even if all works…

Apr  9 12:26:55 pfSense0 pkg: postfix-3.1.4,1 deinstalled
Apr  9 12:26:56 pfSense0 pkg: pkg reinstalled: 1.10.1 -> 1.10.1 
Apr  9 12:27:00 pfSense0 pkg: postfix-sasl-3.1.4,1 installed
Apr  9 12:27:01 pfSense0 pkg: tiff-4.0.7_1 installed
...
Apr  9 12:28:45 pfSense0 php-fpm[47768]: /pkg_edit.php: Writing out configuration
Apr  9 12:28:47 pfSense0 php-fpm[47768]: /pkg_edit.php: Writing rc_file
Apr  9 12:28:48 pfSense0 php-fpm[47768]: /pkg_edit.php: Reloading/starting postfix
Apr  9 12:28:48 pfSense0 postfix/postfix-script[17508]: fatal: the Postfix mail system is not running
Apr  9 12:28:49 pfSense0 php-fpm[47768]: /pkg_edit.php: Postfix setup completed

Excellent work @Marcelloc, thank you!
Riccardo.

marcelloc · Apr 9, 2017, 4:28 PM

The fatal maybe related to a restart call to a process that is not ruining.

Good to see that this package is useful for community.

n3by · Apr 9, 2017, 4:16 PM

I just update my install on 2.3.2-p1 with your new script without any problems ( with sasl changes from Bismark ).

Thank you all.

marcelloc · Apr 9, 2017, 4:27 PM

These changes will be merged soon.

There are other improvements that will be merged soon.

Long life for community contributors :D

Bismarck · Apr 11, 2017, 8:05 AM

Noticed 3 little things,

1. postfix_postwhite.template isn't copied to /usr/local/etc/postwhite.conf

		//save file
		$postwhite_domains = preg_replace("/\s+/"," ",$domains);
		include("/usr/local/pkg/postfix_postwhite.template");
		file_put_contents($postwhite_conf , POSTFIX_LOCALBASE . "/etc/postwhite.conf" , LOCK_EX);

2. additional domains don't get added to postwhite.conf, maybe because of 1.

3. postwhite is looking for postwhite.conf in a different place anyway

# Read config file options
if [ -s /etc/postwhite.conf ] ; then
	printf "\nReading options from /etc/postwhite.conf...\n"
	source /etc/postwhite.conf
else
	printf "\nCan't find /etc/postwhite.conf. Exiting.\n\n"
	exit 1
fi

and I had a cronjob for this every morning, cant see where postwhite is executed here?

Thanks. :)

/edit

We need policyd-spf_time_limit = 3600 in main.cf if SPF Lookup is enabled or we get a lot of those lines in maillog:

Mar  5 14:03:26 mail postfix/spawn[57894]: warning: /usr/local/bin/policyd-spf: process id 58877: command time limit exceeded
Mar  5 14:32:21 mail postfix/spawn[57894]: warning: /usr/local/bin/policyd-spf: process id 60423: command time limit exceeded
Mar  5 15:13:00 mail postfix/spawn[62387]: warning: /usr/local/bin/policyd-spf: process id 62501: command time limit exceeded
Mar  5 15:30:07 mail postfix/spawn[62387]: warning: /usr/local/bin/policyd-spf: process id 63269: command time limit exceeded

Igor Filth · Apr 11, 2017, 11:57 AM

Great job! Package works fine.
Thank you very much guys!

I found a little mistake.
On "Access Lists" page, when i click on field "Sender" and "MIME" i'm redirected to http://www.postfix.org/pcre_table.5.html page :)

n3by · Apr 11, 2017, 1:04 PM

Another bug ??

I found that widget is not displaying correct status.

I have configured postfix as backup relay MX2 on Site2 - it receive external mail from WAN if primary mail server MX1 on Site1 is unreachable and it store and forward all email by VPN Site - to - Site.
I also receive all emails from Site2 - from all LANs - and forward them to primary email server on Site1 by VPN Site - to - Site.

But until it receive a email from WAN ( Update Sqlite was on 1h and now is on 10 min), widget is not display anything for that day; emails from LAN are not showed;
As you can see it is missing day 10 and day 11 was displayed only if I blocked primary mail server to receive emails from WAN and email was delivered to MX2 and then forwarded to MX1…

p.s.
I just found also this one in log probably related to Postwhite:

postfix/postscreen[34560]: error: open /usr/local/etc/postfix/postscreen_spf_whitelist.cidr: No such file or directory

Bismarck · Apr 11, 2017, 1:17 PM

@ecfx:

p.s.
I just found also this one in log probably related to Postwhite:
postfix/postscreen[34560]: error: open /usr/local/etc/postfix/postscreen_spf_whitelist.cidr: No such file or directory

Because you need to run /usr/local/bin/postwhite manually first, can't find any code atm in the package, which would trigger it automatically.

Just running Postfix as a Mail Proxy and my widget stats are very accurate.

n3by · Apr 11, 2017, 2:03 PM

no luck:

/usr/local/bin: ./postwhite
./postwhite: Permission denied.
...
/usr/local/bin: ls -la post*
-rw-r--r--  1 root  wheel  9789 Apr  9 14:59 postwhite
/usr/local/bin: chmod 755 postwhite
/usr/local/bin: ls -la post*
-rwxr-xr-x  1 root  wheel  9789 Apr  9 14:59 postwhite
/usr/local/bin: ./postwhite
./postwhite: Command not found.