Postfix - antispam and relay package

danjeman

This is just for matching the system hostname from the log…

There are 10 instances of that code at lines 253, 263, 275, 284, 295, 300, 305, 309, 319 and 357. Each one is parsing for a particular process log (postfix/smtpd, Mailscanner etc)

Replace

preg_match("/(\w+\s+\d+\s+[0-9,:]+) (\w+)

with

preg_match("/(\w+\s+\d+\s+[0-9,:]+) (\w+([\.-]?\w+))

There's probably a neater way to get the hostname but this does the job for my case at least.

n3by

Thank you now is clear, patched.

danjeman

Ok so widget crashed now due to memory allocation error… also not seeing any sent mails being logged but that could be configuration error (or will it not log those?)...


					Crash report begins.  Anonymous machine information:

amd64
10.3-RELEASE-p17
FreeBSD 10.3-RELEASE-p17 #10 6da131e75c7(RELENG_2_3_3): Wed Mar  8 14:24:33 CST 2017     root@ce23-amd64-builder:/builder/pfsense-233/tmp/obj/builder/pfsense-233/tmp/FreeBSD-src/sys/pfSense

Crash report details:

PHP Errors:
[19-Apr-2017 12:47:34 Europe/London] PHP Fatal error:  Allowed memory size of 262144000 bytes exhausted (tried to allocate 1262797654 bytes) in 

/usr/local/www/widgets/widgets/postfix.widget.php on line 146
[19-Apr-2017 12:47:34 Europe/London] PHP Stack trace:
[19-Apr-2017 12:47:34 Europe/London] PHP   1\. {main}() /usr/local/www/index.php:0
[19-Apr-2017 12:47:34 Europe/London] PHP   2\. include_once() /usr/local/www/index.php:438
[19-Apr-2017 12:48:41 Europe/London] PHP Fatal error:  Allowed memory size of 262144000 bytes exhausted (tried to allocate 1262797654 bytes) in /usr/local/www/widgets/widgets/postfix.widget.php on line 146
[19-Apr-2017 12:48:41 Europe/London] PHP Stack trace:
[19-Apr-2017 12:48:41 Europe/London] PHP   1\. {main}() /usr/local/www/index.php:0
[19-Apr-2017 12:48:41 Europe/London] PHP   2\. include_once() /usr/local/www/index.php:438

Filename: /var/crash/minfree
2048

marcelloc

What is the max file size you configured for widget?

danjeman

Left it blank so assuming default of 100mb? Will specify and test again.

Currently files are less than 1M each and only 4 of them…


-rw-r--r--  1 root     postfix   200K Apr 18 17:05 2017-04-15.db
-rw-r--r--  1 root     postfix   184K Apr 18 16:31 2017-04-17.db
-rw-r--r--  1 root     postfix   516K Apr 19 14:18 2017-04-18.db
-rw-r--r--  1 root     postfix   976K Apr 19 14:20 2017-04-19.db

EDIT: Also noticed widget only has one line with yesterday's date although the totals and reject and spam stats are increasing…

danjeman

Just noticed on the Access Lists page I couldn't edit sender, MIME or body filters and spotted the 'a' tag was still open… Line 109 of /usr/local/pkg/postfix_acl.xml is missing the closing tag

<description>PCRE filters [that are applied to initial message helo info. Hint:
[/code]

Should be
[code]<description>PCRE filters</description>](http://www.postfix.org/pcre_table.5.html) [that are applied to initial message helo info.](http://www.postfix.org/pcre_table.5.html) Hint:
[/code][/code]</description>

marcelloc

@danjeman:

Just noticed on the Access Lists page I couldn't edit sender, MIME or body filters and spotted the 'a' tag was still open… Line 109 of /usr/local/pkg/postfix_acl.xml is missing the closing tag

Fixed. Thanks for the feedback

marcelloc

@danjeman:

Ok so widget crashed now due to memory allocation error…

I've updated the widget file to allow more php memory use.

marcelloc

@danjeman:

Was not getting any sqllite db's from the logs despite all settings being correct then found an old post referencing an issue if you have other characters in the hostname (- for instance

I've changed the preg_match from (\w+) to (\S+)

https://github.com/marcelloc/Unofficial-pfSense-packages/commit/0893d4a2e523688ccd56d419ff1d7c8be5c51cd2

danjeman

@marcelloc:

I've changed the preg_match from (\w+) to (\S+)

https://github.com/marcelloc/Unofficial-pfSense-packages/commit/0893d4a2e523688ccd56d419ff1d7c8be5c51cd2

Re- ran the installer and all looks good so far, thanks! :)

danjeman

Noticed that the options for 'SPF lookup HELO' and 'SPF lookup Mail From' are missing 'Fail' option..

Looks like option Null for 'HELO' has value Fail set as well as option Fail so 'Fail' never displays (also would mean selecting Null didn't do what you expect).

For 'Mail From' option Null shouldn't be an option (according to the commented policyd-spf.conf) so removing this will correct the available options.

marcelloc

@danjeman:

Noticed that the options for 'SPF lookup HELO' and 'SPF lookup Mail From' are missing 'Fail' option..

Looks like option Null for 'HELO' has value Fail set as well as option Fail so 'Fail' never displays (also would mean selecting Null didn't do what you expect).

For 'Mail From' option Null shouldn't be an option (according to the commented policyd-spf.conf) so removing this will correct the available options.

thanks for the pull request on github :)

n3by

Widget start to display strange data…

![2017-04-22 20.53.41.jpg](/public/imported_attachments/1/2017-04-22 20.53.41.jpg)
![2017-04-22 20.53.41.jpg_thumb](/public/imported_attachments/1/2017-04-22 20.53.41.jpg_thumb)
![2017-04-22 18.48.31.jpg](/public/imported_attachments/1/2017-04-22 18.48.31.jpg)
![2017-04-22 18.48.31.jpg_thumb](/public/imported_attachments/1/2017-04-22 18.48.31.jpg_thumb)

marcelloc

Are you waiting first refresh?

Bismarck

Looks good so far here. :)

marcelloc postfwd pkg is missing in the setup script? I've installed it manually and set postfwd.cf and the port to 10045 in the rc file, now its running.

![2017-04-22 19.41.54.png](/public/imported_attachments/1/2017-04-22 19.41.54.png)
![2017-04-22 19.41.54.png_thumb](/public/imported_attachments/1/2017-04-22 19.41.54.png_thumb)

marcelloc

@Bismarck:

Looks good so far here. :)

marcelloc postfwd pkg is missing in the setup script? I

I'll check but I remember including it on install script. Maybe I forgot to push it to GitHub repo

marcelloc

@marcelloc:

I'll check but I remember including it on install script. Maybe I forgot to push it to GitHub repo

Gui creates the file when there are rules on it. what's the behavior on your box?

Bismarck

@marcelloc:

@marcelloc:

I'll check but I remember including it on install script. Maybe I forgot to push it to GitHub repo

Gui creates the file when there are rules on it. what's the behavior on your box?

I had to manually install postfix-postfwd-1.35_1 to make it run and listen on 127.0.0.1:10045, otherwise postfix will give warnings about it.

Apr 21 16:40:16 zonk postfix/smtpd[4740]: warning: connect to 127.0.0.1:10045: Operation timed out
Apr 21 16:40:16 zonk postfix/smtpd[4740]: warning: problem talking to server 127.0.0.1:10045: Operation timed out

marcelloc

@Bismarck:

I had to manually install postfix-postfwd-1.35_1 to make it run and listen on 127.0.0.1:10045, otherwise postfix will give warnings about it.

Fixed the install script to include the pkg add, Thanks again :)

I'll push it to repo soon

n3by

Hi,

Is this re-instalation completed successfully as I also see in /root ?

drwxr-xr-x   5 root  wheel        512 Apr 25 17:07 spf-tools-master

/root: sh ./install_postfix_23.sh
Message from syslogd@fwpl at Apr 25 17:05:19 ...
fwpl php-fpm[61287]: /index.php: Successful loginsh ./install_postfix_23.sh                                                                 fetching  /usr/local/bin/adexport.pl from github
fetching  /usr/local/pkg/postfix.inc from github
fetching  /usr/local/pkg/postfix.xml from github
fetching  /usr/local/pkg/postfix_acl.xml from github
fetching  /usr/local/pkg/postfix_antispam.xml from github
fetching  /usr/local/pkg/postfix_domains.xml from github
fetching  /usr/local/pkg/postfix_recipients.xml from github
fetching  /usr/local/pkg/postfix_sync.xml from github
fetching  /usr/local/share/pfSense-pkg-postfix/info.xml from github
fetching  /usr/local/www/postfix.php from github
fetching  /usr/local/www/postfix_about.php from github
fetching  /usr/local/www/postfix_queue.php from github
fetching  /usr/local/www/postfix_recipients.php from github
fetching  /usr/local/www/postfix_search.php from github
fetching  /usr/local/www/postfix_view_config.php from github
fetching  /usr/local/www/shortcuts/pkg_postfix.inc from github
fetching  /usr/local/www/widgets/widgets/postfix.widget.php from github
fetching  /usr/local/pkg/postfix_dkim.inc from github
fetching  /usr/local/www/vendor/datatable/se-1.2.0.zip from github
fetching  /usr/local/www/vendor/datatable/css/jquery.dataTables.min.css from github
fetching  /usr/local/www/vendor/datatable/js/jquery.dataTables.min.js from github
fetching  /usr/local/www/postfix.sql.php from github
fetching  /usr/local/bin/postwhite from github
fetching  /usr/local/pkg/postfix_postwhite.template from github
Updating FreeBSD repository catalogue...
Fetching meta.txz: 100%    944 B   0.9kB/s    00:01    
Fetching packagesite.txz: 100%    6 MiB   3.0MB/s    00:02    
Processing entries: 100%
FreeBSD repository update completed. 26278 packages processed.
Updating pfSense-core repository catalogue...
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
Fetching meta.txz: 100%    944 B   0.9kB/s    00:01    
Child process pid=77716 terminated abnormally: Segmentation fault
fetch: https://github.com/jsarenik/spf-tools/archive/master.zip: size of remote file is not known
master.zip                                              49 kB  195 kBps 00m01s
Archive:  master.zip
d spf-tools-master
replace spf-tools-master/.gitignore? [y]es, [n]o, [A]ll, [N]one, [r]ename: y
 extracting: spf-tools-master/.gitignore  
replace spf-tools-master/.simplecov? [y]es, [n]o, [A]ll, [N]one, [r]ename: A
 extracting: spf-tools-master/.simplecov  
 extracting: spf-tools-master/.travis.yml  
 extracting: spf-tools-master/AUTHORS  
 extracting: spf-tools-master/LICENSE  
 extracting: spf-tools-master/README.md  
 extracting: spf-tools-master/circle.yml  
 extracting: spf-tools-master/cloudflare.sh  
 extracting: spf-tools-master/compare.sh  
 extracting: spf-tools-master/despf.sh  
 extracting: spf-tools-master/genspfzone.sh  
d spf-tools-master/include
 extracting: spf-tools-master/include/despf.inc.sh  
 extracting: spf-tools-master/include/global.inc.sh  
 extracting: spf-tools-master/include/isincidrange.sh  
 extracting: spf-tools-master/iprange.sh  
d spf-tools-master/misc
 extracting: spf-tools-master/misc/ci-runtest.sh  
 extracting: spf-tools-master/misc/ci-setup.sh  
 extracting: spf-tools-master/misc/tmpl  
 extracting: spf-tools-master/mkblocks.sh  
 extracting: spf-tools-master/mkzoneent.sh  
 extracting: spf-tools-master/normalize.sh  
 extracting: spf-tools-master/route53.sh  
 extracting: spf-tools-master/runspftools.sh  
 extracting: spf-tools-master/shippable.yml  
 extracting: spf-tools-master/simplify.sh  
d spf-tools-master/tests
d spf-tools-master/tests/a24
 extracting: spf-tools-master/tests/a24/cmd  
 extracting: spf-tools-master/tests/a24/in  
 extracting: spf-tools-master/tests/a24/out  
d spf-tools-master/tests/brokendns
 extracting: spf-tools-master/tests/brokendns/cmd  
 extracting: spf-tools-master/tests/brokendns/in  
 extracting: spf-tools-master/tests/brokendns/out  
d spf-tools-master/tests/cname
 extracting: spf-tools-master/tests/cname/cmd  
 extracting: spf-tools-master/tests/cname/in  
 extracting: spf-tools-master/tests/cname/out  
d spf-tools-master/tests/despf
 extracting: spf-tools-master/tests/despf/cmd  
 extracting: spf-tools-master/tests/despf/in  
 extracting: spf-tools-master/tests/despf/out  
d spf-tools-master/tests/despf_chain
 extracting: spf-tools-master/tests/despf_chain/cmd  
 extracting: spf-tools-master/tests/despf_chain/in  
 extracting: spf-tools-master/tests/despf_chain/out  
d spf-tools-master/tests/despf_help
 extracting: spf-tools-master/tests/despf_help/cmd  
 extracting: spf-tools-master/tests/despf_help/in  
 extracting: spf-tools-master/tests/despf_help/out  
d spf-tools-master/tests/despf_qualifier
 extracting: spf-tools-master/tests/despf_qualifier/cmd  
 extracting: spf-tools-master/tests/despf_qualifier/in  
 extracting: spf-tools-master/tests/despf_qualifier/out  
d spf-tools-master/tests/despf_qualifier2
 extracting: spf-tools-master/tests/despf_qualifier2/cmd  
 extracting: spf-tools-master/tests/despf_qualifier2/in  
 extracting: spf-tools-master/tests/despf_qualifier2/out  
d spf-tools-master/tests/despf_skip
 extracting: spf-tools-master/tests/despf_skip/cmd  
 extracting: spf-tools-master/tests/despf_skip/in  
 extracting: spf-tools-master/tests/despf_skip/out  
d spf-tools-master/tests/despf_skip_t
 extracting: spf-tools-master/tests/despf_skip_t/cmd  
 extracting: spf-tools-master/tests/despf_skip_t/in  
 extracting: spf-tools-master/tests/despf_skip_t/out  
d spf-tools-master/tests/despf_torn
 extracting: spf-tools-master/tests/despf_torn/cmd  
 extracting: spf-tools-master/tests/despf_torn/in  
 extracting: spf-tools-master/tests/despf_torn/out  
d spf-tools-master/tests/despf_upper_case
 extracting: spf-tools-master/tests/despf_upper_case/cmd  
 extracting: spf-tools-master/tests/despf_upper_case/in  
 extracting: spf-tools-master/tests/despf_upper_case/out  
d spf-tools-master/tests/fix_32
 extracting: spf-tools-master/tests/fix_32/cmd  
 extracting: spf-tools-master/tests/fix_32/in  
 extracting: spf-tools-master/tests/fix_32/out  
d spf-tools-master/tests/mkblocks-help
 extracting: spf-tools-master/tests/mkblocks-help/cmd  
 extracting: spf-tools-master/tests/mkblocks-help/in  
 extracting: spf-tools-master/tests/mkblocks-help/out  
d spf-tools-master/tests/mkblocks-start
 extracting: spf-tools-master/tests/mkblocks-start/cmd  
 extracting: spf-tools-master/tests/mkblocks-start/in  
 extracting: spf-tools-master/tests/mkblocks-start/out  
d spf-tools-master/tests/mkblocks
 extracting: spf-tools-master/tests/mkblocks/cmd  
 extracting: spf-tools-master/tests/mkblocks/in  
 extracting: spf-tools-master/tests/mkblocks/out  
d spf-tools-master/tests/mx20
 extracting: spf-tools-master/tests/mx20/cmd  
 extracting: spf-tools-master/tests/mx20/in  
 extracting: spf-tools-master/tests/mx20/out  
d spf-tools-master/tests/mx20_upper_case
 extracting: spf-tools-master/tests/mx20_upper_case/cmd  
 extracting: spf-tools-master/tests/mx20_upper_case/in  
 extracting: spf-tools-master/tests/mx20_upper_case/out  
d spf-tools-master/tests/norm_ignore
 extracting: spf-tools-master/tests/norm_ignore/cmd  
 extracting: spf-tools-master/tests/norm_ignore/in  
 extracting: spf-tools-master/tests/norm_ignore/out  
d spf-tools-master/tests/normalize
 extracting: spf-tools-master/tests/normalize/cmd  
 extracting: spf-tools-master/tests/normalize/in  
 extracting: spf-tools-master/tests/normalize/out  
d spf-tools-master/tests/normalize_empty
 extracting: spf-tools-master/tests/normalize_empty/cmd  
 extracting: spf-tools-master/tests/normalize_empty/in  
 extracting: spf-tools-master/tests/normalize_empty/out  
d spf-tools-master/tests/nospf
 extracting: spf-tools-master/tests/nospf/cmd  
 extracting: spf-tools-master/tests/nospf/in  
 extracting: spf-tools-master/tests/nospf/out  
d spf-tools-master/tests/redirect
 extracting: spf-tools-master/tests/redirect/cmd  
 extracting: spf-tools-master/tests/redirect/in  
unzip: skipping non-regular entry 'spf-tools-master/tests/redirect/out'
d spf-tools-master/tests/simplify
 extracting: spf-tools-master/tests/simplify/cmd  
 extracting: spf-tools-master/tests/simplify/in  
 extracting: spf-tools-master/tests/simplify/out  
 extracting: spf-tools-master/tests/test-shell.sh  
 extracting: spf-tools-master/tests/test-subdirs.sh  
 extracting: spf-tools-master/tests/test-unit.sh  
 extracting: spf-tools-master/xsel.sh  
mv: rename spf-tools-master to /usr/local/bin/spf-tools/spf-tools-master: Directory not empty

edit:
deleted /usr/local/bin/spf-tools/spf-tools-master
and retry the install and now it looks ok