Don't receive email notifications

Gertjan

@Sessa45 : I had a closer look.

The button "Send now" from the package Status => Email Reports => Edit Reports doesn't use the same scripts/code used by other notices.
It's uses the /etc/inc/phpmailer/* scripts.

Still it should work fine - at least, for me, it does.

Sessa45

@Gertjan said in Don't receive email notifications:

It's uses the /etc/inc/phpmailer/* scripts.

In which file do I have to adapt which code?
I am still confused that the test e-mail arrives but the actual reports are not.

Gertjan

@Sessa45 said in Don't receive email notifications:

In which file do I have to adapt which code?

Yes and no ;)

When I installed the "email report package" it worked for me right away.
And guess what : you and I use the same code !! ;)

But, as usuall, you and I have not the same settings ^^ (so change your settings).

The fact is : the recent pfSense Notification code uses new code "PHP Pearl" stuff.
The Email report package still uses the old phpmailer.

The phpmailer code is here : /etc/inc/phpmailer
There are debug facilities present in the main file class.phpmailer.php
Set it to

public $SMTPDebug = 3;

and see what happens.

Possibilities are : you see nothing (system logs / in the GUI screen)
Or : The phpmailer code isn't even called (the problem lies above).
Something else.

There is a reason why phpmailer was replaced in pfSense code although phpmailer is still present (because packages might use it, like the Email Report package).

You could also try to change the System => Advanced => Notifications for a gmail email address - I know that that one will work - it does for me.

Sessa45

@Gertjan said in Don't receive email notifications:

Possibilities are : you see nothing (system logs / in the GUI screen)

That is what happened ... :)
I have edited the debug level to 3 but unfortunately, i see nothing.

@Gertjan said in Don't receive email notifications:

You could also try to change the System => Advanced => Notifications for a gmail email address - I know that that one will work - it does for me.

Alright. So i just create an gmail e-mail address and use this? With standard port 25?

Gertjan

Port 25 ????

Never ever a mail client should use port 25.
That was banished last century.
The "25" number should not be used in any mail capable device these days.
There ar no exceptions anymore - not on planet earth.

True : only some old-school ISP's are still exposing a "port 25 realy mail server" for their clients.
Because : your are on their internal network, they know who you are.
You shouldn't use this port.
Use the 587 or better 465 equivalent - I'm sure they propose that also.

GMail can only be used for sending mail on port 465.
You have to instruct GMail up front (as any other public mail provider these days) that it should accept incoming (== smtp) mails using your account.

How to set up gmail for smtp relaying : start up your memory extension and type gmail smtp 465

( actually, they were saying the truth in The Matrix - knowledge is one phone number or click away ^^ )

Btw : if you were really using 25 before, it should be normal that phpmailer bails out .... and thus the mail package .

Gertjan

@Gertjan said in Don't receive email notifications:

Set it to
public $SMTPDebug = 3;

and see what happens.

and .... set

     public $Debugoutput = 'html';

to

    public $Debugoutput = 'error_log';

Now, when sending the test report, go back to the main Dashboard GUI page.

You will find a "PHP error" - download it into a text editor and you see the entire mail-send process with all the details.

Do not forget to set

public $SMTPDebug = 3;

back to the default value when your done.

Sessa45

@Gertjan

Thank you for this answer!

I have followed your instructions. This is the content from the PHP_errors.log file:

[02-Sep-2019 13:43:42 Europe/Berlin] CLIENT -> SERVER: STARTTLS

[02-Sep-2019 13:43:42 Europe/Berlin] SERVER -> CLIENT: 220 2.0.0 Ready to start TLS

[02-Sep-2019 13:43:42 Europe/Berlin] SMTP Error: Could not connect to SMTP host.
[02-Sep-2019 13:43:42 Europe/Berlin] CLIENT -> SERVER: QUIT

[02-Sep-2019 13:43:42 Europe/Berlin] SERVER -> CLIENT: M
[02-Sep-2019 13:43:42 Europe/Berlin] SMTP ERROR: QUIT command failed: M
[02-Sep-2019 13:43:42 Europe/Berlin] Connection: closed

What does this mean? Sure: "SMTP Error: Could not connect to SMTP host." But how exactly can i fix this?

johnpoz

what is the server you have setup for smtp - your saying the test notification works? But mail reports don't?

I tried to duplicate your issue, but working here - just installed the email reports package.. had it send me some logs (dhcp) and got it as soon as I hit send now

Gertjan

This :
@Sessa45 said in Don't receive email notifications:

[02-Sep-2019 13:43:42 Europe/Berlin] CLIENT -> SERVER: STARTTLS

Before, when pfSense was using phpmailer itself, there was a setting in the GUI that controlled the STARTSSL usage.
Now, that is gone because the new pearl PHP mailer is smart enough to find that out for itself.

But : as said, the Email report package usages phpmailer. And phpmailer is probably missing that option now. That might explain why it has troubles to connect to 'some' smtp servers. [ edit : read on : this is not true - a STARTSSL capable mail server on port 587 works for me ]

I use the https version of mailing : smtps from point to point - to make things short : use port 465. That's what I do.

Here it is (I cut out the mail body itself ) : https://pastebin.com/A42BhsB7

I'll set up my pfSense to use the older submission (smtp over port 587) - I'm using my own mail server so I can see on both sides what happens.
These are my settings :

Your settings are any different ?

Mail server log : https://pastebin.com/cbR2h3EY

The PHP_eroor log from pfSense / the Email report package :

And of course I received also the test mail from the Mail report package.

Maybe the mail server you are using is not 100 % RFC xxxx comliant ?
I can :
send mail using 587 (my own mail servers, my ISP mail server)
send mail using 465 (gmail and my own servers)
just fine.

This is my logs - using port "587" on my mail server :

....
[02-Sep-2019 14:20:42 Europe/Paris] Connection: opened
[02-Sep-2019 14:20:42 Europe/Paris] SERVER -> CLIENT: 220 mail.----------.me ESMTP Postfix

[02-Sep-2019 14:20:42 Europe/Paris] CLIENT -> SERVER: EHLO pfsense.brit-hotel-fumel.nxt

[02-Sep-2019 14:20:42 Europe/Paris] SERVER -> CLIENT: 250-mail.----------.me
250-PIPELINING
250-SIZE 31457280
250-VRFY
250-ETRN
250-STARTTLS
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN

[02-Sep-2019 14:20:42 Europe/Paris] CLIENT -> SERVER: STARTTLS

[02-Sep-2019 14:20:42 Europe/Paris] SERVER -> CLIENT: 220 2.0.0 Ready to start TLS

[02-Sep-2019 14:20:42 Europe/Paris] CLIENT -> SERVER: EHLO pfsense.brit-hotel-fumel.nxt

[02-Sep-2019 14:20:42 Europe/Paris] SERVER -> CLIENT: 250-mail.----------.me
250-PIPELINING
250-SIZE 31457280
250-VRFY
250-ETRN
250-AUTH PLAIN LOGIN
250-AUTH=PLAIN LOGIN
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN

[02-Sep-2019 14:20:42 Europe/Paris] CLIENT -> SERVER: AUTH LOGIN

[02-Sep-2019 14:20:42 Europe/Paris] SERVER -> CLIENT: 334 VXNlsdferscm5hbWU6

[02-Sep-2019 14:20:42 Europe/Paris] CLIENT -> SERVER: Z2ViydffGfphbqkBrqcmqsd9lYi5tZQ==

[02-Sep-2019 14:20:42 Europe/Paris] SERVER -> CLIENT: 334 UGFzc3eddvctmQ6

[02-Sep-2019 14:20:42 Europe/Paris] CLIENT -> SERVER: aVbHRzZWNhiNDc=

[02-Sep-2019 14:20:42 Europe/Paris] SERVER -> CLIENT: 235 2.7.0 Authentication successful

[02-Sep-2019 14:20:42 Europe/Paris] CLIENT -> SERVER: MAIL FROM:<pfsense@brit-hotel-fumel.nxt>

[02-Sep-2019 14:20:42 Europe/Paris] SERVER -> CLIENT: 250 2.1.0 Ok
........

After your (and mine )

SERVER -> CLIENT: 220 2.0.0 Ready to start TLS

You did cut out important server - client mail communication - maybe hiding important issues.
something is pulling the plug out of the connection. Probably the mail server in front of you.

My server doesn't do that - neither the mail server from gmail.com - and others.

gmail worked for you ?

What you see in my logs is:

First, over clear pipe communication - and the option that STARTSSL is present as a server option.
Then the mail client uses the STARTSSL command - now the connection becomes "SSL".
Again the options are shown - without STARTSSL this time, because it's already active.
Authentication happens,
The mail is sent.

Sessa45

@Gertjan said in Don't receive email notifications:

You could also try to change the System => Advanced => Notifications for a gmail email address - I know that that one will work - it does for me.

Alright. I have created an gmail e-mail account and tried to send the test e-mail after i have set the following configuration:

I have followed the instruction which are given on the following site which you have given me:

@Gertjan said in Don't receive email notifications:

How to set up gmail for smtp relaying : start up your memory extension and type gmail smtp 465

SMTP:

Postausgangsserver: smtp.googlemail.com

Port: 465 (SSL) oder 587 (TLS/STARTTLS)

When i use port 465, i receive the following message when i try tp send the test mail:
Could not send the message to test.pfsense2019@gmail.com -- Error: Failed to connect to smtp.googlemail.com:465 [SMTP: Invalid response code received from server (code: -1, response: )]

When i use port 587:
Could not send the message to test.pfsense2019@gmail.com -- Error: Failed to set sender: admin@pfsense.de [SMTP: Invalid response code received from server (code: 530, response: 5.7.0 Must issue a STARTTLS command first. m7sm39593166wmi.18 - gsmtp)]

What do i wrong?

Gertjan

@Sessa45 said in Don't receive email notifications:

What do i wrong?

According to the English doc the URL is smtp.gmail.com

See point 2 : I don't understand why you didn't enter a login mail and password .....
These days, you couldn't and you shouldn't send and receive mails without identification/authentication.

Sessa45

@Gertjan said in Don't receive email notifications:

@Sessa45 said in Don't receive email notifications:

What do i wrong?

According to the English doc the URL is smtp.gmail.com

See point 2 : I don't understand why you didn't enter a login mail and password .....
These days, you couldn't and you shouldn't send and receive mails without identification/authentication.

Even if i use this configuration i received an error message when i try to send a test mail:

Could not send the message to test.pfsense2019@gmail.com -- Error: Failed to connect to smtp.gmail.com:465 [SMTP: Invalid response code received from server (code: -1, response: )]

Gertjan

When you use your gmail address and password, pfSense should tell you this (using a red background) :

Could not send the message to gertjan@kroeb.me -- Error: LOGIN authentication failure [SMTP: Invalid response code received from server (code: 534, response: 5.7.9 Application-specific password required. Learn more at 5.7.9 https://support.google.com/mail/?p=InvalidSecondFactor v8sm35044038wra.79 - gsmtp)]

It's URL reading time again.

You have to generate a special password for your pfSense Notification setup.
One done :

Sessa45

@Gertjan said in Don't receive email notifications:

It's URL reading time again.

Done. I set an App password in my gmail account and take this in my pfSense configuration. Unfortunately, i received the same error.

Gertjan

Show these again :

and compare with mine ....

Btw :

means : your pfsense can't connect to smtp.gmail.com (port 465). You can't connect - which is different as being refused for bas user or password.
Some one is blocking you somewhere in between - upstream.

johnpoz

Do a simple check to that port...

If pfsense can resolve and talk to that port you should get back successful test

You can test ipv6 as well

You could also do a packet capture while your testing or trying to send the mail to what is going on

Here is sniff while testing port - you can see it opens syn, and then closes the connection with fin

Sessa45

Hi Guys!

Finally, I received the test e-mail and the report emails with Gmail as the e-mail server.

I've tried to understand how e-mail communication works, but I do not understand (maybe because I'm not the e-mail specialist on earth ^^), what differences there are in the processes you already have here have explained ...

Not using port 25 = standard port and not TLS encrypted
test mail is not report = both uses different code / language
auth with e-mail and pw = to increase the security

Am i right with this?
So the next step is to configure the mail server on port 465 / TLS and configure auth with name and pw?

Thanks and best regards!

johnpoz

Correct, not many home isp allows 25 out, if they do allow it - its only to their smtp servers on their network.

To send to gmail use 465, and setup an app password.

Mr_AJ

Good info, I'll have to try this tonight.