external smtp ip appears as local

vladanpopovic

I have a mail server behind a pfsense firewall and set it to verify the spf record of the sending server but when I send a test from gmail, the email gets rejected because it appears as if gmail's ip address is 192.168.2.1 which it obviously is not.

Not sure where to start looking for a solution. here's the rejection message I get:

host mail.be-o.com[58.177.253.230] said: 550 5.7.1
dusanlazarevic@be-o.com: Recipient address rejected: Message rejected due
to: domain owner discourages use of this host. Please see
http://www.openspf.net/Why?s=mfrom;id=vladanpopovic389@gmail.com;ip=192.168.2.1;r=dusanlazarevic@be-o.com
(in reply to RCPT TO command

Gertjan

Hi,

From your mail server, when you resolve gmail.com using ping, host and dig, do you really receive 192.168.2.1 as a host ?

The SPF of the MX of be-o.com is :

root@ns311465:~# dig be-o.com TXT +short
"be-o.com"
"v=spf1 ip4:58.177.253.230/32"

This :

@vladanpopovic said in [external smtp ip appears as local](/post/862949):
> host mail.be-o.com[58.177.253.230] said: 550 5.7.1
> dusanlazarevic@be-o.com: Recipient address rejected: Message rejected due
> to: domain owner discourages use of this host. Please see
> http://www.openspf.net/Why?s=mfrom;id=vladanpopoxxxxxx@gmail.com;ip=192.168.2.1;r=dusanlazarevic@be-o.com
> (in reply to RCPT TO command

is what your server (log) said to you ?

Btw : woooow, openspf.net and org are down for me ...

vladanpopovic

yes that's the correct spf for our server - what I don't get is why the email from gmail is rejected and why does the ip 192.168.2.1 appear in http://www.openspf.net/Why?s=mfrom;id=vladanpopoxxxxxx@gmail.com;ip=192.168.2.1;r=dusanxxxxxxx@be-o.com

yes it seems that openspf is down

Gertjan

When a mail arrives at your mail server (postfix ?) you instructed to have the mail parsed by a SPF 'plugin'.
The SPF checks if the domain of the sender - declared sender mail or "from:" mail has a SPF record, which is acyually a TXT record these days, with a special format.

This what it does when a mail from gmail comes in - your dusanxxxxxxx@be-o.com :

dig gmail.com TXT +short
"globalsign-smime-dv=CDYX+XFHUw2wml6/Gb8+59BsH31KzUr6c1l2BPvqKX8="
"v=spf1 redirect=_spf.google.com"

There is a redirect, so :

dig _spf.google.com  TXT +short
"v=spf1 include:_netblocks.google.com include:_netblocks2.google.com include:_netblocks3.google.com ~all"

These "netblocks" list all the IP addresses gmail is using, example :

dig _netblocks.google.com TXT +short
"v=spf1 ip4:35.190.247.0/24 ip4:64.233.160.0/19 ip4:66.102.0.0/20 ip4:66.249.80.0/20 ip4:72.14.192.0/18 ip4:74.125.0.0/16 ip4:108.177.8.0/21 ip4:173.194.0.0/16 ip4:209.85.128.0/17 ip4:216.58.192.0/19 ip4:216.239.32.0/19 ~all"

Now, the actual check is easy :
The IP4 or IPv6 address gmail was using when it send your mail must be in one of these blocks.

Or, your SPF check comes back with 192.168.2.1 : did you email come from 192.168.2.1 ? Who is 192.168.2.1 ?
Can the system where your mail server and SPF check is running, do a good DNS resolution (ie : do the dig's) ?

What is in the headers of the mails ?

Btw : this is not an pfSense issue, more a "mail issue". But, don't worry, I'm using postfix with SPF, DKIM, DMARC (they are all needed if you want to send mail to gmail / hotmail / yahoo / etc ....) so we'll figure it out.

Normally, your SPF addon or plugin comes with a manual and is using config files. So it most probable you can activate the SPF debug facilities to see what is happening.
When setting up a mail server that actually works, you'll be looking at log files for the rest of your live, testing all the debug methods - using mail accounts from pretty any mail supplier on the planet.

Is this a home setup ? Else where ? My mail server runs on a Debian on a dedicated server on the net. There isn't even a firewall on this server - just fail2ban to block bots.

edit :
A mail from my gmail to a mail I host myself( gertjan@my-domaine.tld ) :

Sep  5 10:40:34 ns311465 postfix/smtpd[31732]: connect from mail-io1-xd30.google.com[2607:f8b0:4864:20::d30]
Sep  5 10:40:34 ns311465 postfix/smtpd[31732]: Trusted TLS connection established from mail-io1-xd30.google.com[2607:f8b0:4864:20::d30]: TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)
Sep  5 10:40:35 ns311465 postfix/policy-spf[31734]: Policy action=PREPEND Received-SPF: pass (gmail.com ... _spf.google.com: Sender is authorized to use 'my-mail@gmail.com' in 'mfrom' identity (mechanism 'include:_netblocks2.google.com' matched)) receiver=ns311465.ip-188-165-201.eu; identity=mailfrom; envelope-from="my-mail@gmail.com"; helo=mail-io1-xd30.google.com; client-ip="2607:f8b0:4864:20::d30"
Sep  5 10:40:35 ns311465 postfix/smtpd[31732]: A4F7563E0373: client=mail-io1-xd30.google.com[2607:f8b0:4864:20::d30]
Sep  5 10:40:35 ns311465 postfix/cleanup[31737]: A4F7563E0373: message-id=<CAD_ygHUFAgSs4wv_zut_9zR0vTmeNxKssh+uE-eH_VU1sEaZFg@mail.gmail.com>
Sep  5 10:40:35 ns311465 opendmarc[1243]: A4F7563E0373: gmail.com pass
Sep  5 10:40:35 ns311465 opendkim[1229]: A4F7563E0373: DKIM verification successful
Sep  5 10:40:36 ns311465 postfix/qmgr[24576]: A4F7563E0373: from=<my-mail@gmail.com>, size=2855, nrcpt=1 (queue active)
Sep  5 10:40:36 ns311465 postfix/smtpd[31732]: disconnect from mail-io1-xd30.google.com[2607:f8b0:4864:20::d30]
Sep  5 10:40:36 ns311465 return-from-amavis/smtpd[31742]: B334963E1599: client=localhost.localdomain[127.0.0.1]
Sep  5 10:40:36 ns311465 postfix/cleanup[31737]: B334963E1599: message-id=<CAD_ygHUFAgSs4wv_zut_9zR0vTmeNxKssh+uE-eH_VU1sEaZFg@mail.gmail.com>
Sep  5 10:40:36 ns311465 postfix/qmgr[24576]: B334963E1599: from=<my-mail@gmail.com>, size=3940, nrcpt=1 (queue active)
Sep  5 10:40:36 ns311465 postfix/smtp[31738]: A4F7563E0373: to=<gertjan@my-domaine.tld>, relay=localhost[127.0.0.1]:10024, delay=1.9, delays=1.1/0.02/0/0.77, dsn=2.0.0, status=sent (250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as B334963E1599)
Sep  5 10:40:36 ns311465 postfix/qmgr[24576]: A4F7563E0373: removed
Sep  5 10:40:36 ns311465 postfix/virtual[31743]: B334963E1599: to=<gertjan@my-domaine.tld>, relay=virtual, delay=0.14, delays=0.05/0.01/0/0.08, dsn=2.0.0, status=sent (delivered to maildir)
Sep  5 10:40:36 ns311465 postfix/qmgr[24576]: B334963E1599: removed

As you can see, the third line validates that the source is actually 'gmail'.
And opendkim and opendmarc are fine also.

kiokoman

@Gertjan the problem here is that the external ip of gmail is rewritten to 192.168.2.1 and presented to the mail server that reject it obviusly

so the point here is what option/package do that?
i have some postfix servers myself behind pfsense but never encountered such behavior
some misconfigured proxy maybe or you have something wrong in advanced outbound NAT
rule that NAT's from the WAN to your LAN could rewrite the source

vladanpopovic

192.168.2.1 is the gateway ip on the pfsense for the dmz.
the gateway ip for the lan is 192.168.1.1
the email server is indeed postfix and I can send and receive emails (when no spf checking) so I don't think there is any dns problem.

here are the headers of a successful delivery without spf checking:
X-Synology-Spam-Flag: ⁨no⁩
X-Received: ⁨by 2002:a17:90a:c70e:: with SMTP id o14mr2999852pjt.56.1567677993037; Thu, 05 Sep 2019 03:06:33 -0700 (PDT)⁩
X-Gm-Message-State: ⁨APjAAAV5doh0JoEKLeIEuMZNN0RjSmvwLRRn/L02wV8xBydPAw/gr7q8 WpZ63Of3mYsfRpuXVd39jB+UzvwR7os=⁩
Return-Path: ⁨vladanpopovic389@gmail.com⁩
Return-Path: ⁨vladanpopovic389@gmail.com⁩
X-Synology-Spam-Status: ⁨score=-1.5792971014463, required 5, autolearn=ham, RCVD_COUNT_THREE 0, TO_DN_ALL 0, PREVIOUSLY_DELIVERED 0, MIME_GOOD -0.1, DMARC_POLICY_ALLOW -0.25, RCPT_COUNT_ONE 0, FROM_EQ_ENVFROM 0, FREEMAIL_ENVFROM_END_DIGIT 0.25, MX_GOOD -0.01, SUBJECT_ENDS_SPACES 0.5, R_DKIM_ALLOW -0.2, RCVD_IN_DNSWL_NONE 0, __HDRS_LCASE_KNOWN 0, FROM_HAS_DN 0, FREEMAIL_ENVFROM 0, TO_MATCH_ENVRCPT_ALL 0, RCVD_NO_TLS_LAST 0, FREEMAIL_FROM 0.001, RCVD_VIA_SMTP_AUTH 0, IP_SCORE -2.0702971014463, __NOT_SPOOFED 0, R_SPF_ALLOW -0.2, ASN 0, MV_CASE 0.5, __EMPTY_BODY 0, MID_RHS_MATCH_FROM 0⁩
X-Google-Smtp-Source: ⁨APXvYqx3TZvG0tZRPS4u+hd6pZ+impJrOF9B+w6B8jVZqnFzvIvOxS3NU/99hyF05Vb9UlXQ17H/hQ==⁩
Mime-Version: ⁨1.0 (Mac OS X Mail 12.4 (3445.104.11))⁩
Authentication-Results: ⁨decani.be-o.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=CiHo8k/p⁩
X-Synology-Virus-Status: ⁨no⁩
X-Mailer: ⁨Apple Mail (2.3445.104.11)⁩
Dkim-Signature: ⁨v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:content-transfer-encoding:mime-version:subject:message-id:date :to; bh=47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=; b=CiHo8k/pEHnK8ioeo+ekDBbl82nMU/KaWfI5Su+28dGGlNVq7F1LosV0d0Z86dJBgQ hcbO/tdBgVeuepa4xyuZri40FVbaTrktehRcgvObhPDdiZjrUs2deG/MN+LhMfpUAdlE YnOf27mZK3mLRXN5OaRjXJ8lxedQcfjXgKV/8NjodUop7hwY74qgYczf9X0CKIHZVL1a lEplkdRNfvwCt7nWxej8CjzXPzFs/imCGl82Z+1hNedeY4m7DpfahV+vUPDJrEtL1SvS IdpJyJL6gEMhqr79/WKpvQ8HF8XBkoGPF/7U/y4Nl1r5Zt92gJiYPE+5z1RZgVlr/ogl sQaA==⁩
Content-Transfer-Encoding: ⁨7bit⁩
⁨0764AEBB-2331-4EA4-ABA4-3FD49435C900@gmail.com⁩
X-Google-Dkim-Signature: ⁨v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:content-transfer-encoding:mime-version :subject:message-idto; bh=47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=; b=ZeYoS3/G7hllKXaYc5ba09XCaMB6+b/gz/cYt/ORYOan6JM7w4zW5euzmCL+2HLwFZ tLKUGT3LCmhjt+48qbO3K7tziXv58h8yOveVKwwViF/qUFbkEhIx0ClceE4Zq7XHTc37 Oqv0FLm6FkYbC8tBIL8ZwWSVee5q/9wTZDDVwFvoFfMm3BNzg5wUg3I6ADPNTxvf8K8w DC3P4ekGaXW7ueUKVs+d+FUb0FkejCxYmeLXAwibapxU7wPfnInmqL4q3aTJKfbaX6ey z57Pck65CT3Wq5zLd5rfrXDp5oXLxQnFSS25D+ihEm2/BgwsCob/uWj3n/HtLFoxlxjV plsg==⁩
Content-Type: ⁨text/plain; charset=us-ascii⁩
Delivered-To: ⁨vladanpopovic@be-o.com⁩
X-Original-To: ⁨vladanpopovic@be-o.com⁩
Received: ⁨from mail-pf1-f175.google.com (mail-pf1-f175.google.com [209.85.210.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by decani.be-o.com (Postfix) with ESMTPS id 95CC22BBC3B for vladanpopovic@be-o.com; Thu, 5 Sep 2019 18:06:35 +0800 (CST)⁩
Received: ⁨by mail-pf1-f175.google.com with SMTP id q10so1438406pfl.0 for vladanpopovic@be-o.com; Thu, 05 Sep 2019 03:06:35 -0700 (PDT)⁩
Received: ⁨from [192.168.1.188] (061238132018.ctinets.com. [61.238.132.18]) by smtp.gmail.com with ESMTPSA id q20sm4587299pfg.85.2019.09.05.03.06.31 for vladanpopovic@be-o.com (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 05 Sep 2019 03:06:32 -0700 (PDT)⁩

kiokoman

ok so it's not a pfsense problem, maybe there is some problem with the order in wich you pass the email through the milters
can you post a log with spf enabled?

vladanpopovic

actually I can't - it seems to be delivering now with spf check being on ... not sure what changed - I need to clarify before posting more.

thank you very much for your help @Gertjan & @kiokoman

I will post again if I can reliably replicate the problem

Gertjan

@vladanpopovic said in external smtp ip appears as local:

actually I can't

Oh, yes you can.
The filter, or better, as @kiokoman stated, a milter, can be made verbose to see what happens.

I have this in my master.cf file :

policyd-spf-perl  unix  -       n       n       -       -       spawn
	user=nobody argv=/usr/local/bin/perl /usr/lib/postfix/policyd-spf-perl -v

"policyd-spf-perl " is a Debian package, and known as a postfix milter.

You see the -v option ?

I added it.

You can also changing

my $VERBOSE = 0;

to

my $VERBOSE = 1;

in /usr/lib/postfix/policyd-spf-perl

Instead of one line that details SPF operations like DUNNO, fail or pass, I get the whole boat load.

Here are the details https://pastebin.com/qxdg9QKX

Just an idea : you have to open this file : /usr/lib/postfix/policyd-spf-perl and add a gateway like this :

use constant relay_addresses => map(
    NetAddr::IP->new($_),
		qw( 
			92.xxxx.20.243/32
			2001:xxxxx:52:cff::1286/128 )
); # add addresses to qw (  ) above separated by spaces using CIDR notation.
# mail2.aaa-bbbb-fumel.fr
# 92.xxxx.20.243/32 2001:xxxx:52:cff::1286/128

If I didn't do this, mails received by mail MX backup (when the main MX is down) would be marked bad by this SPF filter - sorry : milter.

You use probably another milter for SPF. Just check the doc of the source - or even better : check the source - it's perl or bash or something like that. Making it verbose is always - not difficult -.