SMTP over WANB? (Multi-WAN config)

danswartz

Can you do a numeric one instead? This was on the LAN?

pfnewbe

Yup. This was on LAN.

Here's another one in numeric.

23:43:18.825387 IP 65.55.34.203.19470 > 192.168.2.16.smtp: S 3691429999:3691429999(0) win 65535 <mss 1452,nop,nop,sackok="">23:43:18.825445 IP 192.168.2.16.smtp > 65.55.34.203.19470: S 3797698790:3797698790(0) ack 3691430000 win 5840 <mss 1460,nop,nop,sackok="">23:43:21.747190 IP 65.55.34.203.19470 > 192.168.2.16.smtp: S 3691429999:3691429999(0) win 65535 <mss 1452,nop,nop,sackok="">23:43:21.747222 IP 192.168.2.16.smtp > 65.55.34.203.19470: S 3797698790:3797698790(0) ack 3691430000 win 5840 <mss 1460,nop,nop,sackok="">23:43:23.358357 IP 192.168.2.16.smtp > 65.55.34.203.19470: S 3797698790:3797698790(0) ack 3691430000 win 5840 <mss 1460,nop,nop,sackok="">23:43:27.765613 IP 65.55.34.203.19470 > 192.168.2.16.smtp: S 3691429999:3691429999(0) win 65535 <mss 1452,nop,nop,sackok="">23:43:27.765646 IP 192.168.2.16.smtp > 65.55.34.203.19470: S 3797698790:3797698790(0) ack 3691430000 win 5840 <mss 1460,nop,nop,sackok="">23:43:29.358662 IP 192.168.2.16.smtp > 65.55.34.203.19470: S 3797698790:3797698790(0) ack 3691430000 win 5840 <mss 1460,nop,nop,sackok="">23:43:41.359231 IP 192.168.2.16.smtp > 65.55.34.203.19470: S 3797698790:3797698790(0) ack 3691430000 win 5840 <mss 1460,nop,nop,sackok="">23:43:43.759388 IP 192.168.2.16.smtp > 212.61.26.38.sdo-tls: S 3698667821:3698667821(0) ack 1312522795 win 5792 <mss 6="" 25655941="" 1460,nop,nop,timestamp="" 1212769185,nop,wscale="">23:44:05.570420 IP 192.168.2.16.smtp > 65.55.34.203.19470: S 3797698790:3797698790(0) ack 3691430000 win 5840 <mss 1460,nop,nop,sackok="">23:44:53.782771 IP 192.168.2.16.smtp > 65.55.34.203.19470: S 3797698790:3797698790(0) ack 3691430000 win 5840 <mss 1460,nop,nop,sackok="">Firewall log says:

block
Aug 23 23:42:47 LAN 192.168.2.16:25 65.55.34.203:19470 TCP:SA

pass
Aug 23 23:41:11 WANB 65.55.34.203:19470 192.168.2.16:25 TCP:S</mss></mss></mss></mss></mss></mss></mss></mss></mss></mss></mss></mss>

danswartz

Okay, I am seeing the inbound SYN and the server is sending back SYN/ACK, and the sender is retrying with backoff which all looks good. The question is why the SYN/ACK is not getting to the remote host. Looking at your NAT and Rules, I note they are for the WAN side only. Can you post your LAN rules and outbound (if any) NAT?

pfnewbe

I've only 1 NAT-rule:
If Proto Src. addr Src. ports Dest. addr Dest. ports NAT IP NAT Ports Description

[Firewall rule ID is managed with this rule] WANB TCP * * WANB address 25 (SMTP) 192.168.2.16 25 (SMTP) NAT SMTP

and my LAN-rules are as mentioned in my post of Reply #20

danswartz

That is the inbound NAT rule - you have no outbound one? Can you post /tmp/rules.debug?

pfnewbe

Nope. This is the only NAT-rule!
When i'm back home I'll post the /tmp/rules.debug

Is the LAN-rule not enough? Everything is allowed to go outside. ???

* LAN net * * * * none Default allow LAN to any rule

danswartz

There are rules that can be added invisibly to what you see in the GUI.

pfnewbe

$ cat /tmp/rules.debug
#System aliases

loopback = "{ lo0 }"
WANA = "{ em2 }"
LAN = "{ em1 }"
WANB = "{ em0 }"
WIFI = "{ em3 }"
DMZ = "{ em4 }"
OpenVPN = "{ openvpn }"

#SSH Lockout Table
table <sshlockout>persist
#Snort2C table
table <snort2c>table <virusprot># User Aliases
table <easyruleblockhostswan>{ 211.154.135.19/32 }
EasyRuleBlockHostsWAN = "<easyruleblockhostswan>"

Gateways

GWWANB = " route-to ( em0 192.168.1.254 ) "
GWGW_WANA = " route-to ( em2 94.209.232.1 ) "
GWGW_OPT1 = " "

set loginterface em2
set loginterface em1
set loginterface em0
set loginterface em3
set loginterface em4
set optimization normal
set limit states 96000

set skip on pfsync0

scrub in on $WANA all fragment reassemble
scrub in on $LAN all fragment reassemble
scrub in on $WANB all fragment reassemble
scrub in on $WIFI all fragment reassemble
scrub in on $DMZ all fragment reassemble

altq on em2 hfsc bandwidth 80Mb queue { qACK, qDefault, qP2P, qVoIP, qGames, qOthersHigh, qOthersLow }
queue qACK on em2 bandwidth 19.792% hfsc ( ecn , linkshare (0b, 100, 19.792%) )
queue qDefault on em2 bandwidth 9.896% hfsc ( ecn , default )
queue qP2P on em2 bandwidth 4.948% hfsc ( ecn , linkshare (4.948%, 300, 4.948%) , upperlimit 4.948% )
queue qVoIP on em2 bandwidth 32Kb hfsc ( ecn , realtime (0b, 10, 512Kb) )
queue qGames on em2 bandwidth 19.792% hfsc ( ecn , linkshare (0b, 50, 19.792%) )
queue qOthersHigh on em2 bandwidth 9.896% hfsc ( ecn , linkshare (0b, 200, 9.896%) )
queue qOthersLow on em2 bandwidth 1% hfsc ( ecn , linkshare (1%, 500, 1%) )

altq on em0 hfsc bandwidth 16Mb queue { qACK, qDefault, qP2P, qVoIP, qGames, qOthersHigh, qOthersLow }
queue qACK on em0 bandwidth 19.76% hfsc ( ecn , linkshare (0b, 100, 19.76%) )
queue qDefault on em0 bandwidth 9.88% hfsc ( ecn , default )
queue qP2P on em0 bandwidth 4.94% hfsc ( ecn , linkshare (4.94%, 300, 4.94%) , upperlimit 4.94% )
queue qVoIP on em0 bandwidth 32Kb hfsc ( ecn , realtime (0b, 10, 512Kb) )
queue qGames on em0 bandwidth 19.76% hfsc ( ecn , linkshare (0b, 50, 19.76%) )
queue qOthersHigh on em0 bandwidth 9.88% hfsc ( ecn , linkshare (0b, 200, 9.88%) )
queue qOthersLow on em0 bandwidth 1% hfsc ( ecn , linkshare (1%, 500, 1%) )

altq on em1 hfsc bandwidth 11000Kb queue { qInternet }
queue qInternet on em1 bandwidth 11000Kb hfsc ( ecn , linkshare (11000Kb, 100, 11000Kb) , upperlimit 11000Kb ) { qACK, qDefault, qP2P, qVoIP, qGames, qOthersHigh, qOthersLow }
queue qACK on em1 bandwidth 19.742% hfsc ( ecn , linkshare (0b, 100, 19.742%) )
queue qDefault on em1 bandwidth 9.871% hfsc ( ecn , default )
queue qP2P on em1 bandwidth 4.9355% hfsc ( ecn , linkshare (4.9355%, 300, 4.9355%) , upperlimit 4.9355% )
queue qVoIP on em1 bandwidth 32Kb hfsc ( ecn , realtime (0b, 10, 512Kb) )
queue qGames on em1 bandwidth 19.742% hfsc ( ecn , linkshare (0b, 50, 19.742%) )
queue qOthersHigh on em1 bandwidth 9.871% hfsc ( ecn , linkshare (0b, 200, 9.871%) )
queue qOthersLow on em1 bandwidth 1% hfsc ( ecn , linkshare (1%, 500, 1%) )

nat-anchor "natearly/"
nat-anchor "natrules/"

Outbound NAT rules

Subnets to NAT

tonatsubnets = "{ 192.168.2.0/24 192.168.20.0/24 192.168.30.0/24 10.0.1.0/24 }"
nat on $WANA from $tonatsubnets port 500 to any port 500 -> 94.209.233.165/32 port 500
nat on $WANA from $tonatsubnets to any -> 94.209.233.165/32 port 1024:65535

nat on $WANB from $tonatsubnets port 500 to any port 500 -> 80.126.204.124/32 port 500
nat on $WANB from $tonatsubnets to any -> 80.126.204.124/32 port 1024:65535

Load balancing anchor

rdr-anchor "relayd/*"

TFTP proxy

rdr-anchor "tftp-proxy/*"
table <direct_networks>{ 94.209.232.0/23 192.168.2.0/24 80.0.0.0/8 192.168.20.0/24 192.168.30.0/24 }

NAT Inbound Redirects

rdr on em0 proto tcp from any to 80.126.204.124 port 25 -> 192.168.2.16

Reflection redirects

rdr on { em1 em3 em4 openvpn } proto tcp from any to 80.126.204.124 port 25 tag PFREFLECT -> 127.0.0.1 port 19000

UPnPd rdr anchor

rdr-anchor "miniupnpd"

anchor "relayd/*"
anchor "firewallrules"
#–-------------------------------------------------------------------------

default deny rules

#---------------------------------------------------------------------------
block in log all label "Default deny rule"
block out log all label "Default deny rule"

We use the mighty pf, we cannot be fooled.

block quick proto { tcp, udp } from any port = 0 to any
block quick proto { tcp, udp } from any to any port = 0

Block all IPv6

block in quick inet6 all
block out quick inet6 all

snort2c

block quick from <snort2c>to any label "Block snort2c hosts"
block quick from any to <snort2c>label "Block snort2c hosts"

package manager early specific hook

anchor "packageearly"

carp

anchor "carp"

SSH lockout

block in log quick proto tcp from <sshlockout>to any port 22 label "sshlockout"
block in quick from <virusprot>to any label "virusprot overload table"
antispoof for em2

allow our DHCP client out to the WANA

anchor "wandhcp"
pass in on $WANA proto udp from any port = 67 to any port = 68 label "allow dhcp client out WANA"
pass out on $WANA proto udp from any port = 68 to any port = 67 label "allow dhcp client out WANA"

Not installing DHCP server firewall rules for WANA which is configured for DHCP.

antispoof for em1

allow access to DHCP server on LAN

anchor "dhcpserverLAN"
pass in on $LAN proto udp from any port = 68 to 255.255.255.255 port = 67 label "allow access to DHCP server"
pass in on $LAN proto udp from any port = 68 to 192.168.2.254 port = 67 label "allow access to DHCP server"
pass out on $LAN proto udp from 192.168.2.254 port = 67 to any port = 68 label "allow access to DHCP server"
antispoof for em0

allow our DHCP client out to the WANB

anchor "opt1dhcp"
pass in on $WANB proto udp from any port = 67 to any port = 68 label "allow dhcp client out WANB"
pass out on $WANB proto udp from any port = 68 to any port = 67 label "allow dhcp client out WANB"

Not installing DHCP server firewall rules for WANB which is configured for DHCP.

antispoof for em3
antispoof for em4
anchor "spoofing"

loopback

anchor "loopback"
pass in on $loopback all label "pass loopback"
pass out on $loopback all label "pass loopback"

anchor "firewallout"

let out anything from the firewall host itself and decrypted IPsec traffic

pass out all keep state allow-opts label "let out anything from firewall host itself"
pass out route-to ( em2 94.209.232.1 ) from 94.209.233.165 to !94.209.232.0/23 keep state allow-opts label "let out anything from firewall host itself"

make sure the user cannot lock himself out of the webConfigurator or SSH

anchor "anti-lockout"
pass in quick on em1 from any to (em1) keep state label "anti-lockout rule"

NAT Reflection rules

pass in inet tagged PFREFLECT keep state label "NAT REFLECT: Allow traffic to localhost"

User-defined rules follow

pass log on { em0 } proto tcp from any to any port 25 flags S/SA keep state queue (qOthersHigh,qACK) label "USER_RULE: m_Other SMTP outbound"
pass out from any to any queue (qOthersLow) label "USER_RULE: Penalty Box"
pass out proto udp from any to any queue (qVoIP) label "USER_RULE: DiffServ/Lowdelay/Upload"
pass out proto tcp from any to any port 6880 >< 7000 queue (qP2P) label "USER_RULE: m_P2P BitTorrent outbound"
pass out proto udp from any to any port 6880 >< 7000 queue (qP2P) label "USER_RULE: m_P2P BitTorrent outbound"
pass out proto tcp from any to any port 4660 >< 4666 queue (qP2P) label "USER_RULE: m_P2P EDonkey2000 outbound"
pass out proto tcp from any to any port 6346 queue (qP2P) label "USER_RULE: m_P2P Gnutella-TCP outbound"
pass out proto udp from any to any port 6346 queue (qP2P) label "USER_RULE: m_P2P Gnutella-UDP outbound"
pass out proto tcp from any to any port 6698 >< 6702 queue (qP2P) label "USER_RULE: m_P2P Napster outbound"
pass out proto tcp from any to any port 8887 >< 8890 queue (qP2P) label "USER_RULE: m_P2P OpenNap outbound"
pass out proto udp from any to any port 17477 >< 17489 queue (qGames) label "USER_RULE: m_Game Delta1 outbound"
pass out proto tcp from any to any port 49000 >< 49003 queue (qGames,qACK) label "USER_RULE: m_Game FarCry-1 outbound"
pass out proto udp from any to any port 49000 >< 49003 queue (qGames) label "USER_RULE: m_Game FarCry-2 outbound"
pass out proto tcp from any to any port 27015 queue (qGames,qACK) label "USER_RULE: m_Game HL-1 outbound"
pass out proto udp from any to any port 27650 queue (qGames) label "USER_RULE: m_Game HL-2 outbound"
pass out proto udp from any to any port 27666 queue (qGames) label "USER_RULE: m_Game HL-3 outbound"
pass out proto udp from any to any port 7776 >< 7788 queue (qGames) label "USER_RULE: m_Game ur1 outbound"
pass out proto tcp from any to any port 7776 >< 7788 queue (qGames,qACK) label "USER_RULE: m_Game ur2 outbound"
pass out proto udp from any to any port 88 queue (qGames) label "USER_RULE: m_Game xbox360-1 outbound"
pass out proto udp from any to any port 3074 queue (qGames) label "USER_RULE: m_Game xbox360-2 outbound"
pass out proto tcp from any to any port 3074 queue (qGames,qACK) label "USER_RULE: m_Game xbox360-3 outbound"
pass in log quick on $WANA reply-to ( em2 94.209.232.1 ) proto tcp from any to 94.209.233.165 port 1194 flags S/SA keep state label "USER_RULE: OpenVPN wizard rules."
pass in log quick on $WANB proto udp from any to 80.126.204.124 port 1194 keep state label "USER_RULE"
pass in log quick on $WANB proto tcp from any to 192.168.2.16 port 25 flags S/SA keep state label "USER_RULE: NAT NAT SMTP"
pass in quick on $WANB proto igmp from 192.168.1.254 to 224.0.0.1 keep state label "USER_RULE: Easy Rule: Passed from Firewall Log View"
pass in quick on $OpenVPN from any to any keep state label "USER_RULE: OpenVPN wizard rules."
pass in log quick on $LAN from 192.168.2.14 to <vpns>keep state label "NEGATE_ROUTE: Negate policy route for vpn(s)"
pass in log quick on $LAN $GWWANB from 192.168.2.14 to any keep state label "USER_RULE: mail route via WANB"
pass in log quick on $LAN from 192.168.2.16 to <vpns>keep state label "NEGATE_ROUTE: Negate policy route for vpn(s)"
pass in log quick on $LAN $GWWANB from 192.168.2.16 to any keep state label "USER_RULE: mailgw route via WANB"
pass in log quick on $LAN from 192.168.2.0/24 to any keep state label "USER_RULE: Default allow LAN to any rule"

VPN Rules

package manager late specific hook

anchor "packagelate"

anchor "tftp-proxy/*"

anchor "limitingesr"

uPnPd

anchor "miniupnpd"

havp proxy ifaces rules</vpns></vpns></virusprot></sshlockout></snort2c></snort2c></direct_networks></easyruleblockhostswan></easyruleblockhostswan></virusprot></snort2c></sshlockout>

danswartz

I think this is the problem?

# Gateways
GWWANB = " route-to ( em0 192.168.1.254 ) " <==== this looks bogus!
GWGW_WANA = " route-to ( em2 94.209.232.1 ) "
GWGW_OPT1 = "  "

You seem to have defined the WANB gateway with the internal IP address. The WANA GW is correct, I think.

pfnewbe

That's the address of my ADSL router (192.168.1.254 my internal range is 192.168.2.xxx) which is configured in 'bridge-mode'
All other traffic from 192.168.2.16, like a finger to flush the bSMTP from my provider, or traceroute goes correctly over WANB.
I'll do a reset to factory defaults and try to set it up again from bottom up. This it's also not working. at least I can try is too do it over from fact default.
I'll keep you posted!

danswartz

Ah, okay. I am confused. If the ADSL router has a different subnet on each interface, why do you say it is in bridge mode? If it were truly bridging, WANB would be a routable address, no? What do you see if you do a trace on WANB while doing this?

pfnewbe

Hi dans…. Sorry for the late response. Was a bit busy last days.
Made a complete clean install. 1st created only WAN and LAN interface.
LAN - 192.168.2.254
WANA - DHCP
after upgrade configured WANB - DHCP
When i look on my dashboard it says that WANA is online and WANB, with gateway 'dynamic' (?????), is offline.
See screenshot.
I've done nothing else then only configured WANB with DHCP.
Did I something wrong?

Screenshot-fw1.png_thumb

danswartz

you didn't answer my question :( you seem to think the adsl modem is in bridge mode, but obviously it isn't, since you originally had two different subnets, one on each side of the modem. can you really set it to bridged mode and try again?

pfnewbe

Oh sorry. Yes it's realy set in bridge mode.
The old setting was when it was not in bridge-mode and I forgot to remove the old setting.

danswartz

Are you sure the WANB provider is DHCP? It's odd you're now not getting an address…

pfnewbe

Since I'm using this provider (about 6 years) it has always been DHCP.
And when your looking on the image I've also received my own address at WANB.
On my old firewall's that I've used in the years (WatchGuard and IPCop) it worked always.

![fw1 - dashboard.png](/public/imported_attachments/1/fw1 - dashboard.png)
![fw1 - dashboard.png_thumb](/public/imported_attachments/1/fw1 - dashboard.png_thumb)

danswartz

I know pfsense DHCP WAN works, so continually mentioning ipcop and whatever is not helpful. Since your WANB is not getting an address, you must have set it up wrong?

pfnewbe

Well… if your looking to the screenshot, you'll see that I received the address on WANB! The only question is where is my gateway ;D

danswartz

Sorry, I meant an address for the gateway. That should happen automatically for a DHCP WAN connection. It certainly does for mine. The previous screenshot showed the dynamic gateway that was offline, is what I was referring to. I wonder if you have some junk left over from before?

pfnewbe

I think not. It was a clean install. But I'll put the modem back in routed mode. Only point is that I need to configure incoming services at two places. (the modem and in pfSense)