Reply on wrong gateway
We've a multi-wan environment.
We've 3 WAN (see img1) :
- DSL2 (default gw GW_OPT2)
- FIBER (MPLS) (fiber gw GWFIBREJN)
Our default gateway is DSL1. It used for everything that is not routed via FIBER.
FIBER is used to link our remote site (datacenter) over an MPLS. FIBER is also used for everything critical (SIP in our local site …).
LAN is bridged with WAN_FIBER (see img1b).
On our local site, we've our main mail server : 10.100.1.13. Everything goes out from this server may use the fiber gw (see img2).
Everything is working.
We've an external IP mapped to reach directly our mail server.
Incoming connection via this external IP reach the server but connection reply is not reached. A packet sniffer told us why : the reply (SYN ACK) is outgoing using the default gw (GW_OPT2) instead using the same interface has the packet was incoming (GBFIBREJN).
The problem :
- We've a firewall rules (img2) : everything goes out from 10.100.1.13 may use the FIBER gw (GBFIBREJN). How the reply can use another gw ( ? What's wrong with our setup ?
I tried the option "Disable reply-to on WAN rules" but this didn't solve my issue.
from which firewall-tab is img2.png ? lan or bridgeX ?
do you comply with quote from https://doc.pfsense.org/index.php/Interface_Bridges
y default, traffic is filtered on the member interfaces and not on the bridge interface itself. This behavior may be changed by toggling the values of net.link.bridge.pfil_member and net.link.bridge.pfil_bridge under System > Advanced on the System Tunables tab. With them set at 0 and 1, respectively, then filtering would be performed on the bridge only.
NOTE: Only one interface on a bridge should have an IP address! Do not add multiple IP addresses in the same subnet on different bridge member interfaces. Other interfaces on the bridge should remain with an IP type of None.
it looks like policy routing gone wrong to me, but it could also be something else ;)
also, why would you want to bridge fiber+lan ? (just curious)
img2 is LAN tab.
I do not comply with this quote about the tunables settings. I already saw it. Is it mandatory to apply filtering on the bridge instead of interfaces (like we did) ?
Fiber+lan is bridged because the fiber interface is an MPLS connection to link our remote sites together with our local network, the fiber gw is 10.100.255.254 (our local network is 10.100.0.0 too).
Sorry, I'm hope my explanation is clear ^^
not totally clear. (i'm just not very smart)
so, you bridged fiber+lan (or you basically bridged datacenter-lan & local-lan)
datacenter-lan subnet = local-lan subnet | right ?
then why use a gateway at all ? you are in the same broadcast domain and packets would flow to and from the datecenter without any >=layer3 involvement ?