[Multi-WAN] Routing issue after failover

D.J · Jan 25, 2012, 4:19 PM

Hi folks,

Excuse me for my english, but I will try to define my problem as good as possible. I am having a problem after the pfSense router switches it's gateways after it detects an broken WAN connection (based on 8.8.8.8 and 8.8.4.4 ping results). This is managed by the "Gateway groups", after it detects a broken WAN it removes that interface from the group.

First of all, let's take a look at our network situation:

Here's the configuration for the WAN connections:
WAN 1: (as shown in picture with name WAN 1)
Interface IP address: Static IP provided by Cable ISP
Connected with: fully tranparant modem. Has no router functions. It passes everything through.
Gateway: Provided by Cable ISP.

WAN 2: (as shown in picture with name WAN 2)
Interface IP address: 192.168.2.100
Connected with: Draytek ADSL modem/router combination (has ip-address 192.168.2.1).
Gateway: Draytek ADSL modem/router combination.
Extra information about Draytek modem/router: DMZ to pfSense router (ip-adress is 192.168.2.100).

Here's the configuration for the LAN connection:
LAN: (as shown in picture with name Internal network)
Interface IP address: 192.168.1.1/24.
Extra information: DHCP for the internal network is supplied by a Windows Server 2008 machine

Here's the configuration for the guest wireless:
Interface IP-adress: 192.168.3.1
Functions by pfSense: Captive portal with DHCP active only on this interface.

After pfSense detects that WAN 1 fails, it removes the interface from the gateway group. This is when the problem starts to show itself.

There are a few services (ports) needed for the things that we do. We need VPN PPTP (1723, TCP/UPD and GRE protocol), HTTPS (443), IMAP4 (143) and SMTP (25) to some of our servers.

These are added in the NAT function of pfSense and also added to the firewall rules (on each WAN, no floating rules). Normally these ports function on both interfaces. I have tested a few telnet sessions to these ports and they seem to work when both WAN interfaces are available.

But after WAN 1 isn't working, it "failsover" to the WAN 2 interface (or gateway).

The problem that get's in the way is that at that moment I can't open connections to the services we need (VPN PPTP (1723, TCP/UPD and GRE protocol), HTTPS (443), IMAP4 (143) and SMTP (25)) from external servers.

Firewall monitoring on pfSense says the request is "passed" (green icon), and in my opinion it should work. But it doesn't. The connections don't work.

After WAN 1 gets back online again, all the services work again. Very strange!

Does anyone have a clue?

namezero111111 · Jan 28, 2012, 1:45 AM

What you are describing can occur if the pfsense box is trying to always route out of WAN1.
Did you configure the gateway IP addresses specifically on the interface config? Or are they DHCP assigned?
Secondly, in Advanced System options (forgot which page), make sure that Reply-To is not disabled.

Then, run a tcpdump on both outbound interfaces simultaneously in two different SSH sessions, disconnect WAN1, and try telnet again. Then see if any traffic is coming out of WAN1 back to the telnet client (there should be none).

Post your results here so we can look at it.

-namezero

dparkhill · Jan 30, 2012, 10:54 PM

DJ,

Have you had any success? I'm having an issue with almost the config except that my modems are a sonicwall and ZyXel.