pfSense with multi-WAN on same subnet

Jolly green robot

Hi

I have a network setup where pfSense connects to two gateways on the same subnet with a single network interface but it fails to route any packets to the second gateway.

My configuration:
Interfaces / WAN

IPv4 Configuration Type: Static IPv4
IPv4 Address: 192.168.1.5
IPv4 Upstream gateway: GW_WAN1 - 192.168.1.1
IPv4 Upstream gateway: GW_WAN2 - 192.168.1.2

System / Routing / Gateways

Name                Default         Interface   Gateway         Monitor IP 
GW_WAN1                             WAN         192.168.1.1     xx.xx.xx.xx 
GW_WAN2 (default)   Default (IPv4)  WAN         192.168.1.2     yy.yy.yy.yy 
Default gateway IPv4: GW_WAN2

netstat -r

Destination        Gateway            Flags     Netif Expire
default            192.168.1.2        UGS         xl0
yy.yy.yy.yy        192.168.1.2        UGHS        xl0
xx.xx.xx.xx        192.168.1.1        UGHS        xl0
192.168.1.1/24     link#6             U           xl0

route show 8.8.8.8

   route to: dns.google
destination: default
       mask: default
    gateway: 192.168.1.2
        fib: 0
  interface: xl0
      flags: <UP,GATEWAY,DONE,STATIC>
 recvpipe  sendpipe  ssthresh  rtt,msec    mtu        weight    expire
       0         0         0         0      1500         1         0

So everything should be routed over gateway 2, but a traceroute returns 192.168.1.1 as the first hop and if I physically disable gateway 1 then there is no connectivity.

Further, the configured Monitor IP yy.yy.yy.yy isn't monitored through GW_WAN2 but instead the pings are sent to GW_WAN1. (I assume this is because dpinger uses an interface as the gateway.) If I change the monitor IP to 192.168.1.2 the gateway is monitored correctly.

Has anybody experienced anything similar or have any advice?

PS. I was planning on using VLANs but the switch and the first gateway doesn't support it.

Thanks in advance.

KOM

I can't see anything wrong with that config. I've seen some weird issues disappear with a reboot so you could try that if you haven't already done so.

Jolly green robot

I have restarted it but it seems pfSense can only route on an interface level, not on an IP or subnet.

qctech

You might be able to fix the dpinger issue by adding a routing entry for the IP of the monitor address that sends it via .1.2 maybe?

I doubt however that this is the main issue and might be pointless. Just a thought, can you totally remove .1.1 and leave only 1.2, does it then work? Is 1.2 defiantly routing?

Jolly green robot

I deleted Gateway 1 on pfSense and set the IPv4 Upstream gateway on the 192.168.1.5 interface to 192.168.1.1 and everything was routed over gateway 2.

It would seem that the routing that is configured on System / Routing / Gateways is done per network interface and the interface's gateway is set on Interfaces / WAN
Only the selected IPv4 Upstream gateway is used on the interface even if multiple are configured.

I don't believe there is an easy way around this other than creating another interface, be it a physical one or a VLAN.
Or is there?

Derelict

Looking at the routing table isn't going to help you because pfSense uses route-to to force traffic at a specific gateway on an interface. That gateway is the one you are setting on the WAN interface configuration.

I would make another interface on a different subnet and it will all work.

Jolly green robot

I guess this issue is then solved.
Thanks for the help.