MultiWAN on 2.0 with FW rule for pool-> everything goes through defgault gateway

[mark_orion]

I am trying to set up MultiWAN on the 2.0 snapshot (latest snapshot update from 21st. February). I followed instructions on this forum and set up the gateways and a gateway group. Although I have a firewall rule that matches all incoming traffic on the LAN and directs it to the pool (the log confirms this), I seem to only be using the default gateway.
I tried with different machines at the same time etc., but the MultiWan does not seem to work.

[jimp]

Show the configuration of your gateways, gateway groups, rules, and gateway status. Without that, it's all just guesswork.

[mark_orion]

Setup is one DMZ network (192.168.0.x) with four gateways and pfSense connected to it. One gateway is configured as "default gateway" - I tried unchecking the "default gateway" setting, but it changed nothing. LAN is not using DHCP as this is a test setup and we have another DHCP server on the network. All clients that I use for testing have a static IP configuration to use the pfSense installation.
The gateways are in one group called "EqualRouting" with three of them on Tier1 and one as fallback on Tier2.
I have tried various Firewall rules. The rule matching works according to the logs, but the rule based routing seems not to work.
I tried even to force traffic from one machine through one specific gateway - here 192.168.0.10, but even this does not work.
Although the popup I get in the log viewer shows this, it still routes to the default gateway what is 192.168.0.20:

The rule that triggered this action is:
@28 pass in log quick on em1 route-to (em0 192.168.0.10) inet from 192.168.55.63 to any flags S/SA keep state label "USER_RULE: Getway Test"

The config file (with the admin user password etc. removed) is attached here.

config-pfsense.localdomain-20110223122042.xml.txt

[jimp]

ah, you have all the gateways on a single interface. I don't think that setup has seen much testing. I seem to recall an open ticket about issues with that kind of setup.

[mark_orion]

@jimp:

ah, you have all the gateways on a single interface. I don't think that setup has seen much testing. I seem to recall an open ticket about issues with that kind of setup.

What would you suggest as a workaround for this problem? I cannot separate the four gateways into isolated networks for now, so they have to sit in the single DMZ network.

[mark_orion]

@jimp:

ah, you have all the gateways on a single interface. I don't think that setup has seen much testing. I seem to recall an open ticket about issues with that kind of setup.

I could solve the problem by disabling NAT on the WAN device and setting up static routes from each gateway to the networks behind pfSense.

[francesco_r]

I have upgraded my 1.2.3 box to the latest snapshot 2.0-RC1 (i386 built on Tue Mar 22 11:53:58 EDT 2011) and i have the same exact problem.
I have one Wan interface with two gateways and 20 vlan. In 1.2.3 worked perfectly (with an hack posted on this forum) but now that i have upgraded, all traffic pass through the gateway defined in the Wan interface.

The same bug was report some times ago and is still open: http://redmine.pfsense.org/issues/651

Reading the page http://doc.pfsense.org/index.php/2.0_New_Features_and_Changes
the Gateways/Multi-WAN paragraph says:

"You can have multiple gateways per interface"

…but this sentence is not true! :)

[eri--]

Well it depends on the meaning.
You do not provide any reason why the second monitor ip should not be unreachable.

Can you show system log and the screenshots of when this happens?
Also can you verify that the gateways are not in the same mac address!?

[Maverick]

Hello,

I think we have a similar problem:

http://forum.pfsense.org/index.php/topic,34883.0.html

We have defined a static route (second gateway) for the WAN interface. But the system sends all the traffic via the default WAN gateway.

Greetings
Mav

[francesco_r]

@ermal:

Well it depends on the meaning.
You do not provide any reason why the second monitor ip should not be unreachable.

Can you show system log and the screenshots of when this happens?
Also can you verify that the gateways are not in the same mac address!?

I have attached the screenshots of routes, arp, gateway and firewall rules of vlan3.
I have two gateways (adsl modems): 192.168.1.1 and 192.168.1.5, the wan interface is 192.168.1.3 with selected gw 192.168.1.1.
In routing table i can see that:
77.43.0.8          192.168.1.5        UGHS        0  162674    vr0
but if i do a traceroute from the firewall to 77.43.0.8 (the monitor IP of the second gateway) i have:

traceroute 77.43.0.8

traceroute to 77.43.0.8 ( 77.43.0.8 ), 64 hops max, 40 byte packets
1  192.168.1.254 ( 192.168.1.254 )  1.583 ms  1.320 ms  4.315 ms
2  static-213-205-… etc.

As you can see from the screenshot, the routing table is not considered and all packets pass always through the default route.
In the firewall rules of vlan3 (OPT3) i have selected the backup gateway 192.168.1.5 but if i do a traceroute from a client in this vlan, i have same response of above.
In other words, all the packet pass through the gateway defined in wan interface.

I hope that my explanations are clear.
Francesco

[eri--]

Try either removing the default gateway checkbox from the gateways or putting a floating rule on floating tab with direction out and quick selected

[francesco_r]

@ermal:

Try either removing the default gateway checkbox from the gateways or putting a floating rule on floating tab with direction out and quick selected

I tried to remove default checkbox or creating a floating rule above all but the problem is still present. I think is a bug.

[francesco_r]

Is someone of developer aware of this problem? Should i open a new ticket on issue tracker?