Multi-WAN Gateway option gets ignored in firewall rule

Crunk_Bass

I created the following rule and the gateway option on IPv4 connections is ignored.
alt text

The firewall always uses the gateway that is used by the system itself for new connections.
If I disconnect the interface the system gateway is on, the gateway configured in the firewall rule is used.
Also it doesn't count traffic or stats.

The IPv6 rule is working fine.

I noticed a world icon next to the IPv4 gateway that is used regardless of firewall rules.
alt text

Here is the gateway group that is used:
alt text

Do you have any ideas what could be the issue here?

Crunk_Bass

After disabling Negate rules in System - Advanced - Firewall & NAT it seems to be working okay.
https://docs.netgate.com/pfsense/en/latest/book/config/advanced-firewall-nat.html#disable-negate-rules

quidnunc

what's the downside of "disable negate rules"? I have the same problem as OP where while it will change gateways on failover that it doesn't seem to ever go back to the higher tier gateway, instead always using what has the "world icon" as above. presumably that icon should switch back to the higher tier gateway when it changes the routing rules or whatever but I've never seen it happen yet in months of using this despite the higher tier gateway being online, often for hours

I'm starting to get frustrated and look into switching to a Untangle instead

CvH

I have the same problem since at least a 3/4 year.
Strangely it works for some and does not work for others.
A workaround is to reboot pfsense or maybe, not 100% sure, drop all states - afterwards it looks working. Also I have to say that I did not properly monitored it to be sure that every connection goes through the gateway it should. So I can't rule out that it behaves still randomly.

Added rules without rebooting pfsense, just one rule is working.

quidnunc

Yeah, for sure it reverts to tier 1 gateway if I reboot...

The issue is it's not recovering the gateway group tier, whether it's set in system -> routing -> default gateway or in the advanced section of the lan rule. This also follows for secondary lan rules with the tier order reversed. To be clear - it does fail over initially to tier2 if my tier 1 passes threshold to be down

crond

If it's still doesn't work after you disable default rules, you may be running into the issue I have reported here https://forum.netgate.com/topic/153039/dmz-to-multi-wan-over-vpn

If your GW is set dynamically most likely it's not available when system boots and your firewall rule will end up just allowing all traffic
Check your /tmp/rules.debug it's likely to have something like
pass in on { vmxXYZ } $GWWAN1_IPV4 inet from ..........

If your GW is not available at the boot time the $GWWAN1_IPV4 will be empty and remain empty even after your WAN1 GW is up. So you would just allow all traffic through and will go through default system GW

Easiest way to test if it's the case is to reload the firewall after system up and running without doing any other modification. If it does help, see my post for details, otherwise it's something else.