Problems with the "This Firewall (self)" pf macro in a floating rule on a system with two discreet WAN interfaces

timtrace

Greetings, first post. On my 2.4.5-RELEASE system with two discreet WAN interfaces, I'm trying to use the "This Firewall (self)" pf macro in a floating rule for ICMP echo replies, and there is 100% loss when pinging one of the interfaces.

The documentation makes it seem like it should be a pretty easy thing: "This Firewall (self) - Any IP address assigned to any interface on this firewall (pfSense software version 2.2+)" Excellent. But I think I may have bumbled my way into a fringe case.

In my configuration, WAN and OPT2 are Internet gateways, DHCP from the provider, unique IPs, no HA, nothing fancy, just discreet routes for a couple of VLAN/SSIDs.

The floating rule is set to listen on both WAN interfaces, and it's set to use the default gateway - could this be the problem? If so, must I remove the floating rule and use separate interface rules for each WAN interface?

For troubleshooting purposes, I've written three inbound rules to allow pings from the Internet (xmls copied below, let me know if I need to provide more detail). For now, there's a floating rule and two WAN interface rules but when I'm done here I'll use only the floating rule if it's possible to do so, or only the two interface rules if that's what it takes.

All three rules are disabled, sitting at the top of their respective sets, immediately below the default bogon, private, and anti-lockout rules. When testing, I enable them one at a time, then ping the interface from an external service, and then disable them again.

Floating rule WAN+OPT2: WAN 0% ping loss, OPT2 100% ping loss
WAN interface rule: 0% ping loss
OPT2 interface rule: 0% ping loss

When I filter the firewall log by the OPT2 interface and the floating rule's tracking ID. I see "Pass" events. I'd look for the related reply packet, but I don't know how - if I clear the log filter and searching for the destination IP, there are no logs to display.

Is this known & expected behavior, or have I stumbled into something outside the scope of the "This Firewall (self)" macro as it relates to floating rules, or could it be something else altogether?

Thanks for reading. Please let me know your thoughts.
.

	<rule>
		<id></id>
		<tracker>1589826972</tracker>
		<type>pass</type>
		<interface>wan,opt2</interface>
		<ipprotocol>inet46</ipprotocol>
		<tag></tag>
		<tagged></tagged>
		<direction>in</direction>
		<quick>yes</quick>
		<floating>yes</floating>
		<max></max>
		<max-src-nodes></max-src-nodes>
		<max-src-conn></max-src-conn>
		<max-src-states></max-src-states>
		<statetimeout></statetimeout>
		<statetype><![CDATA[keep state]]></statetype>
		<os></os>
		<protocol>icmp</protocol>
		<icmptype>echoreq</icmptype>
		<source>
			<any></any>
		</source>
		<destination>
			<network>(self)</network>
		</destination>
		<disabled></disabled>
		<log></log>
		<descr><![CDATA[Pass echo request from Internet - https://tools.keycdn.com/ping]]></descr>
		<created>
			<time>1589826972</time>
			<username><![CDATA[admin@192.168.200.21 (Local Database)]]></username>
		</created>
		<updated>
			<time>1589831727</time>
			<username><![CDATA[admin@192.168.200.21 (Local Database)]]></username>
		</updated>
	</rule>
	<rule>
		<id></id>
		<tracker>1589826836</tracker>
		<type>pass</type>
		<interface>wan</interface>
		<ipprotocol>inet46</ipprotocol>
		<tag></tag>
		<tagged></tagged>
		<max></max>
		<max-src-nodes></max-src-nodes>
		<max-src-conn></max-src-conn>
		<max-src-states></max-src-states>
		<statetimeout></statetimeout>
		<statetype><![CDATA[keep state]]></statetype>
		<os></os>
		<protocol>icmp</protocol>
		<icmptype>echoreq</icmptype>
		<source>
			<any></any>
		</source>
		<destination>
			<network>(self)</network>
		</destination>
		<disabled></disabled>
		<log></log>
		<descr><![CDATA[Pass echo request from Internet - https://tools.keycdn.com/ping]]></descr>
		<created>
			<time>1589826836</time>
			<username><![CDATA[admin@192.168.200.21 (Local Database)]]></username>
		</created>
		<updated>
			<time>1589831805</time>
			<username><![CDATA[admin@192.168.200.21 (Local Database)]]></username>
		</updated>
	</rule>
        <rule>
        		<id></id>
        		<tracker>1589824840</tracker>
        		<type>pass</type>
        		<interface>opt2</interface>
        		<ipprotocol>inet46</ipprotocol>
        		<tag></tag>
        		<tagged></tagged>
        		<max></max>
        		<max-src-nodes></max-src-nodes>
        		<max-src-conn></max-src-conn>
        		<max-src-states></max-src-states>
        		<statetimeout></statetimeout>
        		<statetype><![CDATA[keep state]]></statetype>
        		<os></os>
        		<protocol>icmp</protocol>
        		<icmptype>echoreq</icmptype>
        		<source>
        			<any></any>
        		</source>
        		<destination>
        			<network>(self)</network>
        		</destination>
        		<disabled></disabled>
        		<log></log>
        		<descr><![CDATA[Pass echo request from Internet - https://tools.keycdn.com/ping]]></descr>
        		<created>
        			<time>1589824840</time>
        			<username><![CDATA[admin@192.168.200.21 (Local Database)]]></username>
        		</created>
        		<updated>
        			<time>1589833700</time>
        			<username><![CDATA[admin@192.168.200.21 (Local Database)]]></username>
        		</updated>
        	</rule>

viragomann

Responses on traffic which is allowed by floating rules always go out to the default gateway. There are the same limitations effective to floating rules as to rules on interface groups.

So for correct routing with Multi-WAN you have to add the rules to the interface tabs.

timtrace

Thanks for that. It certainly fits the problem.

Running off your lead, I found this: Inbound traffic from Internet: On current versions of pfSense software, from the perspective of traffic coming in to services on pfSense from the Internet, connections will return through the WAN which they entered. This works for TCP-based services and UDP services bound to specific IP addresses.

Plugging this into my situation, it seems to say that inbound pings to a particular WAN interface will be returned from that same interface. Am I misinterpreting it?

Thank you --

viragomann

That's basically true. That behavior is controlled by the "reply-to" flag in pfSense.
And now read again the paragraph I linked above at "limitations".

Since floating rules are applied to multiple interfaces, these are affected as well.

serbus

@timtrace said in Problems with the "This Firewall (self)" pf macro in a floating rule on a system with two discreet WAN interfaces:

This works for TCP-based services and UDP services bound to specific IP addresses.

Hello!

It looks like you are doing ICMP...?

John

timtrace

@viragomann - Thanks - the paragraph which you linked and I read the first time :) appears to be contextual to Interface Groups. I'll quote it for convenience:

*Using Interface Groups for WANs will not put “reply-to” on the resulting pass rules because it can’t do so for rules in groups. Due to this, traffic that enters via an alternate WAN (non-default route) and is allowed by pass rules on an interface group will exit back via the default route. This only applies to rules that pass traffic, blocking is fine either way.

To keep the proper return routing, place pass rules for inbound services behind the firewall on their respective WAN tabs, and not on the group tab.*

I feel like this doesn't apply here because I don't have any Interface Groups configured.

@serbus - Yea, that's a good catch. Thank you. Because TCP and UDP are explictly mentioned, the omission of ICMP has to have been intentional. I guess I haven't found the part of the documentation which applies here, or there's some small undocumented grey area, or @viragomann's advice applies even though the interfaces aren't grouped - at this point my money's on this.

jimp

You'll need to use the interface tab rules. Anything that uses multiple interfaces (groups, floating rules, etc) can't properly form a reply-to parameter which makes sense, since it can only have one target.

You can block using floating rules and groups, but passing won't work like that for multi-wan.

timtrace

Thanks man!