Reverse Path Forwarding problem using pfSense

[mauro.tridici]

Dear Users,

two years ago I created a firewall/router using pfsense v.2.7.
pfSense has 2 WAN interfaces, WAN1 and WAN2 with 2 different public IP to connect to the ISP using two different route (2 different gateway)
BGP is configured and running without any problem.

PROBLEM: ISP need to monitor both the two WAN interfaces with a simple ping. WAN2 is responding since it has an higher priority than WAN1 (now). WAN1 is not responding.

The issue is due to an asymmetric routing: WAN1 receives the ICMP packets, but WAN2 is the one that is replying.

ISP calls it a RPF problem. How can I solve it?
I tried to set the specific gateway in the firewall rules related to WAN1, but it doesn't work.

Thank you in advance,
Mauro

[stephenw10]

Are you running 2.7.2 or 2.7.0?

The reply-to tags on states created on WAN1 should mean replies go back that way. That only works if a gateway is defined on WAN1 directly. And if the upstream gateways are unique.

However 2.7.1 had a bug with reply-to/route-to you could be hitting.

[mauro.tridici]

@stephenw10 thank you for your answer.

pfSense version is 2.7.2.
I created two gateways: x.x.x.x on WAN1 and y.y.y.y on WAN2
The upstream gateways are unique.

I noticed that gateway on WAN2 is the default gateway, but I didn't specify that. If I'm not wrong BGP should decide the routes.

[stephenw10]

You should have a system default anyway, it should be in the routing table.

It's not sufficient to have gateways defined to get reply-to tagging. They must be on the interface directly. So in Interfaces > WAN1

[mauro.tridici]

@stephenw10 I can confirm that in each interface I "attached" related gateway.

I don't know what else I can do. I tried to use alo floating rules...but nothing changed.

[stephenw10]

The incoming pings must be passed on the interface specific rules for reply to work.

If you check the created states with: pfctl -vvss | grep -A2 icmp

You should see something like:

igc3 icmp 172.21.16.11:8 <- 172.21.16.8:5       0:0
   age 00:00:52, expires in 00:00:10, 52:52 pkts, 4368:4368 bytes, rule 92
   id: 7562b76700000000 creatorid: db86a6e3 reply-to: 172.21.16.1@igc3

Where you can see the reply-to tags set for the gateway on that interface.

[mauro.tridici]

@stephenw10 ops, reply-to tag is not visibile in the output I captured:

pfctl -vvss | grep -A2 icmp
all icmp WAN1_IP:35978 <- MY_TEST_IP:35978 0:0
age 02:48:28, expires in 00:00:04, 9986:9986 pkts, 838824:838824 bytes, rule 82, log
id: 3c6b2b8200000000 creatorid: d9ee1784

where rule 82 is:

@82 pass in quick on bxe0 route-to (bxe0 WAN1GW) inet proto icmp from <GARR_MONITOR:546> to (self:6) keep state label "USER_RULE" label "id:1740065187" label "gw:WAN1GW" ridentifier 1740065187
[ Evaluations: 427617 Packets: 658 Bytes: 55272 States: 1 ]
[ Inserted: uid 0 pid 89367 State Creations: 251 ]
[ Last Active Time: Thu Feb 20 20:15:31 2025 ]

"disable reply-to" is disabled (not checked)

[mauro.tridici]

@mauro-tridici I tried to unset and set the WAN1GW for the WAN1 interface and now I see the route-tp tag (not the reply-to tag)

all icmp WAN1_IP:47506 <- MY_TEST_IP:47506 0:0
age 00:00:04, expires in 00:00:10, 5:5 pkts, 420:420 bytes, rule 82, log
id: 8f5b388200000000 creatorid: d9ee1784 route-to: WAN1GW@bxe0

[mauro.tridici]

@mauro-tridici I solved the issue recreating also the floating rule. I don't know if it is the best solution for this kind of issue, but it works :)

thank you for your help

[stephenw10]

Looks like those states are created outbound. I assume bxe0 is an internal NIC? The 'route-to' tag there implies policy routing in a firewall rule for that.

Only inbound states on a WAN will get tagged reply-to.