Multiple IPv6 gateways, can't prevent asymmetric routing
-
I have multiple IPv4 and IPv6 connections. I'm trying to allow my pfSense instance to be reachable via both connections. Notably, at this point I am not talking about devices behind pfSense - just the firewall itself.
┌─────────────────────┐ ┌────────────────────┐
│VDSL modem (ethernet)│ │LTE modem (ethernet)│
└─────────────────────┘ └────────────────────┘
│ │
│ │
│ │
┌───────────────────┐ ┌───────────────────┐ ┌───────────────────┐
│ │ │ │ │ │
│ PRIMARY WAN │ │ SECONDARY WAN │ │ SECONDARY WAN GIF │
│ static /24 ipv4 │ │ static /32 ipv4 │───│ static /64 ipv6 │
│ │ │ │ │ │
└───────────────────┘ └───────────────────┘ └───────────────────┘
│
│
│
│
│
┌───────────────────┐
│ │
│ PRIMARY WAN PPPOE │
│ ppp /27 ipv4 │
│ dhcp6 /60 ipv6 │
│ ┌────────────────┴─────┐
└──┤ default gateway ipv4 │
│ default gateway ipv6 │
└──────────────────────┘The issue I'm having is that with the setup depicted, using an outside box with native IPv6, I can ping the IPv6 WAN address of 'PRIMARY WAN PPPOE' perfectly fine. The problems happen when I try to ping the IPv6 WAN address of 'SECONDARY WAN GIF'. I don't get any response.
I've sniffed traffic on the pfSense instance, and ICMP ping requests are coming in on SECONDARY WAN GIF as intended, but replies are leaving through PRIMARY WAN PPPOE which won't route the secondary subnet's traffic, so are being dropped after they leave pfSense on the PPPoE interface.
I checked the pf ruleset, and found the following (as expected). Objects in angle brackets have been replaced for privacy but represent the correct addresses.
@154(1000013267) pass out route-to (gif0 <gif remote="" ipv6="">) inet6 from <gif ipv6="">to ! <gif ipv6="" subnet="">/64 flags S/SA keep state allow-opts label "let out anything from firewall host itself"
However, this seems to be ignored, and the default gateway used. Ping monitoring of the gateway's monitor IPv6 works fine, because a specific route is added for the monitor IPv6.
Any insights would be very welcome. Please ask if you need more detail.</gif></gif></gif>