IPv6 / CARP (failover) / NPt

mdpugh

This topic falls into multiple categories, but I guess IPv6 is the most generally applicable. I have two pfSense boxes setup with CARP in both IPv4/IPv6. Obviously CARP/IPv6 isn't working correctly because I can't advertise the CARP address. I also have NPt set up on each box and each box has a separate HE tunnel. I've assigned unique-local addresses (prefix fd00:ac10::/120) to the clients on the link and this address is translated to a different global address by each router (for our purposes, addresses with prefixes 2001:470:8ac10::/120 and 2001:470:8ac10::/120, respectively). I suspect the "blackhole" behavior mentioned in response to another post is akin to what is shown in the following diagram.

Say the server at fd00:ac10::a receives a ping from 2001:470:8:5678::10 originally addressed to 2001:470:8ac10::a. Because of the destination address in the Echo Request, the packet is forwarded by pfSense 1. But if the server sends its reply through pfSense 2, the return (source) address has a different prefix: 2001:470:8ac10::a. Naturally, this is not what is expected by the ping sender and even if the packet is allowed through all firewalls, the client will not recognize it as a response to its ping. This behavior is not restricted to ICMPv6 packets (you couldn't build a TCP connection, for example) or to NPt configurations, but is unique to IPv6 failover (not necessarily CARP as CARP isn't really getting used here). The solution is to make sure the client replies via the same router it received the message through. What is the preferred (non-hack) method of doing this?

On a related note, the thread mentioned says to disable failover in IPv6 for the present by only having one router send RAs. I do not have RA enabled on any LAN interfaces in either pfSense machine, but both machines are sending out RAs with the correct prefix, the M and O flags set, and their respective addresses as recursive DNS server options. Fortunately, Windows clients are ignoring the DNS option even with SLAAC enabled as this would interfere with my DNS structure. Otherwise, these RAs contain exactly the information I want. However, I couldn't shut them off if I wanted to. Any idea what's causing this?

EchoReplyFailover2.png_thumb

databeestje

What you have created is a illegal configuration basically.

NPt only works if you have 1 pfSense box with 2 tunnels, that way pfSense keeps state and failover just works. What you are trying to do is just not possible with tunnels. Tunnels terminate on each pfSense, you would need something with a shared medium so both pfSense nodes can talk to both tunnels.

What you really want is BGP.

You need to make sure that RA is set to disabled on the dhcpv6 server page so that rtadvd isn't running, that way you can configure everything manually.

mdpugh

I am intrigued about a shared medium, so when I'm up to speed on BGP, I'll revisit the matter. ;)

It finally sank in. I was so focused on getting the server to respond through the same router (which I think is doable) that I totally lost sight of failover, i.e. the server is only reachable at a particular address through a particular router. So to get the equivalent of IPv4 CARP on the WAN side (which requires a shared public IP), I need a single public IPv6 reachable through either router (and this is where the analogy breaks down because I can't share a tunnel).

Now, you say NPt only works for one pfSense and multiple tunnels. What about the converse? If you have one pfSense box and multiple tunnels, is NPt the only way to make this work. Multi-WAN is virgin territory for me but, in light of what you've said, it seems this may be the case.

Finally, I stress that RA is disabled on my LANs, but both pfSense boxes are sending RAs anyway. I haven't minded because I wanted RAs on those links, but I can't turn them off.

databeestje

Ah, well to be fair, NPt does work for the traditional carp setup. Which is what I originally developed it for.

Both pfSense nodes need to be able to access both networks, NPt works fine that way no different from the NAT we have today on v4.
So just like you do with v4 multiwan and carp, both nodes share a single external and internal carp address on each WAN. Then NPt works fine.

So tunnels, ehm, no.

Not being able to disable RA is a bug. I need to fix that then.