Does WireGuard work in a High Availability (pfsync, "CARP") mirrored firewall environment?
-
If I set up two pfSense firewalls in parallel as a High Availability ("CARP") cluster, and I install WireGuard with identical configurations on both, will a WireGuard tunnel served by the firewall stay in sync if the firewall switches from one appliance to the other?
-
@rich-w No, as WG is currently not "aware" of who is master or backup and as it isn't "bound" to an interface or specific IP, AFAIK the current implementation isn't really compatible out of the box with that. The consensus currently is, that to have a "failover" like scenario, you'd have to create two separate tunnels from the other side and use a higher level routing protocol to route either way (e.g. using FRR/OSPF on top). But I somewhere read around the forums, that there are patches in testing(?) to make WG aware of a CARP address and if the peer is configured to use that to answer with that interface/IP instead of using/falling back to the interface IP instead.
When that is working a more "normal" (as in like every other VPN) way of using might be possible (as the secondary node doesn't get the packets sent for the CARP IP).
And the current service would have to be made CARP aware so it's only started when the node is primary/active. Don't know if @cmcdonald already included that in the current state.Cheers
-
@jegr said in Does WireGuard work in a High Availability (pfsync, "CARP") mirrored firewall environment?:
@rich-w No, as WG is currently not "aware" of who is master or backup and as it isn't "bound" to an interface or specific IP, AFAIK the current implementation isn't really compatible out of the box with that. The consensus currently is, that to have a "failover" like scenario, you'd have to create two separate tunnels from the other side and use a higher level routing protocol to route either way (e.g. using FRR/OSPF on top). But I somewhere read around the forums, that there are patches in testing(?) to make WG aware of a CARP address and if the peer is configured to use that to answer with that interface/IP instead of using/falling back to the interface IP instead.
When that is working a more "normal" (as in like every other VPN) way of using might be possible (as the secondary node doesn't get the packets sent for the CARP IP).
And the current service would have to be made CARP aware so it's only started when the node is primary/active. Don't know if @cmcdonald already included that in the current state.Cheers
Do you know if these patches has been applied yet?
-
Any Updates to pfsync for Wireguard in year 2024?
-
Along with the ability to restore Wireguard information separately, I want this feature so bad as well that I might visit forum once a week to keep this post on the top page...
-
I proposed a feature request on Netgate's Redmine for this very feature.
I hope someone at Netgate reviews it in the future.
https://redmine.pfsense.org/issues/15554
-
Just Checking to see if anything happened in the last few days.
-
I do not believe that CARP and WireGuard have to technologically align perfectly for failover of WireGuard functionality to occur.
The beauty of software engineering is making things happen despite limitations in the underlying technologies.
pfSense is in charge of running components like WireGuard and CARP. Therefore, pfSense should be aware of what is happening in the cluster and could compensate for WireGuard's limitation of not knowing states.
So, I am checking again to see if anything interesting has happened...
-
I too would love to see this happen using CARP. Seems to me that keeping the wireguard configuration sync'd and let the passive pfsense monitor the services on the active. If it's not responding then should be matter of starting the services on the passive to make it active.
Wireguard wouldn't care about connection status on the wg's interface since it's a passive protocol. During switchover wg peers shouldn't notice anything.
-
Checking again to see if there was any news in the last 9 days. :)
-
@Darkk said in Does WireGuard work in a High Availability (pfsync, "CARP") mirrored firewall environment?:
If it's not responding then should be matter of starting the services on the passive to make it active.
That would be bad. A passive device in a CARP cluster (especially a master/backup setup) should never actively initiate any kind of service bar some deep routed network stuff (e.g. FRR for OSPF etc. in some cases but even then, often it's better to stop them and only start if you're the active node). Otherwise your VPN would terminate on the standby node that isn't the gateway of your networks and you'd be stranded. So NO.
But that's not even the problem. A trigger like "if you are active node, start WG" isn't the problem here but wireguards inherent lack of using interfaces at all. When you're not using interfaces or IPs when hovering in kernel space you can't easily force the dumb process to use the CARP IP on e.g. WAN1 to use it for outgoing connections and with WGs way of just switching IPs as it likes you have quite a nice problem at hand to handle getting it matched to a specific WAN (e.g. run only on WAN2 but if that fails switch to WAN1). That's stuff that works with IPsec or OpenVPN without a hitch because both are bound to interfaces or IPs, WG is not. "Just make things happen" won't do easily when the whole service is written around don't caring a flying frack about the correct IP to use. That's why its not just a "make it happen" thing to do. At least not, when you want it to function well in a cluster where you want it to "just work" and not "fumbling around to make it happen somehow in case of a failure".
If the latter one is enough for you, you can make that partially happen right now with a lot of floating rules that deny traffic in/out via specific port/IP combos. But that isn't how one normally would like a VPN service in a cluster/HA setup to work as it should be done smoothly and switching from A to B or IP 1 to 2. And that's the hard(er) thing. And with OpenVPN DCO and the performance boost of it, it gets less important anyways.
Cheers
\jens -
@JeGr said in Does WireGuard work in a High Availability (pfsync, "CARP") mirrored firewall environment?:
(e.g. FRR for OSPF etc. in some cases but even then, often it's better to stop them and only start if you're the active node).
-
Can the same logic be applied to Wireguard?
-
If CARP is capable of judging who is active and who is standby, can this be used as a signal for where to run one instance of Wireguard and kill all the other Wireguard processes in the cluster?
-
IPSec and OpenVPN could tell who is active and who is standby because they are bound to interfaces, right? So those two VPN protocols can form a cluster without relying on something like CARP because they are bound to interfaces, right?
-
Then what about making Wireguard dependent on CARP and run only single instance of Wireguard where CARP status is confirmed to be active?
-
I know that VyOS is doing something similar to this. They combine VRRP and transition scripts to do this to make sure that if a node becomes a VRRP master, Wireguard comes up and if you are not a master anymore, kill wireguard.
-
If something like what is mentioned in #4 and #5 is not possible with PfSense,
Is it because PfSense software itself is limited in some software designs?
or Is it because CARP has inherent limitations and is different from VRRP?- Or Something like #4 and #5 is possible with PfSense but there are other problems when doing so, may be?
Regards,
-
-
@prudentcircle said in Does WireGuard work in a High Availability (pfsync, "CARP") mirrored firewall environment?:
Can the same logic be applied to Wireguard?
IPsec or OpenVPN are doing that, if a node is standby and it's configured on a CARP IP. But as Wireguard does not have an interface binding it's a bit more complicated.
@prudentcircle said in Does WireGuard work in a High Availability (pfsync, "CARP") mirrored firewall environment?:
If CARP is capable of judging who is active and who is standby, can this be used as a signal for where to run one instance of Wireguard and kill all the other Wireguard processes in the cluster?
Perhaps - I'm not sure.
@prudentcircle said in Does WireGuard work in a High Availability (pfsync, "CARP") mirrored firewall environment?:
IPSec and OpenVPN could tell who is active and who is standby because they are bound to interfaces, right? So those two VPN protocols can form a cluster without relying on something like CARP because they are bound to interfaces, right?
To the first part: yes. To the second: I don't understand what you mean by forming a cluster without relying on CARP etc. A cluster is a cluster because of things like CARP, keepalived or stuff. What do you mean by "form a cluster without relying on sth like CARP"?
@prudentcircle said in Does WireGuard work in a High Availability (pfsync, "CARP") mirrored firewall environment?:
Then what about making Wireguard dependent on CARP and run only single instance of Wireguard where CARP status is confirmed to be active?
Wouldn't change a thing as the problem with Wireguard still remains, that it is interface-agnostic and doesn't bind to the VIP (virtual IP) of a cluster. You simply don't want Wireguard to use your interface IP instead of the cluster IP as your communication would always come from the wrong IP and you can't that easily set it up to work on a fixed interface.
@prudentcircle said in Does WireGuard work in a High Availability (pfsync, "CARP") mirrored firewall environment?:
I know that VyOS is doing something similar to this. They combine VRRP and transition scripts to do this to make sure that if a node becomes a VRRP master, Wireguard comes up and if you are not a master anymore, kill wireguard.
Could maybe work. Still don't know how they'd treat WG to fix it's tendency to use the wrong IP or wrong interface though.
@prudentcircle said in Does WireGuard work in a High Availability (pfsync, "CARP") mirrored firewall environment?:
Is it because PfSense software itself is limited in some software designs?
or Is it because CARP has inherent limitations and is different from VRRP?Nothing to do with FreeBSD or pfSense, wireguard is simply weird that way. And as I don't know what VyOs does with keepalived and if it's really using VRRP and runs WG only on the VRRP IP - I can't say that.
Also check that post in VyOS forums, that describes exactly what I said. Wireguard simply ignores the VRRP interface and communicates via the physical IP what you don't want in a cluster:
-> https://forum.vyos.io/t/wireguard-does-not-work-with-vrrp-ip-address/14909
