HEADS UP: relayd deprecated on pfSense 2.5.0

[jimp]

[EDIT: relayd has been removed from 2.5.0, see posts below]

The built-in load balancer daemon, relayd, does not work with OpenSSL 1.1.x which is now the default on FreeBSD 12.x and thus on pfSense 2.5.0 development snapshots. It will take significant work to address its issues, and the service does not see much use these days compared to the more robust and feature-filled HAProxy package.

At this time we are leaning toward deprecating the relayd-based load balancing service. Most, if not all, scenarios possible in relayd can be done in superior ways in HAProxy.

A few notable differences with respect to converting from relayd:

• HAProxy is more complex, thus can be a little trickier for beginners to configure
• HAProxy is a true proxy, not NAT-based, thus may consume more resources
• HAProxy will need IP Alias VIPs to bind in some cases that relayd did not, since relayd worked via NAT.
• HAProxy may conflict with the WebGUI if both are set to bind to port 443, though it's trivial to work around either by binding HAProxy to an alternate port and using NAT rules to redirect external traffic, or by moving the GUI to another port.
• HAProxy does not preserve the client IP address by default, but has an option to do so (Transparent Client IP). In most cases this is not necessary as HAProxy can add headers such as X-Forwarded-For which can be used in logging on the web server or by applications on the web server to achieve similar effects

There are more details and a comparison in the Server Load Balancing hangout at https://www.netgate.com/resources/videos/server-load-balancing-on-pfsense-24.html and the slides at https://www.slideshare.net/NetgateUSA/server-load-balancing-on-pfsense-24-pfsense-hangout-july-2017

It may be possible for us to write some code to import settings from relayd into haproxy as well, but it is not likely to be fully automated for all cases.

The decision is not yet final, however. If you use relayd but absolutely cannot use HAProxy, now is the time to speak up with specifics so we can take these scenarios into account.

If you are not sure if you can use HAProxy, now is the time to try it out. Replicate existing relayd setups in HAProxy and post on the cache/proxy category if you need assistance. You can do this now on any release, no need to wait for snapshots.

For those affected, if you convert your setup over to HAProxy before upgrading to 2.5.0, then there will be no downtime due to the change.

[jimp]

The FreeBSD port is broken, the upstream repository hasn't seen an update in almost 5 years, and there is no sign of anyone working to port it to the new OpenSSL. Thus, relayd is currently slated to be removed from 2.5.0.

https://redmine.pfsense.org/issues/9386

[jimp]

FYI- relayd has now been removed from master (pfSense 2.5.0)

https://github.com/pfsense/pfsense/commit/586c623a943f59486a461c1af9873dd6cc11a3b3

[kossie]

@jimp
Would the removal of "relayd" effect Dual WAN load balancing that's currently integrated into pfsense?

[jimp]

@kossie said in HEADS UP: relayd deprecated on pfSense 2.5.0:

@jimp
Would the removal of "relayd" effect Dual WAN load balancing that's currently integrated into pfsense?

No, that never used relayd. This is server load balancing only.

[kossie]

@jimp

Perfect thanks for the quick reply! I look forward to testing 2.5.0 when released.

[adam65535]

It looks like I posted originally in the wrong section.

https://forum.netgate.com/topic/142176/relayd-going-away-in-2-5-0

haproxy unfortunately doesn't have the one big feature relayd has... established connections are not broken when switching to the backup firewall. We have some applications that get load balanced that have long living connections and 24/7 use that customers will get disconnected if an established connection breaks. Relayd was a nice solution for that.

Anyone know if this will ever return if relayd gets fixed or anyone else with similar requirements have a plan when 2.5.0 is released?

[jimp]

At this point it is unlikely to return. The upstream code hasn't been touched in years, and I don't see any sign of anyone working on it. It's not something we have the resources to take on, either.

Pretty much all the solutions out there now are proxies. I don't think there is anything left that would work purely with NAT.

If you only need load balancing and not failover or taking out servers from the pool, you could just make a port forward NAT rule with an alias for the target. Not fancy, but it may get the job done. Problem is that it can't handle server failures. Maybe with some custom scripting to monitor and add/remove from the target table... Still seems like a lot of room for error.

Failover with haproxy should be pretty quick. If your application can't handle a minor TCP connection hiccup gracefully, it probably has more problems than reconnecting on failover, though. You're far more likely to have the connection die due to random Internet happenings than failover.

[adam65535]

I will research the alias option with port forward NAT. That might be ok for some scenarios. Thanks for that info. I will certainly miss relayd's failover connection handling. It worked well for our scenario.

[skullnobrains]

there is another reason for keeping relayd or an equivalent firewall-level service

i believe running and exposing extra services on a firewall to be a potential security hole. this is especially true since haproxy is not isolated in a jail nor reasonably protected. and anyway, attacks such as hartbleed clearly demonstrated this offers a whole lot of other L4 protocol-based attacks

... and i do use haproxy since the earliest versions and really love it. actually i use relayd in front of haproxy quite often : the job of the firewall is to detect a dead haproxy. the job of haproxy ( or nginx or lighttpd or whatever ) is to perform better L7 health checks and possibly replay dead queries on a different server when needed.

using aliases does work, but afaik pf does not properly fail over to a different server when needed. additionally, there is no failover pool

[jimp]

Actually that's not an argument for keeping a service like relayd, it's an argument for not having anything like it on the firewall at all. If you are concerned, then the service should be performed by a dedicated and isolated server, not by the firewall.

[skullnobrains]

sure. but how are we supposed to achieve redundancy ? obviously carp, vrrp and the likes come to mind but i believe it is much more efficient to have minimal intelligence on the firewall level so we can use multiple independent proxies efficiently, and possibly running differnet operating systems and software on different isolated networks. it is also very helpful during migration tasks to achieve complete transparency.

[JeGr]

What does CARP has to do with security on the firewall platform? Or with additional services like relayd or haproxy? CARP is protocol and low level services, how'd you want to "exploit" that from WAN?
If you're arguing (and that is OK and depends on your level of security you want to define) that additional services on the firewall should be minimal, then yes, you'd have to put them in some sort of DMZ behind the (first) filter/gateway. I don't get the relation between stuff like CARP or L4 stuff like relayd or even L7 proxies. That's all over the place ;)

[jimp]

Those are all topics for another thread. You would achieve it the same way you would for any other service off the firewall.

I've locked this thread as it's purpose has been served.