Update to pfSense packages with XMLRPC sync functionality

[marcosm]

Some quick background:
There are several packages that implement XMLRPC sync functionality which allows the primary node in a High Availability system to sync its configuration with the secondary node. Some packages even allow entering addresses manually to send the configuration to multiple nodes, or even chain the syncs from node to node. The current XMLRPC sync functionality for packages works separately from the normal config sync that happens between HA nodes; because of this there's a potential race between the system's config sync and the package config sync.

New:
A few new XMLRPC plugins have been introduced which address the race condition and improve maintainability - see here. Packages can use these plugins to hook into the configuration sync process that happens in High Availability setups.

Changes:
Packages that implement their own XMLRPC code are being updated with the following changes:

• The "auto" sync code, if applicable, will instead use the new plugins.
• XMLRPC sync code for the (very) old pfSense versions (older than 2.4) will be removed.
• If applicable, additional functionality in the package (e.g. manual/chained config sync and file sync) will be kept as is to minimize changes and potential issues.

These changes should take effect starting with pfSense software releases after CE 2.8.0 and Plus 25.03.

[RobbieTT]

@marcosm

The HA and non-HA setups could also be improved by a shorter boot time for recovery. I've never really understood why pfSense is so slow to boot, even with high-end hardware.

Can we chop a minute or two off the reboot time as a project in its own right, to help reduce downtime and/or restore HA redundancy as fast as possible?

️

[stephenw10]

Yup it probably could be improved. There are a number of deliberate delays that were added in the past to allow for various hardware quirks. Some of them may no longer be required.

However making changes like that will almost inevitably break something for somebody! So we'd likely need to make it optional.

[LukasInCloud]

I’d advise running parallel sync tests to ensure the plugins don’t choke under load.

@RobbieTT, on boot times: have you tried tweaking the loader.conf to cut delays, or is it mostly hardware quirks causing the slowdown?

[RobbieTT]

@LukasInCloud said in Update to pfSense packages with XMLRPC sync functionality:

@RobbieTT, on boot times: have you tried tweaking the loader.conf to cut delays, or is it mostly hardware quirks causing the slowdown?

I'm running on a Supermicro Xeon D-1736NT network box that is well supported by pfSense and has similar interfaces to current Netgate products. It has a little more to load I guess with more hardware acceleration support available (eg QAT) but it is running on bare metal. It's fast, has 32GB of RAM and Optane storage to pull from at very low latency.

I also have a Netgate 6100, on Optane, that boots in a similar time to the more powerful Xeon unit. I'm not aware of any hardware quirks with either device but I guess both are vulnerable to running code that caters for old hardware or to address problems long since vanished.

My loader.conf is pretty close to original, if not actually original given the recent pppoe changes. I'm not aware of any loader / system tuneable tweaks that could make pfSense boot faster but always receptive to anything that can reduce downtime on reboots, especially when running betas or dev loads.

️

[LukasInCloud]

@RobbieTT You're right that boot time isn't really about hardware anymore, especially on systems like yours. Many of those legacy delays in pfSense probably don't apply to modern setups and could be safely skipped. It might be worth trying a trimmed-down loader.conf just to see how much it helps, especially if you're running dev builds often.

[RobbieTT]

@LukasInCloud
Thanks but I would not know where to start to help trim the boot time. Is there a guide somewhere or a resource I could lean on?

️