HA XMLRPC sync appears to “merge” but does not actually write changes on the Backup

w0w

I will not be able to try a minimal configuration in the near future, unfortunately. But I might have some time to dig a bit deeper. For now, at least I am sure that the receiving side receives everything, and I can even see my test rule in the dump, but it is not clear why the block is not being written… Maybe it is failing a validation check.

marcosm

If you're referring to the changes from pfBlockerNG then it's likely the cron thing already mentioned. Otherwise something else to try is temporarily removing packages from both nodes and testing.

w0w

@marcosm
I have removed only pfBlocker, and the configuration has synced successfully.

w0w

Even with synchronization completely disabled, simply having pfBlocker installed prevents synchronization between the firewalls.

SteveITS

@w0w oh do you mean any change, not just pfB? Then disregard my post above. That’s only pfB.

w0w

@SteveITS said in HA XMLRPC sync appears to “merge” but does not actually write changes on the Backup:

do you mean any change, not just pfB?

Exactly. Anyway it looks like this bug is related to pfB somehow.

w0w

It looks like config sync stops working when pfBlocker is installed on the secondary node. Even if I completely remove all pfBlocker settings, on a new install sync still stops, even when pfBlocker is not configured at all.

w0w

<package>
			<name>pfBlockerNG</name>
			<descr><![CDATA[Manage IPv4/v6 List Sources into &#039;Deny, Permit or Match&#039; formats.&lt;br /&gt;
			GeoIP database by MaxMind Inc. (GeoLite2 Free version).&lt;br /&gt;
			De-Duplication, Suppression, and Reputation enhancements.&lt;br /&gt;
			Provision to download from diverse List formats.&lt;br /&gt;
			Advanced Integration for Proofpoint ET IQRisk IP Reputation Threat Sources.&lt;br /&gt;
			Domain Name (DNSBL) blocking via Unbound DNS Resolver.]]></descr>
			<pkginfolink>https://docs.netgate.com/pfsense/en/latest/packages/pfblocker.html</pkginfolink>
			<version>3.2.9_1</version>
			<configurationfile>pfblockerng.xml</configurationfile>
			<include_file>/usr/local/pkg/pfblockerng/pfblockerng.inc</include_file>
			<plugins>
				<item>
					<type>plugin_xmlrpc_send</type>
				</item>
				<item>
					<type>plugin_xmlrpc_recv</type>
				</item>
			</plugins>
		</package>

If I remove the section shown below on the secondary firewall, sync starts working again immediately.

			<plugins>
				<item>
					<type>plugin_xmlrpc_send</type>
				</item>
				<item>
					<type>plugin_xmlrpc_recv</type>
				</item>
			</plugins>

w0w

This post is deleted!

w0w

I have ended up modifying function function pkg_call_plugins from

/etc/inc/pfsense-utils.inc

During XMLRPC config merges the core calls pkg_call_plugins("plugin_xmlrpc_recv", $pkg_sections). In stock code this invokes every installed package that registers a plugin_xmlrpc_recv* handler, regardless of whether that package’s own config is present in the incoming payload. That means unrelated plugins (e.g., pfBlockerNG on the secondary) can run on every merge and interfere with sync.

I modified pkg_call_plugins() in /etc/inc/pfsense-utils.inc to add one small guard:

For plugin_xmlrpc_recv* calls only, call a package’s handler iff the XMLRPC payload contains installedpackages/<pkgname>* keys for that package.

All other plugin types (send, normal hooks, etc.) are untouched.

In other words: only the packages whose config is actually being synced get their recv-hook called. This is just a workaround. Please don’t judge me too harshly.

function pkg_call_plugins($plugin_type, $plugin_params) {
    $results = array();
    $is_recv = (strncmp($plugin_type, 'plugin_xmlrpc_recv', 19) === 0);

    foreach ((array)config_get_path('installedpackages/package', []) as $package) {
        $items = (array)array_get_path($package, 'plugins/item', []);
        if (empty($items)) {
            continue;
        }

        // Derive package code name from configuration file (e.g. "pfblockerng.xml" -> "pfblockerng")
        $cfg = $package['configurationfile'] ?? '';
        $pkgname = $cfg ? substr(reverse_strrchr($cfg, '.'), 0, -1) : ($package['name'] ?? '');

        foreach ($items as $plugin) {
            if (!is_array($plugin) || (($plugin['type'] ?? '') !== $plugin_type)) {
                continue;
            }

            // Minimal guard: on XMLRPC receive, only call the plugin if payload has installedpackages/<pkgname>*
            if ($is_recv) {
                $ip = $plugin_params['installedpackages'] ?? null;
                $has_match = false;
                if (is_array($ip)) {
                    foreach ($ip as $k => $_) {
                        if (strncmp($k, $pkgname, strlen($pkgname)) === 0) {
                            $has_match = true;
                            break;
                        }
                    }
                }
                if (!$has_match) {
                    continue;
                }
            }

            $inc = $package['include_file'] ?? '';
            if (!$inc || !file_exists($inc)) {
                continue;
            }
            require_once($inc);

            $fn = $pkgname . '_' . $plugin_type;
            if (function_exists($fn)) {
                $results[$pkgname] = call_user_func($fn, $plugin_params);
            }
        }
    }

    return $results;
}