XMLRPC restore_config_section Error

[SzyslaK]

Hello,
I recently deployed a HA pair of pfSenses using pfsync and XMLRPC Sync and everything was running fine for the first 3 weeks. All of a sudden I started seeing the following error after some performing some minor config changes to the MASTER. It seems to happen on any config change I do and it appears that all states are flushed when this happens.

4 php-fpm 8040 /rc.filter_synchronize: New alert found: A communications error occurred while attempting to call XMLRPC method restore_config_section:
Feb 13 14:25:54 php-fpm 8040 /rc.filter_synchronize: A communications error occurred while attempting to call XMLRPC method restore_config_section:
Feb 13 14:25:52 php-fpm 32319 /rc.filter_synchronize: XMLRPC reload data success with https://192.168.10.2:443/xmlrpc.php (pfsense.restore_config_section).
Feb 13 14:25:50 php-fpm 32319 /rc.filter_synchronize: Beginning XMLRPC sync data to https://192.168.10.2:443/xmlrpc.php.
Feb 13 14:25:50 php-fpm 32319 /rc.filter_synchronize: New alert found: A communications error occurred while attempting to call XMLRPC method restore_config_section:
Feb 13 14:25:50 php-fpm 32319 /rc.filter_synchronize: A communications error occurred while attempting to call XMLRPC method restore_config_section:
Feb 13 14:24:54 php-fpm 8040 /rc.filter_synchronize: Beginning XMLRPC sync data to https://192.168.10.2:443/xmlrpc.php.
Feb 13 14:24:54 php-fpm 8040 /rc.filter_synchronize: New alert found: A communications error occurred while attempting to call XMLRPC method restore_config_section:
Feb 13 14:24:54 php-fpm 8040 /rc.filter_synchronize: A communications error occurred while attempting to call XMLRPC method restore_config_section:
Feb 13 14:24:53 php-fpm 36920 /rc.restart_webgui: Creating rrd update script
Feb 13 14:24:51 check_reload_status webConfigurator restart in progress
Feb 13 14:24:51 php-fpm 43899 /system_advanced_admin.php: webConfigurator configuration has changed. Restarting webConfigurator.
Feb 13 14:24:50 php-fpm 32319 /rc.filter_synchronize: Beginning XMLRPC sync data to https://192.168.10.2:443/xmlrpc.php.
Feb 13 14:24:50 php-fpm 32319 /rc.filter_synchronize: XMLRPC versioncheck: 19.1 -- 19.1
Feb 13 14:24:50 php-fpm 32319 /rc.filter_synchronize: XMLRPC reload data success with https://192.168.10.2:443/xmlrpc.php (pfsense.host_firmware_version).
Feb 13 14:24:50 php-fpm 32319 /rc.filter_synchronize: Beginning XMLRPC sync data to https://192.168.10.2:443/xmlrpc.php.
Feb 13 14:24:49 check_reload_status Reloading filter
Feb 13 14:24:49 check_reload_status Syncing firewall

This is all I'm seeing in the System Log. Any ideas?

[jimp]

Any errors in the logs on the secondary node when that happens?

Do you maybe have the option enabled to kill states on gateway failure, and one of your gateways is marked down on the secondary node?

[SzyslaK]

@jimp I do have the option to kill states on gateway failure and yes one of the gateways is down on the secondary. How does this cause this behavior I am seeing?

[jimp]

When the first sync happened, it would trigger a filter reload. Filter reload sees the down gateway and resets states, terminating the connection currently used for XMLRPC. And if the connection gets re-used, it would fail as well.

Fix the gateway or turn off that option and you should be fine.

[SzyslaK]

@jimp Brilliant. Thank you for clarifying that.