Unable to Sync Rules without XMLRPC Code 2 error

sullrich

1. Both machines need to be on the same version
2. Both admin passwords must match
3. Both machines need to be either https or http, not mix matched.
4. Remove any special characters in descriptions

gjungle

@sullrich:

1. Both machines need to be on the same version
2. Both admin passwords must match
3. Both machines need to be either https or http, not mix matched.
4. Remove any special characters in descriptions

1. Yup, reloaded them from the same embedded image yesterday.
2. Yup, no special characters but upper case, lower case, and numbers.
3. Yup, both machines are https.
4. Yup, just went through the exported filter set and removed everything that may have been classed as a special character - including the following -,(,),>,=,;,:,, Then re-imported the rules into the master firewall. Now the only thing in the rules is alpha numeric characters.

Still get the same error when I try to sync the rules though :-\ :'(

sullrich

Ensure that you have pass rules between the two CARP sync members on either 80 or 443 or the custom webConfigurator port.

Also, both hosts need to be running on the same webConfigurator port.

gjungle

G'day Scott,

I followed the instructions (and just double-checked them) and the SYNC port has a any => any rule on it so all traffic can pass.

Also both hosts are running on the same webConfigurator port (https).

I can also get it to sync properly as long as I tell it not to sync the rules, which indicates that the synchronization settings themselves are correct…

I have now gone out and purchased a new cat 6 cross-over cable and tried that, but to no avail. It also takes a good 4+ minutes before it gives me the error - I'm wondering if there I'm hitting a timeout or something? I didn't think my 230 odd rules was excessive, maybe it is?

sullrich

Do you have special characters (non-ascii) in the description field of the rules? If so, remove them.

Even German glyphs(sp) if they exist.

gjungle

G'day again,

Nope, no special characters. Am situated in Australia and don't even have special characters on my keyboard :)

After reading a bit more on this forum I tried enabling the device polling on both boxes. This did make a change - now the error was back to what it was originally:

php:: New alert found: An error code was received while attempting XMLRPC sync with https://10.126.0.2:443 - Code 2: Invalid return payload: enable debugging to examine incoming payload
php:: An error code was received while attempting XMLRPC sync with https://10.126.0.2:443 - Code 2: Invalid return payload: enable debugging to examine incoming payload.

So I disabled the device polling and am now back at this error:

php: : A communications error occured while attempting XMLRPC sync with https://10.126.0.2:443.
php: : New alert found: A communications error occured while attempting XMLRPC sync with https://10.126.0.2:443.

Tomorrow I will try setting the communication to HTTP instead of HTTPS, and resetting the passwords on both boxes to something simple like 'pfsense'.

Let me know if you have any other thoughts :-\

sullrich

Please send your config.xml to sullrich@gmail.com

gjungle

I'll send it first thing tomorrow when I get to work..

sullrich

Before you send and just for grins, reboot the secondary CARP cluster member and try to sync again.

gjungle

I've rebooted both of them multiple times whilst making changes and verifying the situation to no effect :-\

However this morning I have had a breakthrough! In reconfiguring the firewall in preparation for sending you the config.xml, I found that it worked if I set the protocol on both firewalls to HTTP. Just to verify I changed the protocol on both firewally back to HTTPS and sure enough got the "communications error" again when trying to sync.

I'll reset the password to a more cryptic one and re-enable device-polling in HTTP mode to see if it still works.

At least we've managed to narrow it down a bit further… did you still want a copy of my config.xml?

sullrich

Did you change the admin username?

http://wiki.pfsense.com/wikka.php?wakka=CARPConfigurationSyncTroubleShooting

gjungle

No, I didn't even know you could change the admin username :o

smurfb

I got the same problem. Running 1.2-RC2. Did you guys ever figure out what was wrong?

superwutze

i recently updated 2 carp-firewalls from 1.0.1 to 1.2-rc2 (because of the failover-pool-feature) and am now stuck with the same problem:

php: : An error code was received while attempting XMLRPC sync with username admin https://10.10.11.252:8443 - Code 2: Invalid return payload: enable debugging to examine incoming payload

as long as there are no changes on node1 the sync works, states get synced, carp works, but when changes are made then the error shows up. but the states sync on. so live carp works, changes in rules or aliases not.

i tried all the above to no success.
any new ideas?

thanks in advance,
andy

nic

I too am having this problem. 1.2-RC2, recent upgrade. We have tried everything else listed above and are unable to get anything other than:
Oct 2 16:28:00 pri php: : Beginning XMLRPC sync to http://192.168.255.2:80.
Oct 2 16:28:00 pri php: : An error code was received while attempting XMLRPC sync with username admin http://192.168.255.2:80 - Code 2: Invalid return payload: enable debugging to examine incoming payload
Oct 2 16:28:00 pri php: : New alert found: An error code was received while attempting XMLRPC sync with username admin http://192.168.255.2:80 - Code 2: Invalid return payload: enable debugging to examine incoming payload

We have tried HTTP as well as HTTPS. We have checked the user name and password. We have no luck with sync of rules turned on or off.
Our question is how do we "enable debugging to examine incoming payload"? We see know way to do this, and cannot find mention in the documentation. We have tried changing the default setting for debug in the class constructor for XML_RPC_Client and in the "new" call for XML_RPC_Server (in xmlrpc.php). We are not even sure where we should expect to see this debugging information emerge.

Can anyone provide some guidance on this?

Thanks in advance,
-nic

cmb

nic: Run this from a SSH session:

tcpdump -i fxp0 -s 1515 -tttt -w /tmp/sync.pcap src or dst 192.168.255.2

replace fxp0 with the interface you're using to sync. Then go to your CARP Settings page, verify your settings, and click Save. Wait a couple minutes, check your logs and make sure it's failed, and go back to your SSH session and hit ctrl-c.

Then go to exec.php and download /tmp/sync.pcap and email (cmb at pfsense dot org) it to me.

superwutze, I'd ask you to do the same but you're using HTTPS so examining the network traffic to find the underlying cause isn't possible.

sullrich

http://devwiki.pfsense.org/CARPConfigurationSyncTroubleShooting

superwutze

i tried http/https, various ports and passwords, various carp-configurations (what to sync) and so on.
the link to the wiki was already posted above and i considered it carefully but to no success.

a note to special characters: the default generated rules already contain '-' in their description, also the aliases get comments added with timestamps in them containing ':'. so i guess those characters are ok (but i have non other than [[:alnum:]] in my own rules and descriptions, not even blanks).