@gringorion said in a communications error occurred while attempting to call xmlrpc method host_firmware_version: request timed out due to default_socket_timeout php.ini setting:

pfsense 2.6.0-RELEASE (amd64)

Hello,
We have worked a lot with Netgate support but there is no improvement.
In 15 days, I have to migrate my 7100 to version 22.05 and I will see if that solved the problem.
I come back to you to give you news

Well, it looks like my expectations about the self-protection were wrong! I found in the system logs of the pfSense firewalls that it was flagging the checks from zabbix as an attack, and would periodically block all access from the zabbix server IP. I was able to whitelist that IP from the login protections, and I haven't seen any issues since. I still have no idea why this issue only manifested for the backup firewalls and not the master ones, seeing as their configurations are nearly identical, but hopefully this helps someone else in the future!

Hello
Like said on documentation :
pfsync Synchronize Peer IP

If left blank, the firewall will send state data using multicast to all hosts on the chosen Synchronize Interface.

In practice, state synchronization is more reliable when sent directly and not via multicast.

@empbilly said in Compatibility between VRRP and CARP:

The vlans I have are in a lagg with 4 physical interfaces.

Would this be a problem?

No. In former pfSense versions the network ports for a (virtual) network interface have to be the same same on both nodes. E.g. the port for VLAN 305 has to be lagg0.305 on both.

Configuring a lagg was a way to achieve this if the hardware was different.
But as far as I know, this is not necessary anymore since FreeBSD 12. However, I configured it only this way.

Do I need to have one network (10.10.10.0/24) or can it be one IP only (10.10.10.1) for each VIP in the vlans?

You have to configure each IP and as well the VIP with the correct mask.

I have the vlan ADM_LAN with the network 10.60.0.0/23 and GW 10.60.0.1

On pfsense backup can I put the GW 10.60.0.2?

If you have 10.60.0.1 already configured as gateway on all your internal machines it might be easier to turn this into the CARP VIP and change the interface IP on the primary to anything other, maybe 10.60.0.2 and use 10.60.0.3 for the secondary.

Another point is that we have an AD in our infrastructure, and the AD IP is the DNS in some vlans. How does this work with VIP?

This has nothing to do with HA. It should work like before.
Maybe I'm getting you wrong?

@viragomann Or put the pfblocker file on an inside network that both nodes have ready access to. Sync it to a reachable server or something.

@thale

BTW:

CARP IP on LAN interface works fine and no any issues.
The packet loss issue is only happened on CARP IP on WAN interface.

@nikim
Simply hit "Add new patch", enter a description like "CRL lifetime fix" and the patch ID below and save it.

pfSense will pull and apply the patch then.

Thank you. I thought it is correct behavior, just wanted to confirm.

@william-mandell I'm guessing one is a WAN IP or other interface, since it's the same device?

The traffic graphs use some level of smoothing so they are probably just being generated enough apart to appear different.

Is there a second one? (you posted this in the HA subforum...)

@spunky_surveyor It appears that even if you specify

listening_ip=eth0/24

in

/var/etc/miniupnpd.conf

it won't bind to the CARP VIP.

As a result UPnP will work with some applications that don't mind the fact that the router IP advertises itself. But NAT-PMP and many others will fail because the VIP isn't getting picked up by the miniupnp daemon. This appears to be fixed in miniupnp upstream and is an old bug in PFSense due to an ancient historical lack of multicast support in CARP VIPs.

A workaround for NAT-PMP is to create a NAT Port Forward for:

CARP IP : UDP 5351 to Router IP : UPD 5351

@decibel83 said in CARP VIPs with different states on secondary firewall:

Anyway I expect that VIPs with the same vhid should have the same status on a single firewall.

No, that's not how it works. The VHID has nothing to do with anything on different broadcast domains.

You should probably read and understand this:

https://forum.netgate.com/post/719523

Can anyone give some pointer on this?

@mrpete

I also have a century link connection that runs on VLAN 201. I currently have the modem in bridge mode and have PFSense taking care of the log in.

I am currently struggling with setting up the CARP properly on the boxes. Do you have a guide that I could follow?

@wifi75 Up until relatively recently pfSense needed the same hardware on both in order to sync states. However as of 22.01/2.6, that's no longer a requirement. So it should be possible to use any hardware.
https://docs.netgate.com/pfsense/en/latest/highavailability/pfsync.html#pfsync-and-physical-interfaces

Well alllllrighty then haha. Thanks for the quick reply!

So from an old ticket:

https://redmine.pfsense.org/issues/7010?tab=notes

I'm confused why aliases on loopback interfaces would need a sync for HA cluster

I think there needs to be some work done e.a redesign of the whole xmlrpc process thing.
I could easily see times that one firewall is broken and it takes weeks to perhaps months ( depending on supply of hardware vendor ) to get replaced and sycing can be moved back to original primary device.

There should become an option to track changes on secondary device and have information tracking on primary device and as soon primary comes online there should become an option to sync the rules between devices.

So basically what I am saying here is that a secondary node should have more involvement in this whole xmlrpc config process.

Like there should also become an option when primary comes back online you can still keep the secondary running as the main firewall rule util you are sure the primary firewall is working correctly again.

Just my 2 cents of thoughts.

Since CARP does not work on cloud virtual environments (AWS, Google, Oracle cloud, etc), is there any other way to make pfSense work in HA configuration for cloud environments?
If not, is there any plan to make HA cloud configuration to work in the near future?

@spectre-988
The clients use for DNS, what you tell them to use.
Enter the CARP IP as DNS server, and the will send request to it.

If they are configured by DHCP, tell the DHCP to send the CARP IP for DNS.
In pfSense DHCP server you can enter it at "DNS servers".

Well, it can be done, with minimal changes.
You need to change local ip's and make ha ones as vip
Not a big thing

But, do keep in mind that all interfaces have to be created in the same order in both ha instances.

You will need some experience with the ha setup.
Many things can go wrong if you don't know what you are doing. (as is usually the case too)

I strongly suggest to setup a lab and experiment with ha setup. When you will feel confident, you can proceed with the real thing.
Doing such chores on a live system without prior experience will probably cause significant downtime.

@dara said in pfSense CARP + Cisco N5k vPC:

@philippe-richard Hi Philippe, Thanks a lot. This is more complete and interesting than our setup.

I wonder how you configured the connection between the routers and switches?

In my setup, each router has a single connection to a single switch configured as an Orphan port. For now it is working perfectly.

I am not sure however how it will handle different link and device failure scenarios but I will test it sometime soon and post my findings here.

Hello, have you made progress on your configuration?
Have a good day

Could someone post an example for the necessary NAT rule(s), please?
EDIT: got it already, at least I think so 😊

@neilewing When an interface with a CARP VIP loses carrier, all VIPs on that host are demoted. This makes the VIPs on the other node "better" and the rest of the VIPs on the first node swing to BACKUP status (because they see the "better" advertisements) and the ones on the backup node assume MASTER (because they see that they are the "best" VIP status).

@viragomann We indeed had very strange routing issues on the location the pfSense instances are deployed. It's really nothing wrong with them but we had a strange situation in combination with our WAN Switches and the LACP upstream to the provider.

OpenVPN to the CARP Address is now running stable.

@pfsense7515 Need to look at what caused that. That is only part of the event. There are also logs like links going down and up, etc.

@derelict
IT WORKS!
Thank you

@brunoroza If that is really the case then your switch is likely not properly passing the CARP advertisements. They are multicast to 224.0.0.18.

20:17:32.490656 IP 172.25.228.18 > 224.0.0.18: CARPv2-advertise 36: vhid=228 advbase=1 advskew=0 authlen=7 counter=2770184658337638700

If those are not received by the secondary node, it will also become MASTER and begin advertising its CARP VIP.

Update: after turning the whole infrastructure from left to right we found the solution.
It's the limiter bug that is already known. After removing the limiter from the firewall rule (it was a just one catch all rule for the whole NAT traffic), it works as before.
Which also means: the same setting worked perfectly fine before the upgrade.

I am some much hoping for a soon fix of the limiters in an official update or release!

@m4rek11 After applying the patches, I did not notice that the routers changed the roles of Master-> Backup, Backup-> Master.
All the problems went with those when I made any changes to the rules, dns or DHCP.

I found my configuration error early. For unknown reason, for 2 different networks I sent the same vhid for Virtual IP. But the problems were still there. After applying the patches, the problem was gone.

@digger30 Perfect! Glad I could be of assistance.