@cmcologne said in Only VIP no Interface IP:

Why is it not possible to only assign a CARP-IP to a Interface?

Cause the interfaces which are sharing a CARP VIP must be able to communicate over Layer 3. So they need to have a unique IP each within a common subnet.

It's possible to assign IPs out of a private subnet to the interfaces with some drawbacks.
You may find threads discussing that topic here in this forum when you search for "CARP with only one public IP".

@jimp Brilliant. Thank you for clarifying that.

@jimp said in Sync error with packages since today:

That's just one possibility, but something to consider.

Absolutely, thanks. As this was some elevated by my boss because of the constant nagging ;) I can report, that Paighton from Support has found it out. To my surprise our old FreeRadius configuration (since the FR2 package times) contained a manual sync setting instead of using the systems sync (which would be the right way but I can remember it sometimes being bugged in the beginning). So as we switched UI Port a few weeks ago we never had any problem until there was a request for a new customer VPN server and Radius User. Didn't see that coming and perhaps would have found it in the end after debugging hours, but happy to say that support got it faster :) So always check your packages that allow syncing to the cluster peer and make sure the sync is using the right ip/port/credentials or is using the system ones in the first place :)

Should have found that myself, but sometimes especially in your own setup environments you get stuck in a rut... In a customer setup I'm fairly certain we would've found that ;)

I created a new vm, assigned the offending IP to that VM, let it hang out for a day, deleted the VM and the DNS records in Infoblox. Finally, after the weekend the record has cleared in Infoblox and presumably the switch.

Switch weirdness in the end, I suppose.

Thanks to Derelict for the suggestions in trying to track this down.

That's called out in the documentation in several places

https://docs.netgate.com/pfsense/en/latest/book/highavailability/high-availability-troubleshooting.html#issues-inside-of-virtual-machines-esx

https://docs.netgate.com/pfsense/en/latest/highavailability/configuring-high-availability.html#vmware-esx-users

https://docs.netgate.com/pfsense/en/latest/highavailability/troubleshooting-high-availability-clusters.html#hypervisor-users-especially-vmware-esx-esxi

[...]

As long as the L1/L2 between them is clean and fast that should be OK. You can adjust advbase if it isn't. With a higher advbase it won't fail over as fast, but it is more tolerate of latency between the nodes. Having them in different places in the same building is not unheard of, or even separate buildings on the same campus. It's not ideal but it can work. I think we even had someone try to use CARP for HA to a node in a DC as some attempt at DR, but I can't remember if that ended up working in the end.

When working with skew and base values for CARP VIPs, Skew values are 1/256th of a second, and base values add whole seconds. So if you set advbase to 1, then it would wait 1 second + skew, rather than only the skew time.

In other words:

Using the CARP VIP you get guaranteed failover and consistent behavior across all client platforms. Using both you are completely reliant upon the client to behave in specific ways, which only gets worse on networks with many different types of clients.

@Derelict

Thanks for the confirmation that the behaviour is as expected. Very much appreciated.

I think I'm at least partially encountering this bug, which I updated with what I'm seeing: https://redmine.pfsense.org/issues/6579

Thanks for the reply

Thanks for contacting me. We did mostly static reservations but then it got too much overhead with devices coming online.

And yes, when the backup dhcp answers it registers with unbound on the backup DNS where I can see the entry. but the backup DNS does not sync back to the master, and when there is an update from the master the entry in the backup gets wiped (i think, didn't verify).

I thought I had a solution by blocking dhcp from 0.0.0.0 to lan-address:67 which gets propagated to the backup. and dhcp traffic is only received via the carp address. That seemed to work but then I found a client with which it did not work haven't found why.

The proper way is to turn DDNS and have a separate DNS server to take the registration from either dhcp server. This will also solve one deficiency of pfsense where it can not resolve/access DNS servers on the other side of an ipsec tunnel (if you have branch offices) We had to resort to having a second caching DNS server for that purpose to forward inquiries too.

@JeGr

Hi JeGr

Thank you for you explanation.

I've talked to my provider and they can supply me with a transit network and route a /29 through it.
Though their /29 is more expensive than renting a /24 from a provider. My concern is if they will
be willing to announce this /24, if they have to or they can refuse? the price they will charge for it
I will clarify it tomorrow.

PROBLEM SOLVED:

Call on OVH to activate the promiscuity mode on our WAN interfaces.

From now on everything is working, thank you for your help.

OMG. My bad! I have protected the HTTP directory password. The password was stored on the external system in the browser. So LE could not access it. Sorry for my misfortune :-(