When I try to change the Vlans I get a message:

The following input errors were detected:
•The VLAN tag cannot be changed while the interface is assigned.

But in "Interfaces/ Interface Assignments" the WAN is the first interface , hence the only one that doesn't have a delete button.

I cannot temporary assign another network port since they are all in use, and I cannot create a new one in the console because then I will clear all my Vlans config.

----while typing I thought of the workaround----

-I deleted one of the other interfaces (OPT11)
-assigned the network port of that interface to my WAN connection
-Changed the Vlan of the WAN Network Port
-Reassign the Network port with the correct Vlan to the interface
-Recreate the deleted OPT11 interface and assign it's original network port.

Not a clean way of working , but it did the trick.
Now I'm updating the version and I'll get my config Sync working.

Thank you thesurf & Rico for the very valuable help! 👌

Absolutely. That's what I would see.

If as he wrote sees on the switchs advertisement and both send them my assumption is that he has a switch for each wan line. To work with carp as failover there are two options.

A) connect both switches so the advertising packages can be seen by both pfsense.

B) from each pfsense connect a port to each switch and setup two virtual IP with the regarding interfaces.

Hope that is clear. Else please ask. Maybe I can later provide a drawing.

@alexniko finally I resolved. I reinstalled the system some times ago and have not checked the promiscuous mode on Wan interface.
So lesson learned is check, check and check again.

Sounds like your pfsync (state sync) is not working.

https://www.netgate.com/docs/pfsense/book/highavailability/pfsync-overview.html

You probably need to post your 1:1 NAT rules, port forward rules, and the rules on that WAN interface. Then be specific about what connections are not working, such as protocol, source address (outside is probably good enough) and destination address and port.

When connecting into WAN, the port forwards will be processed first, then 1:1 NAT. Note that 1:1 NAT does not automatically add WAN rules as port forwards can do. In either case the WAN rules need to pass to the POST NAT address/port (the real, listening address/port on the destination server).

Additionally the client is mentioning a host with a VIP cannot reach certain sites from the LAN side of the router. The logs at the time indicated seeing the LAN IP of the device hitting the WAN interface on the pfSense. How the hell does that happen?!

Probably Outbound NAT. I would separate these two issues and treat them separately.

Does anyone have any ideas what could be causing this, or where I could look next to get any more useful debugging info?

Hello,

resolved by move the "Configuration Synchronization Settings (XMLRPC Sync)" on the pfs1 that it is also the Master node.

They can't see each other's multicast CARP advertisements.

Check your switching layer that they are both connected to.

HA needs static addresses, not PPPoE, DHCP, etc.

The nodes also need identical interfaces on them.

I understand you may have a lot of support questions but would you mind answering my actual questions at the bottom if possible?

Yeah, that's what it is supposed to do.

I would set a maintenance window, put the primary in maintenance mode, do what you have to do, and remove it from maintenance mode.

And I'd stop moving cables around.

@actualrootwyrm

Hi, I wrote that code some years ago, freely publishing a pfsense customisation I had made for a service provider who had hired me some time before.

As you repeat here, I was surprised in seeing that such a feature (i.e. dns update on a CARP failover) required an ad-hoc script, so reading that another user was looking for the same, I had made it available with some remarks, knowing that it could have been useful later.

Still, as many changes have been introduced in the following pfsense releases, to make that code working again you have (and you will always have, because being a custom patch it will require continuous check/maintenance at every pfsense update, unless it becomes a standard feature as you hope) to:

ensure php is still the current scripting language for pfsense verify the current release php syntax for the functions required to manipulate strings (I had already slightly modified it for a next pfsense release) verify the current config.xml structure for setting the configuration keys to enable/disable dynamic dns entries (check the similar code used for the GUI) verify the current rc.carpmaster/rc.carpbackup (see parameters and structure)

I don't have time for committing into this now, but let me suggest you'll have just to insist with some tests (possibly displaying intermediate string manipulation results) to get to the desired behaviour.

Let me say that even if you defined it just a "kludge", I had always been proud of that smart and quick snippet of code, tailored to solve a specific issue.

As it is your effort to create and maintain it (I really doubt it can be raised to a feature being it so specific), it'll be up to you to decide whether to publish it or keep it for yourself.

Good luck.

Again, more details needed. See above.

"Can't ping out" is a symptom. You need to diagnose to find out what is not in place that is put back when you save the interface. My guess is something like a default gateway. But that's just a guess.

Got it. I wasn't really thinking about it. Thinking about it, you're right. It makes no sense for me to have obfuscated them.

EDIT: Deobfuscated them through all posts.

EDIT 2: So I'm not convinced I've got my problem solved just yet, but it's possible. I reset my pfSense slave to factory defaults and have been reconfiguring it from the ground up. So far DNS is still working, but I still have a handful of interfaces to configure. At this stage, I would expect it to not be working on any interfaces if it was going to have any issues, so I'm hopeful. If this does fix it, I have absolutely no idea what was broke.

@citronvolcano said in Sync captive portal logged in state:

is there a way to sync captive portal logged users between the Master and the Backup?

Not that I know of. Last time I ran an HA captive portal I am pretty sure I told it not to sync the CP settings and just disabled the captive portal on the secondary. In the event of a failover it was better to just allow the traffic than to break 3000 CP sessions all at once.

Yes, there would be a "vulnerability" in that a savvy user could just manually set their gateway to the secondary's interface address and bypass the portal but that was deemed a lesser concern. The access was "free" anyway. The primary reason HA was implemented was keeping the front desk from getting slammed in the event of a failure, which equates to keeping the guests happy.

Each ISP modem is connected to a Layer 2 unmanaged switch, which then one port is connected to one FW the other the other FW.

Different switches per WAN correct?

Each box is identical, except one is Master and the other Backup of course so I know my HA sync is working.

The SYNC interface has nothing to do with the CARP VIP status on each interface or which node is master or backup at any given time.

https://forum.netgate.com/post/719523

My problem here is when I have one ISP connected the IP address assigned to the VIP never shows up on the modems ARP table.

The CARP MAC only shows up in the upstream MAC address table due to the CARP advertisements.

When the node holding the CARP MASTER status sees an ARP request for the CARP VIP, it answers with an ARP response. This ARP response is sourced from the interface MAC address but contains the CARP MAC address as the ISAT MAC address.

There is no reason for the modem to contain the CARP VIP in its ARP table unless it needs to route traffic from itself to the CARP address.

That said, MANY ISP devices simply do not do what is necessary for CARP to function correctly. They might only allow one MAC per port or any of a number of silly things.

Some work fine.

Yes, just change the route to direct the traffic to the virtual IP.

Then maybe it is just multicast connectivity.

With both as MASTER you should be able to see the CARP hearbeats from the other node when you capture CARP on VLAN10 or VLAN20. If you only see the hearbeats from the local node you are capturing on, there's your symptom.

CARP VLANs work fine.

Are the CARP VIPs MASTER and BACKUP on the primary and secondary respectively (Status > CARP)?

Did you instruct your DHCP server to give the CARP VIP as the default gateway in its leases?

but it does not work as well

What does "does not work" mean?

https://www.netgate.com/docs/pfsense/routing/connectivity-troubleshooting.html

https://www.netgate.com/docs/pfsense/highavailability/troubleshooting-high-availability-clusters.html