If you have a reproducible case, please open a report at redmine.pfsense.org outlining the expected behavior, the steps to reproduce, and the actual behavior.

??? :-[ :-\

I found the problem. One server has access via HTTP and another HTTPS, this was the problem with synchronization.

So I changed the two to HTTPS, normalized.

Thanks all!

I had a very good chance. I have found a site that provides useful information this good.

Yes I can ping that specific IP. I have checked the firewall and it is completely open. I even created specific rules along with everything open, Tried copying the same rule that was there before the upgrade (completely open) it seems to be just that port since I can connect/test port 80 but I don;t want to change the GUI connection from being unsecure. I checked the tables and no lockouts either.

Thanks for the help/suggestions

Perfect. Thanks for reporting back. DHCP servers (if any) are all normal/normal?

@Derelict:

WAN Interface: Static IPv4 10.10.75.251/29
Gateway: x.x.x.17

Having your gateway not included in the interface subnet is an odd configuration.
Or is the interface really a /24 and you can only use that /29 out of it?

Sorry, I doesn't mentioned it!

The gateway is a public IP address, 62.x.x.17 and "use non local gateway" is set. Outbound NAT is also set.

I read all the threads here about this setup with version > 2.2 and someone mentioned, that the mask on the WAN interfaces should be the same as the public networks. I changed it to /24, master 10.10.75.251/24, slave 10.10.75.252/24 but there is no change. Master to Slave runs perfectly with only some lost packets, Slave to Master lets the default gateway missing on master. If I add it manually with route add default 62.x.x.17 all is up immediatly.

I have done some debugging on console:

a) console on master

enter persistent CARP maintenance mode on MASTER failover to slave, all connections established default gw lost on master (netstat -r) leave persistent CARP maintenance mode on MASTER all interfaces and services "green" only default gw lost route add default 62.x.x.17 all is up

b) console on master

ifconfig ibg4 down (WAN interface) failover to slave, all connections established default gw present on master ifconfig ibg4 up go back to master as active all interfaces and services "green" only default gw lost route add default 62.x.x.17 all is up

c) console on master

sysctl net.inet.carp.demotion=250 failover to slave, all connections established default gw present on master sysctl net.inet.carp.demotion=-250 go back to master as active all interfaces and services "green" default gw present on master!!! all is up

I tried c) several times and pf always switches perfectly between master and slave
without lost of any connection.

If I simulate a lost WAN interface with b) the default gw will be present. The default
gw not lost during failover, but when the Master takes over again.

If I set the Master in maintenance mode a) , the default gw is lost immadiatley.

What are the differences between these scenaries, so that only c) function correctly?

Tom

Following up on this topic.  I never got anywhere with it.  After reading many posts which suggested solutions such as utilizing STP to handle the potential issues that would arise from the mac address changes on the same physical interfaces - loops, etc, I've had to abandon the idea of using pfSense as a high availability solution when bridging.

There just doesn't seem to be a way to really handle it without complex networking and/or adding more hardware across each side.  pfSense seems to need a bridge-specific solution to make this work without significant effort or alteration in my environment - so I've had to end the search and just use pfSense as a router as well.

To start, your provider is probably not CARP compatible, or it would likely be working.

When it comes to CARP VIPs and ISPs there are two general principles that they must support.

CARP advertisements egress sourced from the CARP MAC address. This performs two tasks:

The switch sees the CARP MAC address and adds it to its MAC address table

The BACKUP CARP node sees the advertisement and does not switch to MASTER

The ISP, having traffic for the CARP VIP address does an ARP request.

The pfSense WAN responds to the ARP "WHO HAS" from the interface MAC address but says the address IS AT the CARP MAC. This directs traffic from upstream to the CARP VIP to the CARP MAC address which has previously been installed in the switches MAC address table by virtue of the CARP advertisements.

Upstream has to support multiple MAC addresses and multicast for CARP to function. ISP gear does some silly-ass crap. Especially residential gear.

There has to be solid layer 2 between the two CARP nodes and the upstream.

If you cannot upgrade the secondary, backup the configuration, install 2.4.2 fresh, and restore the configuration.

It will be a surprise to nobody that you are experiencing problems with that disparity between node versions.

Think of the DMZ as another LAN segment. It will need a CARP VIP to float between the firewalls. The Public IPs you are using for 1-1 NAT will just be CARP VIPs off the WAN.

In all honesty, I would go to 2.2.6 first.

It is much more tolerant of being installed with the WAN disconnected.

After you can do it in Maintenance mode with the WAN connected the other upgrades will go a lot smoother.

So, I may have solved my own problem. I created a new CSR and created a cert with an Alternate name, so that the cert would work for both nodes. It seems to work. I'll report any oddities.

John

Thanks for pointing me to that bug.