@Falko:

now to the questions:

can i use multiple CARP VIP as a base for a 1:1 NAT? (i need 10-20 1:1 NAT ip addresses)

Yes.

@Falko:

is it more useful to use multiple default gateways (iproute2) in the linux machines or a set of shared LAN CARP VIP? (one for each VLAN)

Having multiple default gateways on the Linux machines will introduce complications unless you're doing policy routing within Linux. Without policy routing, you'll have issues because only one default gateway will be used, and that will route return traffic out the wrong way in some cases. Single homing everything is easiest for that reason.

@Falko:

do i need a specific switch support/configuration to enable the in/outbound CARP VIPs? (i have a cisco switch)

If it's a real Cisco switch and not a Linksys Cisco, should be fine. The Linksys Cisco switches at times have security-related settings enabled that break multicast. It's also possible to break multicast on a real Cisco switch but such configs are very uncommon.

@Falko:

i using LACP ports with CARP a problem?

no, lots of people do that.

Podialrius,

Thanks so much for your time and help.

The APs were set up there some time ago and not certain why things changed but they did. One good thing, guests can't try and hack the APs when they can't see the webgui or their IP. We can access them but just a heck of a lot easier if it was simpler.

I did not see in the 1.2.3  advanced page anything about bypassing firewall rules. Still need them and NAT for everything else this box is doing.

Will probably move the net back to 50.X. Less hassle other than an occasional attack against one of the APs.

John

solved the problem, was a corruption between packages on the 2 firewalls. removed the packages and the problem was fixed. Seemed to be the squid packages but due to some testing between squid and squid reverse. They were no longer needed.

Scratch this topic. My co-lo provider was handing BGP to us incorrectly. They've since configured it to hand off to us correctly, so I am no longer confused.

Ok so one I asked the DC to route the /24 to the CARP ip everything works. Outbound ips are showing correctly and I am very happy now :-)

Thanks for all the help people.

When both systems are master it's because the CARP multicast isn't making it between the primary and secondary, most commonly because of a general connectivity issue between them, but at times because the switch(es) aren't passing it which can happen for a variety of reasons.

@miloman:

I had a similar problem on one of my firewalls.

My solution was to edit the VIP in question on the primary firewall and just put in a - in the description, then save… Then everything started working.

That couldn't be anything more than a coincidence, the description field does nothing at all other than display a description.

Thanks for this hint.
I missed the check box to enable sync on the secondary… m(
Now it works as expected! Very nice.

Nope … I am refering to something like

8.8.8.8 -> <your_external_real_ip_address>-> 10.1.1.1    ->  10.1.1.2              -> 192.168.1.1      -> 192.168.1.2
Internet      WAN on your router                    Lan on router    WAN on pfsense        Lan on pfsense      Server

You are having to go through 2 private nets to get to the internet ... this is double nat.
It is not usually a good idea to double nat. Usually because of the administration headache and over complicating the network setup. Sometimes it is necessary and I would only use it if absolutely needed. You have to make sure that the correct ports are open all the way through your setup.</your_external_real_ip_address>

I'm referring to real chaos (switches flaking out, other extreme network flakiness), not high load. CARP preemption is enabled, it should always switch all IPs over.

You should see the same on both of them. What you're seeing there shows the two can't see each other on the network. The primary's CARP should show up exactly the same on the secondary, and then the secondary won't send any CARP traffic. If it doesn't show in tcpdump, it's not getting there, even if the firewall were blocking it, it would show in tcpdump.

Post a reply that is was solved and perhaps change the subject on the either the original post or on this one. If that does not work, then a mod will have to do that.

@podilarius:

You cannot have more that 1 or 2 on the same VHID … iirc. I usually just use the last octet as my vhid.

One per VHID per broadcast domain. Using the last octet is a good option most of the time.

Back to one of your first questions, on the secondary, you do not set a Sync Config to IP. There is a clear warning: NOTE: Do not use the Synchronize Config to IP and password option on backup cluster members!
I don't usually assign an ip to  pfsync Synchronize Peer IP option. Leaving it blank uses multicast on the XOVER cluster network.

Set your interfaces and CARP up on internal addresses… then NAT the internal CARP vip to your public /30 address. Might take some tweaking but should be a workable solution.

@podilarius:

If you are trying to connect to them from the internet, the process is the same. If you are looking for inter-LAN communication, you should be able to access them via their internal IP address in the new subnet. The allow all rule you have on each LAN should pass all the communications.

If you want to restrict that, you are going to have to create a series of aliases and change the default rule.

If you are using DNS names, then I would use the DNS forwarder's override to create a split-brained DNS. This way if you are internal and you are using the pfSense firewall for DNS services, then when someone from the inside requests the DNS name they will get the internal address. But if you are on the internet, you get the VIP address.

I am sure there are more options as well depending on what exactly you are doing.

I got it sorted and I want to thank you publicly (not just by PM). I learned a lot during the process and I will make an effort to check the forum to try and help others.

Thanks again!! I appreciate the help.

When you're syncing users, it syncs the admin password on the secondary, and then you have to change the admin password in the sync settings on the primary to match. That's usually what people don't change when it breaks after the first sync. It works fine in every version, you probably had something mismatched there from that.

The MAC of a CARP VIP is determined by its VHID - it's shared in common between all CARP VIPs on all nodes. So from the router's point of view, the MAC would be identical, but it would have switched to another port.

Sometimes CPE switches can be odd with CARP - can you try plugging the master/slave into a small switch and then uplinking to the modem?