Hey Jim, That appears to have worked great!  Everything ran fine through my tests, both Master and Backup.  Thanks very much for your help.  Much appreciated. Aaron

Hi jimp, Wouldn't it be prudent for pfsense to have a CARPaware IGMP client? So it can correctly register its multicast membership with the local switch? This would allow CARP on pfsense to become compatible with IGMP Snooping. It might also lift the requirement on ESXi to set the VDS port group in promiscuous mode. I know Windows 2k8 R2 NLB (Network Load Balancer) uses multicast together with an IGMP client, and that runs just fine without compromising security in promiscuous mode. Maybe a feature request? Thanks for sharing your thoughts! Jori Huisman

Yes it does support and use search. you'll find a lot of discussion.

Excellent; I should have known.  Ya'll are all over everything else. ;D

Sadly I would have to switch to one of my ISPs business offers. That would cost more than the double and deliver some telephony service I'd have to pay but would not use. Anyway, as I found no clean way doing this with pfSense, I tried to verify that setup with OpenBSD and I'll stick with that for now. Using ifstated together with CARP you can simply ifconfig down the external interface automatically so it will not interfere with anything on the outside. It's a pity I'm not really good at anything related to programming but a shell little scripting, otherwise I would try to implement some system triggers (i.E. Disable Interface X) in case of CARP failover for pfSense. I really like pfSense, it's the best Open Source Firewall distribution there is. Maybe someone else likes the idea and does implement it ;)

@Falko: now to the questions: can i use multiple CARP VIP as a base for a 1:1 NAT? (i need 10-20 1:1 NAT ip addresses) Yes. @Falko: is it more useful to use multiple default gateways (iproute2) in the linux machines or a set of shared LAN CARP VIP? (one for each VLAN) Having multiple default gateways on the Linux machines will introduce complications unless you're doing policy routing within Linux. Without policy routing, you'll have issues because only one default gateway will be used, and that will route return traffic out the wrong way in some cases. Single homing everything is easiest for that reason. @Falko: do i need a specific switch support/configuration to enable the in/outbound CARP VIPs? (i have a cisco switch) If it's a real Cisco switch and not a Linksys Cisco, should be fine. The Linksys Cisco switches at times have security-related settings enabled that break multicast. It's also possible to break multicast on a real Cisco switch but such configs are very uncommon. @Falko: i using LACP ports with CARP a problem? no, lots of people do that.

Podialrius, Thanks so much for your time and help. The APs were set up there some time ago and not certain why things changed but they did. One good thing, guests can't try and hack the APs when they can't see the webgui or their IP. We can access them but just a heck of a lot easier if it was simpler. I did not see in the 1.2.3  advanced page anything about bypassing firewall rules. Still need them and NAT for everything else this box is doing. Will probably move the net back to 50.X. Less hassle other than an occasional attack against one of the APs. John

solved the problem, was a corruption between packages on the 2 firewalls. removed the packages and the problem was fixed. Seemed to be the squid packages but due to some testing between squid and squid reverse. They were no longer needed.

Scratch this topic. My co-lo provider was handing BGP to us incorrectly. They've since configured it to hand off to us correctly, so I am no longer confused.

Ok so one I asked the DC to route the /24 to the CARP ip everything works. Outbound ips are showing correctly and I am very happy now :-) Thanks for all the help people.

When both systems are master it's because the CARP multicast isn't making it between the primary and secondary, most commonly because of a general connectivity issue between them, but at times because the switch(es) aren't passing it which can happen for a variety of reasons. @miloman: I had a similar problem on one of my firewalls. My solution was to edit the VIP in question on the primary firewall and just put in a - in the description, then save… Then everything started working. That couldn't be anything more than a coincidence, the description field does nothing at all other than display a description.

Thanks for this hint. I missed the check box to enable sync on the secondary… m( Now it works as expected! Very nice.

Nope … I am refering to something like 8.8.8.8 -> <your_external_real_ip_address>-> 10.1.1.1    ->  10.1.1.2              -> 192.168.1.1      -> 192.168.1.2 Internet      WAN on your router                    Lan on router    WAN on pfsense        Lan on pfsense      Server You are having to go through 2 private nets to get to the internet ... this is double nat. It is not usually a good idea to double nat. Usually because of the administration headache and over complicating the network setup. Sometimes it is necessary and I would only use it if absolutely needed. You have to make sure that the correct ports are open all the way through your setup.</your_external_real_ip_address>

I'm referring to real chaos (switches flaking out, other extreme network flakiness), not high load. CARP preemption is enabled, it should always switch all IPs over.

You should see the same on both of them. What you're seeing there shows the two can't see each other on the network. The primary's CARP should show up exactly the same on the secondary, and then the secondary won't send any CARP traffic. If it doesn't show in tcpdump, it's not getting there, even if the firewall were blocking it, it would show in tcpdump.

Post a reply that is was solved and perhaps change the subject on the either the original post or on this one. If that does not work, then a mod will have to do that.

@podilarius: You cannot have more that 1 or 2 on the same VHID … iirc. I usually just use the last octet as my vhid. One per VHID per broadcast domain. Using the last octet is a good option most of the time.