Problem with CARP and inbound load balancing

gthornock · Mar 26, 2007, 9:50 PM

I have a pfSense 1.0.1 firewall running in a rack at a local data center, and one of the things it's doing is load balancing a pair of web servers. Everything seems to be working fine, as long as it's just the one firewall.

Last weekend, I tried to set up a second firewall and CARP failover. At first, everything still appeared to be working fine, but as the weekend progressed, more and more people were unable to access (by HTTP, HTTPS or even ping) the load balanced server pool. The web servers also have individual public IPs, which were accessible without any problem: it was only the load balanced pool that was having problems. I shut down the second firewall, restored the first to the configuration it had previously, and since then things have been fine (but without the failover).

Basically, here's the single firewall setup:
xxx.yyy.zzz.84 -> 192.168.1.1: fw1 actual IP
xxx.yyy.zzz.85 -> load balanced pool, 192.168.1.2 and 192.168.1.4
xxx.yyy.zzz.87 -> 192.168.1.4: web server
xxx.yyy.zzz.91 -> 192.168.1.2: other web server
with a couple of other servers that worked fine under both configurations.

Here's the setup with CARP:
xxx.yyy.zzz.84 -> 192.168.1.1: fw1 - CARP virtual IPs
xxx.yyy.zzz.85 -> load balanced pool, 192.168.1.2 and 192.168.1.4
xxx.yyy.zzz.87 -> 192.168.1.4: web server
xxx.yyy.zzz.91 -> 192.168.1.2: other web server
xxx.yyy.zzz.93 -> 192.168.1.98: CARP master actual IP, with 10.0.0.1 as OPT1 for pfsync
xxx.yyy.zzz.94 -> 192.168.1.99: CARP backup actual IP. with 10.0.0.2 as OPT1 for pfsync

The network cards for the OPT1/pfsync interfaces are connected with a simple crossover cable. xxx.yyy.zzz.87, 91 and the others I haven't listed are all proxy ARP virtual IPs with 1:1 NAT, and 85 is a proxy ARP virtual IP assigned to the load balanced machines (which are getting traffic on HTTP and HTTPS).

This is the first time I've tried setting up CARP, but I think I followed the instructions from http://doc.pfsense.org/index.php/Setting_up_CARP_with_pfSense accurately, so I'm not sure why it didn't work with the load balancing. Is that just a combination that doesn't work?

aldo · Apr 4, 2007, 8:18 PM

i would not mix and match carp vip and proxy arp vip addresses in any carp configuration.

make all the ips vips add the rules to your master they will come up on your slave.

it sounds like a broadcast storm causing your issue due to the replicated proxy arp interfaces

dbuckle · Apr 24, 2007, 2:24 PM

Running: 1.0-BETA1 on 2 * WRAP boards & CARP

I'm having a similar problem but in my situation i had a pair of CARP'ed WRAP boards and using Windows NLB as my web balancer.

I decided to use the inbound load balancer since Windows NLB was causing problems.

One thing I did discover is that in a CARP setup, the "CARP Settings" tab do not automatically tick the "Load Balancing" and "Syncronize Load Balancer" options after you've configured the Load Balancing service- you have to do that yourself.

CARP Status still shows a lot of (about 30-40) pfSync nodes which I'm worried about.

I think its balancing properly now but i still can't access the load balanced cluster from within the LAN nodes :\

sullrich · Apr 24, 2007, 5:04 PM

@dbuckle:

CARP Status still shows a lot of (about 30-40) pfSync nodes which I'm worried about.

This is normal.

Also see http://wiki.pfsense.com/wikka.php?wakka=InBoundLoadBalancingTroubleShooting