Thanks Eugene. I'll give it a whirl when everyone is off with the plague or something.  ;)

Please see this: http://forum.pfsense.org/index.php/topic,16345.0.html and this: .http://forum.pfsense.org/index.php/topic,16373.html  I suspect you have discovered the same feature as me and several other people:

!NAT + loadbalancer + MultiWAN = Multicast storm.

After quite a bit of testing and so on I seem to have the same issue as this: http://forum.pfsense.org/index.php?topic=16373

I added the sync components one by one and as soon as the load balancer config copied over then things went beserk.  By unplugging all WAN connections bar one I was able to run tcpdump and see multicast traffic flooding the systems.

It seems that load balancers forward MCAST from your LAN to the other box which then repeats it back for a loop.  CARP needs MCAST to work though.  The bug is also seen in 1.2.3-RC1.  I also have systems broadcasting in over VPNs connected to my ADSL routers so it is a bit hard to block all this.

See: http://forum.pfsense.org/index.php/topic,16566.0.html for potential fix.

I have the similar problem.. I am trying to configure VIPs for the mail server (Exchange2007), I have 1.2 pfSense version, 3 interfaces (WAN, LAN and OPT1) the email server is connected to OPT1 interface together with domain controler. I made an identical configuration like "jits" showed in screenshots (with my IP adresses of course :)), but I can't connect or ping to my external IP of email server.
Can anyone help me with this issue ?

screens in attachment.

Reagrds Michal

VIPs.JPG
VIPs.JPG_thumb
NAT.JPG
NAT.JPG_thumb
Rules.JPG
Rules.JPG_thumb

it's interesting that you claim it happened after 1.2 -> 1.2.2 upgrade…
can we see tcpdump from wan when you try to use proxy arp VIP?

On both boxes do```
pfctl -ss > states.log

pfSense does not support FreeBSD alias addresses via the GUI. See this for a manual method http://doc.pfsense.org/multiple-subnets-one-interface-pfsense.pdf
Alias addresses are in 2.0, but aliasing CARP interfaces is probably not supported. You'd have to test on a 2.0 snapshot.

sorry, but I can't imagine 'high DNS traffic' which can bring pfSense down unless you prepare and carry on some attack. -)))

In such scenario our nameservers become unresponsive and clients get stuck with dns queries bringing the state count higher.

State count higher? usually local computers use some local dns server which does not bring any load on fpSense and local name server(s) usually use one-two-three (ok - several) external dns but again it can not increase states count very high because for every dns-query you use the same external name server(s).
There is some inconsistency in your statement. I think yes, you should investigate further.

what  if you try to add on master firewall some weird distinguishable rule on carp interface (i suspect this is interface for pfsync), does it appear on slave firewall on any other interface?

Update.
The problem appears only when you have load-balancer configured (in failover mode). So initially I was struck by this problem in configuration with two WAN interfaces. But it is not important to have the second one. Just configure laoad-balancer at WAN interface (with only 1 member) and create rule on LAN allow from all to all with gateway=load-balancer. As soon as you do it you will get two issues:

broadast packets go easily from LAN to WAN if you have outgoing NAT for these broadcasts then everything is ok. You can see src=38.x.x.3 dst=192.168.0.255 at WAN interface. BUT if you do not have outgoing NAT for the packet (ip on connected to LAN device uses different subnet) then you end up with broadcast storm with packets src=a.b.c.x dst=a.b.c.255.

I do not understand why it happens as there is nothing connected to wan interfaces - just cable connecting two firewalls -((( It seems as both WAN interfaces try to route these broadcast traffic.

192.168.0.1/24 _________  38.x.x.1/25
        –----------|pfSense1|---------
        |          lan  ---------- wan      | C:38.x.x.3
------|C:192.168.0.3                        |
        |                ________            |
          -----------|pfSense2|---------
    192.168.0.2/24 ---------  38.x.x.2/25

If anybody interested I can send config.xml from these boxes but setup is pretty simple...

Hi,
We have solved the issue by engaging a network engineer to review our firewall configuration. We do not actually require virtual IP. Just the 1:1 NAT will work. Thanks anyway!

You're right. I enabled multicast filtering on both switches and it works now. Thanks for your help.  :)

Hi,
i'm testing redundancy between the two firewalls by enabling/disabling CARP, and testing WAN failover by blocking traffic to ISP1 gateway(blocking access in a firewall further out in the network).

Primary FW + ISP1 : Means, primary CARP member carrying traffic towards ISP1.
Primary FW + ISP2 : Means, primary CARP member carrying traffic towards ISP2, connection towards ISP1 is down.
Secondary FW + ISP1 : Means, secondary CARP member carrying traffic towards ISP1.
Secondary FW + ISP2 : Means, secondary CARP member carrying traffic towards ISP2, connection towards ISP1 is down.

I have adressed this towards premium support, and Chris Buechler has found a problem and is looking for a solution.

I've been playing with CARP for a few weeks now, and it would seem that you are correct.

For each VLAN/subnet you require 1 IP for each real machine and 1 IP address for CARP to use, shared across all of the machines.

Can't try until this evening when the users go home…... no longer aloud to play with it during business hours... Will post back later.  ;D

I was working on using the PHP interface to create a CARP VIP and NAT to an IP and the first attempt on this involved me just adding to the config array, and not actually calling any of the functions that I needed to reset things.  After doing this, I noticed that the result looked exactly like what it looks like when I create a CARP VIP on my Primary:  namely that the info is synced, but no interface is assigned.  Basically, I'd run something like:

$GLOBALS['config']['virtualip']['vip'][7]['mode'] = 'carp';
$GLOBALS['config']['virtualip']['vip'][7]['interface'] = 'wan';
$GLOBALS['config']['virtualip']['vip'][7]['vhid'] = 8;
$GLOBALS['config']['virtualip']['vip'][7]['advskew'] = 0;
$GLOBALS['config']['virtualip']['vip'][7]['password'] = '******';
$GLOBALS['config']['virtualip']['vip'][7]['descr'] = 'new interface';
$GLOBALS['config']['virtualip']['vip'][7]['type'] = 'single';
$GLOBALS['config']['virtualip']['vip'][7]['subnet_bits'] = 32;
$GLOBALS['config']['virtualip']['vip'][7]['subnet'] = '1.1.1.1';
write_config("Adding new interface");
exec

and it would add it but there would be no carp interface assigned.  So I looked through the PHP used in the pages to add a VIP and discovered this particular group of functions:

config_lock();
services_proxyarp_configure();
reset_carp();
filter_configure();
config_unlock();

After running that, the interface came up.  I'm wondering if the reset_carp() function is not being executed when the new VIP is synced over to the secondary firewall.