I've been playing with CARP for a few weeks now, and it would seem that you are correct. For each VLAN/subnet you require 1 IP for each real machine and 1 IP address for CARP to use, shared across all of the machines.

Can't try until this evening when the users go home…... no longer aloud to play with it during business hours... Will post back later.  ;D

I was working on using the PHP interface to create a CARP VIP and NAT to an IP and the first attempt on this involved me just adding to the config array, and not actually calling any of the functions that I needed to reset things.  After doing this, I noticed that the result looked exactly like what it looks like when I create a CARP VIP on my Primary:  namely that the info is synced, but no interface is assigned.  Basically, I'd run something like: $GLOBALS['config']['virtualip']['vip'][7]['mode'] = 'carp'; $GLOBALS['config']['virtualip']['vip'][7]['interface'] = 'wan'; $GLOBALS['config']['virtualip']['vip'][7]['vhid'] = 8; $GLOBALS['config']['virtualip']['vip'][7]['advskew'] = 0; $GLOBALS['config']['virtualip']['vip'][7]['password'] = '******'; $GLOBALS['config']['virtualip']['vip'][7]['descr'] = 'new interface'; $GLOBALS['config']['virtualip']['vip'][7]['type'] = 'single'; $GLOBALS['config']['virtualip']['vip'][7]['subnet_bits'] = 32; $GLOBALS['config']['virtualip']['vip'][7]['subnet'] = '1.1.1.1'; write_config("Adding new interface"); exec and it would add it but there would be no carp interface assigned.  So I looked through the PHP used in the pages to add a VIP and discovered this particular group of functions: config_lock(); services_proxyarp_configure(); reset_carp(); filter_configure(); config_unlock(); After running that, the interface came up.  I'm wondering if the reset_carp() function is not being executed when the new VIP is synced over to the secondary firewall.

Hmm, that's an interesting workaround. You just added an OPT interface with the public, and then it let you add the CARP IPs on the WAN? I never tried that. I'm glad you got everything working.

Aghhh….just saw those, Oh well, looks like I need to move some IPs around then!

I agree mate, however just about to upgrade to 100Mb WAN connection, so upgrading the firewall with new Dell R200 with Intel Quad NIC, can't really afford both at present, so going to use the original firewall as failover (have found pfsense so stable I don't think it'll be needed, you can never tell hardware failures etc) so not overly worried about failover as long as it'll support the connection albeit at a lower speed. Cheers for your help. J

1)  I'm not sure what you mean by WAN access as opposed to Internet access, but if you mean the network that pubip[1-3] are in, then yes.  Same with the Internet (assuming that you've set up NAT and firewall rules allowing access in the first place. 2)  PFSense will replicate rules from the "master" machine to the other machine as long as you have configured CARP (in the CARP Settings under Firewall->Virtual IPs->CARP Settings) to do so.

Figured out where i was going wrong on this - on the first set of firewalls i was using VHID 1 /2 and the same for the second internal firewalls, silly mistake - after setting the internal to VHID 3 /4 it is all working correctly. ;D

Which version are you using. I've been trying to do the same thing and I can't get thenew PARP addresses + NAT to work ?

Yes, that's exactly what I did. I added a PARP IP via the Virtual IPs menu and allowed it to auto create the rule for me (allowing HTTP back to the web server running internally). I then added an additional rule allowing ICMP to internal address. Both rules are created on the WAN interface and reference the internal address that I'm allowing traffic to. I've done this many times for IP's that are on the WAN interfaces primary network. I then used the packet capture option to sniff the traffic destined for the PARP address on the WAN side and I do see both ICMP and HTTP traffic hitting the WAN interface when I test the respective protocols. I then did the same thing sniffing the LAN interface to see if there's any packets being sent to the internal NAT's IP - and I don't see anything. I've also run a packet sniffer on the web server to ensure that I'm not missing something and that traffic is in fact not hitting the machine somehow - there's nothing coming out on the LAN side. Downloading the config XML and looking at the version element - it says I'm running version 2.9. Looking at index.php it says I'm running 1.2-RC1 (built on Sat Jul 21 13:42:54 EDT 2007 ) Thanks Warrick

That's what happens when multicast traffic isn't passed properly. Some switches block or break multicast.

Yeah it didn't appropriately warn before 1.2.1, it's supposed to do that. CARP interfaces can't be removed on a running system. You don't have to reboot right away, but the VIP won't be removed until you do.

This doesnt really make much sense. Set the subnet to what you actually have on the main WAN IP.

You can also do it this way: http://doc.pfsense.org/multiple-subnets-one-interface-pfsense.pdf

Hrm, there shouldn't be any way even when you're switching from proxy ARP to CARP to accomplish that. I know the input validation works when adding, maybe it's missing when switching from one type to another. Thanks for the report.

@GruensFroeschli: Diagram: internet                 |                 |    [WAN] 10.10.10.2/27            pfSense   [OPT1] 192.168.1.1/25 –------ servers   [LAN] 192.168.10.1/24                 |                 |             clients Can you please clarify where you have the 10.20.20.0/26 subnet? Are these IP's used directly on the servers or as VIP's on the WAN? Was the 10.20.20.0/26 subnet assigned to you by your ISP? (since these are public IPs) The 10.20.20.0/26 subnet is allocated for the real IP addressing needs of some boxes that reside in the 192.168.1.0 subnet (OPT1) It only exists in the 1:1 NAT configuration page and the NAT-ing is done on the WAN interface. I didn't make any VIP for this subnet, except for the PAT-ing of the LAN Yes, this real-ip subnet is unique If so you could add these public IP's to the servers directly. –> Assign 10.20.20.1 to the pfSense. This actually is the option that I am considering too. It involves changes on the servers though so was trying a workaround. The case is that if this was a new installation that would be my approach from the begining. This was an existing setup that worked having an openbsd box in place before I used pfSense as a way better managed PF solution, especially for the non-BSDers. Basically: if you 1:1 NAT something you cannot access this forward from the inside itself. Use for that normal port forwards, since pfSense is able to reflect normal portforwards. Search the forum for my username and "normal portforward" since i posted in quite a few threads how to do that. Another alternative is, that you use 1:1 NAT from the outside, and for access from the inside you set up split DNS. This is described here. http://forum.pfsense.org/index.php/topic,7001.0.html Thank you for this tip :) Found lots of interesting info there and the especially the text about adding aliases on the physical interfaces is very interesting. If I remember correct this was what I did, adding the 1:1 IPs as aliases on the WAN that is, on the previous box to make it see this, otherwise "virtual" subnet, as local thus appearing on the fw's routing table. The way I understood the whole VIP concept made me think that it was the way to make something similar. SplitDNS would be a viable option too if this was an isolated enviroment (not different links going to different places interconnecting private networks which share the same NSs) Thanks again for your time! George

If the responses aren't coming back, it sounds like the servers are missing their default gateway or have it set to something other than pfSense.