Thanks to everyone I now have what "appears" to be a working config. Here are the steps I took. For the basic /30 link to ISP I setup the /30 IP on the WAN interface. I then added my /27 public ips as VIPs using IP Alias - as pointed out above, use /32 and add them 1 at a time. Since I have a Layer 3 switch attached to my LAN interface, with 3 routed subnets + the pf LAN interface subnet, I had to create 3 routes in pfSense with the gateway pointing to the switch IP on the pf LAN Subnet. THEN - TO MAKE MY LIFE EASIER (and this may differ for you), I created a Network-Type "ALIAS" in pfSense and added my 4 LAN subnets to that alias. Then I turned on MANUAL OUTBOUND NAT (AON = MANUAL). I edited the default 2 LAN subnet rules and changed the LAN Subnet to my "Network-type ALIAS". Then finally, I edited the default LAN Firewall Rule and where it originally said "LAN SUBNET" I simply changed that to my "Network Type Alias". EVERYTHING SEEMS TO BE WORKING – AT LEAST AS FAR AS OUTBOUND TRAFFIC USING VIPs. My next project is to get VPN working using one of those VIPs as the destination.  Whether this will continue to work remains to be seen. MANY MANY THANKS TO ALL WHO HAVE HELPED IN MY NUMEROUS THREADS ON THIS ISSUE. ALSO -- A Special thanks to "Metu69Salemi" who was relentless in his efforts helping me through PM.

Bingo. I'm not a network admin by any means (sysadmin is my primary role), but that gave me enough information to figure out the problem. I didn't explain the details of how all of the switches were connected (which appeared to be more important than I thought), but here's the diagram for future people.   Internet       |       | ------------- | L L L L L | Switch 1 -------------     |  |          |  --------- (to router WAN)     |          | WAN |        -------------     |        | W L L L L | Router 1     |        ------------- firewall1        |  |   |    |        |  |   |    |        |  | LAN  OPT2 -------  -------- (to secondary LAN switch) The CARP multicast was going out of the firewall WAN, then reaching the router WAN. The router WAN (D-Link DI-524) was setup to pass multicast traffic through to the LAN, which then made it show up on the same network as OPT2. I disabled the multicast forwarding on the DI-524 and the packets stopped. For those that wonder why I'm doing it this way, the two firewalls on the secondary LAN also use load balancing on their WAN interfaces and need three IP addresses. I don't have enough public IP addresses to assign to the two pfSense firewalls plus the other two firewalls, so I'm doing a 1-1 NAT of one public IP through the DI-524 onto the virtual IP of the two firewalls on the secondary LAN. The OPT2 interface is for creating an IPsec tunnel between the two networks that doesn't traverse any of the public Internet. Thanks for your help.

Well if you don't want the CARP VIP settings to sync then don't sync the VIPs. Just uncheck that option under the CARP settings. Then you can adjust the VIPs however you like. I will note, however, that splitting the MASTER role of VIPs between the boxes is not a config that is currently supported, so do not be surprised if some bits don't work as you expect.

Is there a reason you want to disable it? It's not usually very useful to have that disabled. You could add a system tunable to set net.inet.carp.preempt=0 (but it may not stick since it gets reset by the card code). The GUI option was removed back in 2006, so it hasn't been in any recent version.

@anagh: use isp wan series on wan side and isp lan series i.e first public ip on lan side open firewall nat click Manual Outbound NAT rule generation and SAVE delete all  auto generated  nat rule This doesn't provide the private IP network interfaces he requires.

I had the same type of issue with switching from pfsync setup from multicast to peer mode, i finally gave up on trying to switch them and just used a crossover cable and a dedicated port for syncing. i didnt want additional traffic (multicast sync packets) on my lan interface.

If the wireless doesn’t need direct access to your network resources, then by just having it on a separate VLAN and making sure there is an ACL to block any packets that are part of the WIFI network to your network. However, if you’re WIFI needs access to your network that should also work. What we have done for our installation at our office and data center is to have two switches,  and two firewalls. A trunk between the two switches but then VLANed them into different groups, for example VLAN1 is our LAN VLAN2 is WAN1 VLAN3 is WAN2 Your ISP hand-off would go into the right VLAN group and then so would your firewall connections. If you’re not using trunks then you will have a direct cable for each of the VLANs back to your firewalls with correct addresses. For the DMZ you can add extra ports to the VLANs you need and have them outside the firewall. And then the placement for the WAP again depends if it’s part of your internal or external network access requirements. What this design allows you to do is have redundancy, what if your WAN switch went out? You would loss all WAN connections, where if the wans were both on two different switches the risk of a failed switch is migrated. To answer your question about if it would be secure, if VLANs are done correctly, its taking a physical switch and logically breaking it down to multiple logical switches.

hey, no problem, the fact your helping is amazing so again thank you. i will try  this, we are only allowed to make changes of Tuesday nights, so i will try to  slide this in for this weeks testing. you know that makes sense, you have to have an identifier at each of the end of the tunnel. if it was the direct interface and not the CARP then when fail over occurred the tunnel would have been built with the wrong address (since it was built by the master with one IP which is no longer valid as its down).

I had some trouble at first, what I had to do to fix it is first Verify that ONLY the master sync server has the various sync buttons checked. And just to be safe remove any IP address in the Sync form on the slave servers. Found in the PFSense Documents at: http://doc.pfsense.org/index.php/CARP_Configuration_Sync_Troubleshooting next make sure that snyc is set up correclt by checking: Enable pfSync in Firewall -> Virtual IPs -> CARP settings -> Synchronize Enabled (check it) on all cluster members. -> Synchronize Virtual IPs [ X ] -> Synchronize to IP [ insert Slave IP ONLY on Master! ] -> Remote System Password [ do not forget! ] Select the dedicated Sync interface with the Synchronize Interface dropdown on all cluster members, if it’s on a dedicated port select that port if not then select the port on switch your syncing across.. Afterward visit Firewall -> Rules and add an allow all from any to any rule on each cluster member for the newly created pfsync interface. Found in the PFSense Documents at: http://doc.pfsense.org/index.php/Configuring_pfSense_Hardware_Redundancy_%28CARP%29 I know that this might be a common mistake, but I am new to PFSense and I did the above and it fixed my syncing issues.

Just add new dns entrys for your dns server. I have windows AD with something.local as interal domain and outside domain is actually something_else.com, if i want to connect webserver internally i've created another domain into my AD-server(DNS), now it has internal and outside domains side-by-side. Only different thing to outside domain entrys is that i use only internal ip-addresses for those A & PTR Records at my own server. And ofcourse this AD-server is the one which i share via dhcp to clients

Thanks for the reply.  The switch involved doesn't seem to matter.  The test jig uses a netgear 1000M capable 5 port switch on the lan side, connecting the two routers  (192…2 and 192...3) carp sharing 192..1.  Whichever of them is the carp master can't ping .1, while all other systems can.  The system hosting as the carp master .1 can ping its own native address, but not .1 It seems the actual mac address on the outbound frames routed via the carp interface is not the carp lladdr, but the interface native address.  So, the mediacom cable modem, which must bind to a specific mac address, won't bind to the carp llaaddr (which is VHID specific).  This makes PFSense useless in a failover router setup with that ISP. I've 'worked around' the problem by setting up a third little pfsense box with just two ports, acting as an extension of the cable modem, can't really use 1-1 nat, so I just port forward what little I need.  It's a single point of failure, but then so is the cable modem and it still only risks that ISP's connection, the others are still protected by the failover router pair. Still, if CARP could be improved to use the CARP logical link address when transmitting packets sent out the carp interface, and not the interface's native MAC address, then I could avoid maintaining an extra router and dealing with NAT issues on what should have been a native connection to the net.

Is this the right way to do it? /- 192.168.202.11 xx.xxx.xxx.178 = 192.168.202.10 (VIP7)                                                       - 192.168.202.12 /- 192.168.203.11 xx.xxx.xxx.179 = 192.168.203.10 (VIP8)                                                       - 192.168.203.12 /- 192.168.204.11 xx.xxx.xxx.180 = 192.168.204.10 (VIP9)                                                       - 192.168.204.12 /- 192.168.205.11 xx.xxx.xxx.181 = 192.168.205.10 (VIP10)                                                       - 192.168.205.12 (where .11 is for fw1 and .12 for fw2)

Having two subnets on the same switch offers you -zero- security gain. There is nothing stopping anyone from simply coding in an IP on the other subnet and using it to talk to those machines. Security by obscurity is not effective against anyone who really wants to get in. You also cannot run DHCP on multiple subnets on the same interface/broadcast domain. You really need to separate them physically or by VLANs if you want to achieve any of this effectively. That said - in 2.0 you can add an IP Alias VIP on the LAN to act as the "gateway" of the second subnet, and then hardcode people into that subnet using that as their gateway/dns. Adjust your firewall rules to let the traffic through, and make sure they're covered by your outbound NAT rules, and it should work. It's possible, yes, but not recommended, especially not for security.

If your ISP is routing that additional subnet to your OPT1 CARP VIP, then you need only add them as "other" type VIPs, not CARP VIPs. Then they can be used for NAT. If it's delivered directly, where the ISP gear expects you to have "real" IPs there, you'll need to upgrade to 2.0. Then you can make an IP alias VIP on each firewall in the new subnet, and then you can add CARP IPs for that subnet.

You will need at least a /29 on WAN, each machine needs an IP in that subnet, plus the CARP IP. There isn't enough room in a /30. How it normally works is that you have the IPs as above, and your ISP routes your /24 to the CARP IP on your WAN. I'm not sure how much help that other thread might be in your case since I'm not sure how many, if any, modems like that could handle static routes. The modem would have to terminate the /30, you'd need a privately numbered subnet on the inside of the modem, and the modem would have to forward that /24 back to the CARP VIP in what is now your firweall's WAN subnet.

check if you have a carp on dmz and if the server's gateway on dmz are set for this carp.

First of all: Do not forget to backup config on both boxes before and after config changes. Answers: 1)2)3)If you have two boxes, use only carps for fail-over. Configure a full redundant firewall configuration. all ips on each interface(vlan or real) must be on same subnet too. The vhid must be unique for each virtual ip and it is used to check health between boxes on each interface(vlan or real) with carp enabled. If it is a layer2 switch, you will not be able to route between vlans. you will need a layer3 switch. Create vlans as much as you need at pfsense, vmware and switch. create carps on each interface for fail-over between firewalls. Set these virtual ips as gateways on each vlan/virtual machine. Considerations: Do not forget to configure a sync interface between boxes. After all carp settings done, use firewall rules do block/permit what you need(ping, www, ssh,etc). Read this if you want to setup a first level DOS prevention on your network. http://forum.pfsense.org/index.php?topic=38273.0

If you have carps, the best way(in my opinion) is to separate these gateways on pfsense. If you can't enable another interface on pfsense, create some vlans on your switch and configure it on pfsense. If both of your links are internet links, this will be very hard to set two 'default' gateways on same interface. att, Marcello Coutinho

If you have only one ip on each subnet, you can do it combining forces with you router(s). ;D see (http://forum.pfsense.org/index.php/topic,35281.msg200865.html#msg200865) for a detailed explanation of how to do this. after nat on router, at pfsense: You can't have two subnets on same interface, you need to create a interface for each subnet. the minimun amount of ips for it will be 4 on the same subnet. 1 for the router 1 for pfsense1 1 for pfsense2 1 to be published as a carp ip between two pfsenses. (this can be as much as you need 1, 2,…10 ips) If you plan to have each pfsense pluged into different switches, you will must have a dedicated interface between both for sync. My suggestion is to plug all interfaces of each firewall in only one swtich(using vlans), this prevents some carp mistakes between master and slave when not all interfaces are offline. FIREWALL1 <-> SWITCH 1 FIREWALL2 <-> SWITCH 2 FIREWALL1 <-CROSSOVER-> FIREWALL2 If you have two gigabit interfaces on each firewall you can do everything. one for sync and other with a lot of vlans. att, Marcello Coutinho