Hi Jimp, I tried that, it did reduce the errors but they were still there. As a last ditch attempt I stuck in a 160gb SATA disk i had laying around and that worked perfectly. So it must have been something strange with the converter. Strange thing is, I have the exact same setup on my primary firewall, with a 4GB CF card and converter, upgraded that to 1.2.3 and worked without any problems. So I am not sure why I had issues with the backup firewall, it would be a very strange coincidence if there was a hardware failure at the same time as upgrading the software. Either way things are back up and running, thanks for your help, much appreciated.

Actually, this problem may be related to another problem I posted at the same time: http://forum.pfsense.org/index.php/topic,25874.msg135322.html#msg135322 I've been concentrating on the other problem because that one made pfSense unusable for my application. For now I'm forced to go with another solution. JBB

I'm not sure that what SuperMule is suggesting makes sense in this situation. This is where I'd suggest you start. In the VMWare VIC (virtual infrastructure client): On the HOST: Configuration - Networking Get 'properties' on the switch associated with these IP addresses.  Then, clicn on the vSwitch, and click "Edit" Under "Securty" - set all three (Promiscuous Mode, MAC Address Changes, Forged Transmits) to "Accept" PLEASE NOTE that this has security implications!  You may want to be more specific in how you configure this, etc.

What is the recommended procedure for recovery? Let's say FW-A fails, FW-B becomes master… if FW-A returns to service before any changes are required, GREAT - but what it you have to make changes? Are you better to re-connect FW-A as a slave to FW-B? Or backup and restore selective parts of the FW-B config to FW-A? Thanks!

Ya unfortunately I have to double NAT for now since we are running two (3 actually…) firewalls in parallel all off one modem.  This is all part of an overall plan to get it down to just one.

@GruensFroeschli: You got the AoN rule wrong. This rule is for outbound traffic. So the source sgould be the server and not any, and the destination should be any and not the server. Awesome! It's working! Thanks a lot GruensFroeschli

Not usually. The WAN IP is still needed since it will be used for the firewall itself, just not for traffic leaving your LAN. As for the outbound NAT rules, they are processed in a first-match-wins fashion. If you have three rules that specify traffic from LAN uses a VIP, it will use whichever one is on top, it won't skip it to use the next one down to do any kind of balancing. If you want to use them all for your LAN, you'd have to specify the rule in such a way that it matched a different portion of your LAN for each VIP.

@jimp: The bridging is what creates the loop, not being plugged into WAN and OPT1. When you bridge two interfaces, you essentially bond them together and combine the WAN and OPT1 networks. Doing this once is fine, doing this twice creates a loop. Bridged interfaces do NOT have a CARP IP assigned, and work nothing like traditional interfaces with CARP IPs, which is why there are so many warnings. Unless you deactivate the bridge somehow (STP, script, devd, etc) both bridges are always active. Thanks for your reply. Now I understand why its creates a loop, because the bridged interfaces are both active. Then I will try using the STP method. Thanks for your help and time.

Correct, you will add one VIP for each additional IP that you want pfSense to own.  The WAN interface already owns one of the IPs and the Comcast gateway own another.

Are there any tutorials out there that show a typical virtual IP setup?  The interface seems relatively self-explanatory, the port forwarding just isn't happening. Another weird thing that happened when I had pfSense in place is that I couldn't rdp to one of my customer's servers.  When I put the Endian back in place it worked fine.  I didn't dig into anything at the time to figure out why, but all outbound connections from the lan to the wan were supposed to be permitted based on the first firewall rule that is there by default. Thanks -Rich

Ok, many thanks! i'll install VMWare and i'll try with it.