edit your first post subject with [SOLVED]

Yeah, no problem.  The router had already been in production for a while and had some NAT port forwards configured, and the associated firewall rules autoconfigured.  I assumed those rules would carry right over to the CARP setup because the destination was WAN.  I went to make a new rule for some reason or another and noticed that there was a new destination choice called WAN CARP (what I had named that VIP).  When I realized the firewall was discriminating between real IPs and virtual IPs, I had my answer.  I guess I just assumed that my rules were all per-interface, but they're actually more granular than that.  Changed all my regular stuff to the CARP destination and set ICMP to pass on anything and everything worked correctly.

I thought I'd have to do some manual outbound rules as well, but so far that doesn't appear to be necessary.  I'll have to read more about that to know for sure, though.

@rootlurker:

Hi,

I thought you have 5 STATIC IP assigned by your ISP, and you also said it on dynamic IP?

Also, "So how do I get the WAN IP to respond to one of my static IP's?" which WAN IP?

Hi,

The weirdness of BT's setup is that when you connect using PPPoE they assign your connection a dynamic IP.

However, we have 5 static IP's that we pay extra for, so somewhere in BT's network, they know to route traffic for those 5 IP's to our dynamic IP, this works fine when we're dealing with another machine on the network, we just set up 1:1 mapping and the packets coming in on that IP go to the local machine and packets destined to go out over the WAN appear to originate from the static IP.

It all works fine using 1:1 mapping when there's another machine on the network, but in my case the pfsense box needs to be a VPN endpoint, so I need it to act on traffic coming in from one of the static IP's itself which is what I can't figure out what to do!

Thanks

Hi,

pfsense can do the LoadBalancing and the Failover ability you know from your actual system.

The difference between LoadBalancing in pfsense and Barracuda is, that - as you wrote - Barracuda can detect the less busy WAN and redirect traffic to this WAN.

pfsense is "just" doing a round robin. There is no difference if a WAN has high load or not.

To configure this in pfsense 2.0 just create the three Gateways, put them all into a Gateway Group with same Tier and chose this Gateway Group in your firewall rules as the Gateway for outgoing/outbound traffic. This is a really easy setup in pfsense 2.0

You may need to change dsl modem to router because CARP needs at least 3 ips on same subnet.

I don't know if there is a feature to do not start wan auth While in backup mode.

That setup would work for failover, yes, though you might want to call that dedicated interface "SYNC" to avoid confusing people when posting about it.

pfSense doesn't support active-active, so you can't do load balancing between the two boxes.

Hi,
  thanks to the help of Jim, the problem has been identified. Some of the outbound NAT rules had "source=any", so also the CARP packets were natted somehow and this brought to an "inconsistent" CARP state.
The problem was solved assigning to each outbound NAT rule a proper source different by "any".
After this, Ermal added some code (that will be released with 2.0.1 RELEASE) to avoid this issue in any case (http://redmine.pfsense.org/issues/1954).

Thanks to Jim and Ermal for supporting!

Michele

K. Thank you guys. You helped me a lot. Really nice to have such a good working community here. Maybe someday I will add my part to make it even better.

In my opinion we can close this thread.

You can balance firewall work when using packages. Use nat/firewall on box1 and squid on box2 for example.

The active/active firewall can be done with carps, but its not desiged for it so, not supported.

Take a look in this forum topic
http://forum.pfsense.org/index.php/topic,40917.0.html

Wait some seconds until your switch flush mac address table.

Also connect to your switch and see if there is any problem with mac table(full) or cpu usage.

I've just noticed that if i change the VIP in PfSense from CARP to IP Alias, then the problem disappears.

Any thoughts?

Nevermind, I'm pretty sure this falls under the "well, duh" heading. I probably had the wrong password set on the primary as well as the "sync usernames/passwords" enabled. So when I set the admin password on the backup machine to the correct one, the XML got synced across, which reset its admin password to the wrong value which caused future syncs to fail and locked me out of the web guis.

I reset the password on both, reset them to the proper values I want and then made sure syncing of usernames and passwords was not enabled in the virtual IP XML sync settings. So far, so good.

Did you use port forward or 1:1 NAT? If you are using port forward, then you will need to use advanced outbound NAT (manual mode) to transform the outgoing ip to 201.73.17.178. Remember that it is first matching rule in AON so if your LAN rule is above your custom outbound, then the custom outbound will never happen.

@zeratoun:

Exactly,

i want that, from the localhost of the pfsense firewall itself it uses the VIP LAN or WAN …. it's possible ?

Best regards,

It is possible but highly NOT recommended. I got that running in my test environment and CARP was not happy as ping stopped to the gateway on the secondary firewall. I think this will have an adverse effect on the clusters ability to fail over correctly. I didn't have a chance to test fail over, but i did notice that I could not download packages or ping the gateway. There is not reason I can think of to do this. Would you mind telling us why you would like to do that?

The CARP bad gateway has nothing to do with that. I would upgrade to the latest 2.0 release regardless, though I don't think that will fix your problem, 1.2.2 is a very dated release.

Not enough info there to have much idea what's happening, exactly what pings work and don't, and where they're initiated and destined isn't clear.

Thanks to everyone I now have what "appears" to be a working config.

Here are the steps I took.

For the basic /30 link to ISP I setup the /30 IP on the WAN interface.

I then added my /27 public ips as VIPs using IP Alias - as pointed out above, use /32 and add them 1 at a time.

Since I have a Layer 3 switch attached to my LAN interface, with 3 routed subnets + the pf LAN interface subnet, I had to create 3 routes in pfSense with the gateway pointing to the switch IP on the pf LAN Subnet.

THEN - TO MAKE MY LIFE EASIER (and this may differ for you), I created a Network-Type "ALIAS" in pfSense and added my 4 LAN subnets to that alias.

Then I turned on MANUAL OUTBOUND NAT (AON = MANUAL).

I edited the default 2 LAN subnet rules and changed the LAN Subnet to my "Network-type ALIAS".

Then finally, I edited the default LAN Firewall Rule and where it originally said "LAN SUBNET" I simply changed that to my "Network Type Alias".

EVERYTHING SEEMS TO BE WORKING – AT LEAST AS FAR AS OUTBOUND TRAFFIC USING VIPs.

My next project is to get VPN working using one of those VIPs as the destination.  Whether this will continue to work remains to be seen.

MANY MANY THANKS TO ALL WHO HAVE HELPED IN MY NUMEROUS THREADS ON THIS ISSUE.

ALSO -- A Special thanks to "Metu69Salemi" who was relentless in his efforts helping me through PM.

Bingo. I'm not a network admin by any means (sysadmin is my primary role), but that gave me enough information to figure out the problem. I didn't explain the details of how all of the switches were connected (which appeared to be more important than I thought), but here's the diagram for future people.

  Internet       |       | ------------- | L L L L L | Switch 1 -------------     |  |          |  --------- (to router WAN)     |          | WAN |        -------------     |        | W L L L L | Router 1     |        ------------- firewall1        |  |   |    |        |  |   |    |        |  | LAN  OPT2 -------  -------- (to secondary LAN switch)

The CARP multicast was going out of the firewall WAN, then reaching the router WAN. The router WAN (D-Link DI-524) was setup to pass multicast traffic through to the LAN, which then made it show up on the same network as OPT2. I disabled the multicast forwarding on the DI-524 and the packets stopped.

For those that wonder why I'm doing it this way, the two firewalls on the secondary LAN also use load balancing on their WAN interfaces and need three IP addresses. I don't have enough public IP addresses to assign to the two pfSense firewalls plus the other two firewalls, so I'm doing a 1-1 NAT of one public IP through the DI-524 onto the virtual IP of the two firewalls on the secondary LAN. The OPT2 interface is for creating an IPsec tunnel between the two networks that doesn't traverse any of the public Internet.

Thanks for your help.

Well if you don't want the CARP VIP settings to sync then don't sync the VIPs. Just uncheck that option under the CARP settings. Then you can adjust the VIPs however you like.

I will note, however, that splitting the MASTER role of VIPs between the boxes is not a config that is currently supported, so do not be surprised if some bits don't work as you expect.