Thanks for the reply.  That's another thing all together.  From the OpenBGPD package info - the filter rules are evaluated in sequential order, from first to last.  But, in the Raw config tab the top line of the config says # This file was created by the package manager.  Do not edit!  And I'm fairly certain those rules were created for me.  I might delete all config entries and start from scratch to see if those come up.  I'm mostly concerned about my advertisements right now and making sure everything goes to the right places.

Aaron

We typically suggest editing /etc/rc.carpmaster and /etc/rc.carpbackup so that the OSPF daemon gets stopped while in backup mode, and started when it's in master mode.

Given the dynamic nature of OSPF, it really doesn't matter if it's advertising the CARP VIP, if the adjacency changes it'll need to update anyhow. Might have a few seconds of downtime as the neighboring comes back up during a switch, but afaik there's no way to have the OSPF daemon actually advertise the CARP VIP as the router address.

Alternately, you can also just setup OSPF as normal on the secondary box but with higher costs than the primary box has, so it would fail to that one quicker, perhaps. I can't remember if I've tried that one before.

Thanks for your answer cmb.

I will consider switching to IP aliases. Seems the right move specially having the advantage of log notifications in case of IP conflict.

You can't do active/active CARP nodes in that way. Only one router node can/should be active at a time for handling any traffic. You can't spread the load between multiple firewall units in an effective way at this time.

Some people have hacked things up manually to make some VIPs master on the primary box and some VIPs master on the secondary box but that's really not something I would suggest.

As you noted, multiple WANs should be connected to all carp nodes to be used properly. Anything else is just ugly and asking for trouble…

Shame on me!
I forgot to enable the first 3 options in the carp setting page on the slave firewall. I didn't traslate well page 393 of pfsense book!

Thank you so much for your reply, I will be working on this now and let you know the results. I appreciate your time very much. (methinks the PFsense boards needs a "buy me a beer" button)

FYI, this problem was due to a firmware bug in our switch that caused it to stop learning MAC addresses after 49.5 days.

There is not any mention about need of having IP addresses from the same subnet regarding IP Alias in the documentation (http://doc.pfsense.org/index.php/What_are_Virtual_IP_Addresses%3F).

Any ideas if I did something wrong or the mistake sits at the provider side? Or does somebody have an answer to question how am I able to alternate configuration so the publishing of the servers on the public IP addresses from the other subnet could work?

Michal

So I tested the previous code and it works perfectly. Allows the same VHID to be used on the same interface and other interfaces, provided the broadcast domain is different.

And all other functions look to work just fine with the modification of the VIP/Carp interface name.

This is just a quick hack until 2.1 becomes stable.

@rkrenzis:

I can't re-merge the thread because the moderator separated it.  Same issue still exists with the de driver.

Any solution for "kernel: ifa_add_loopback_route: insertion failed" ?

Locking since this thread was resolved, please start a new thread instead of hijacking an old resolved one.

@jimp:

I split this into its own topic. The other topic was sufficiently answered and yours may or may not be the same thing.

If it shows that it's blocking from "self" that implies you have a network loop somewhere, or two interfaces on the same firewall plugged into the same switch/vlan.

Dual master situations are almost always switch/layer 2 related, as mentioned in the other thread.

Maybe is a layer 2 problem.
Today, after the two boxes are upgraded to 2.0.1-stable the log is not anymore on the two boxes.
Now the Master is the Master on the problematic VIP and the Slave is as Slave. That seems to be ok, but the slave cannot communicate with the gateway on that VIP.

I have two physical boxes with two nics each one.
One nic is used for pfsync and the other is splitted into 11 vlans.
That nic is connected to a trunk port on a layer 2 switch.
The Gateway is connected to an access port on the same switch on vlan6 with ip 172.16.0.17/29
The first box has on vlan6 ip 172.16.0.19/29
The second box has on vlan6 ip 172.16.0.20/29
Bot share VIP 172.16.0.18/29

172.16.0.17 can ping two boxes ip but 172.16.0.20 can't ping gateway nor master on vlan6.
Each box uses 172.16.0.18 as source when they ping to 172.16.0.17. I am really confused with this  :(

Also I did tcpdump -i re0_vlan6 -ttt -n proto CARP and get this on two boxes:
Master
00:00:01.017627 IP 172.16.0.19 > 224.0.0.18: VRRPv2, Advertisement, vrid 1, prio 0, authtype none, intvl 1s, length 36
Slave
00:00:01.017052 IP 172.16.0.19 > 224.0.0.18: VRRPv2, Advertisement, vrid 1, prio 0, authtype none, intvl 1s, length 36

Yesterday before the upgrade I had:
First box (Master):
00:00:00.011262 IP 172.16.0.18 > 224.0.0.18: VRRPv2, Advertisement, vrid 1, prio 100, authtype none, intvl 1s, length 36
Second box (Slave):
00:00:00.382580 IP 172.16.0.18 > 224.0.0.18: VRRPv2, Advertisement, vrid 1, prio 100, authtype none, intvl 1s, length 36

You cannot get dynamic virtual IP's.

What you could do:
Add as many interfaces as you can get public IPs, (Alternatively get a VLAN capable switch) and configure the additional interfaces to get their IP via DHCP.

Thanks for the response.
I did figure out how to set the time manually. Times are now within a few seconds of each other.
This seems to have cleared up the time offset issue but I am seeing some strange behavior. From the system Logs on the master:

Oct 9 10:53:52 dhcpd: uid lease 10.5.4.234 for client 64:20:0c:90:1e:70 is duplicate on 10.5.0.0/21
Oct 9 10:53:44 dhcpd: uid lease 10.5.3.9 for client 64:20:0c:96:19:bf is duplicate on 10.5.0.0/21
Oct 9 10:53:31 dhcpd: uid lease 10.5.6.45 for client bc:67:78:12:04:c3 is duplicate on 10.5.0.0/21
Oct 9 10:53:19 dhcpd: uid lease 10.5.7.208 for client 64:20:0c:7d:c2:2c is duplicate on 10.5.0.0/21
Oct 9 10:53:15 dhcpd: uid lease 10.5.3.91 for client d4:20:6d:da:8c:ab is duplicate on 10.5.0.0/21
Oct 9 10:53:09 dhcpd: uid lease 10.5.4.164 for client 74:e2:f5:91:a5:bb is duplicate on 10.5.0.0/21

Even though I had disabled the DHCP services on the slave, I was seeing stuff like the following in the DCHP logs on the master:
Oct 9 10:59:22 dhcpd: DHCPREQUEST for 10.5.3.254 (10.5.1.3) from b0:65:bd:ec:32:12 (iPad) via em1_vlan2

10.5.1.3 is the slave…

Also, if DHCP is enabled on the Slave, I will see lines like the following in the DHCP logs:
dhcpd: DHCPREQUEST for 10.5.4.183 from 88:c6:63:23:e7:86 via em1_vlan2: lease owned by peer

After looking at it some more, it seems like the CARP_WAN is also not setting the master/slave correctly.  I am seeing both machine as the Master for the VIP CARP XXX.XXX.XXX.250 and YYY.YYY.YYY.250  :'(

Ok, Done

Thank you very much. As from now I understand much better how I can mould the pfsense box to our needs.

Thanks again, Fons

Yeah, WAN is set to DHCP and gets the same IP everytime (212.o.o.9)
But every now and then (5-15 days) rc.newwanip detects a VIP as its normal IP and doesn't respond with the 212.o.o.9.
Additionally it's blocking access to a random server as it takes one of those IP addresses.

I'd file that as a bug with Draytek, that's not proper behavior. I've never heard of anything else that behaves that way. It's not just CARP that does that, other routing redundancy protocols are no different.