HA/CARP/VIPs

Your browser does not seem to support JavaScript. As a result, your viewing experience will be diminished, and you have been placed in read-only mode.

Please download a browser that supports JavaScript, or enable it if it's disabled (i.e. NoScript).

D

Firewall Issue

• • davidjtsteele

1

0
Votes

1
Posts

1.1k
Views

No one has replied
N

PfSense loadbalance and failover

• • nicolas010

3

0
Votes

3
Posts

2.6k
Views

G

You can use CARP for failover but you cannot load balance pfsense servers. You can load balance webservers with pfsense but not pfsense its self.
E

Master stops sending CARP advertisements after HTTP-based load test

• • echoranger

2

0
Votes

2
Posts

1.4k
Views

E

Doing some more digging it appears a watchdog timeout may have been the initial culprit:

pfsense01:/var/log/system.log:
Jun 10 13:28:05 pfsense01 kernel: vip36: MASTER -> BACKUP (more frequent advertisement received) Jun 10 13:28:05 pfsense01 kernel: vip36: link state changed to DOWN Jun 10 13:28:05 pfsense01 kernel: vip4: MASTER -> BACKUP (more frequent advertisement received) Jun 10 13:28:05 pfsense01 kernel: vip4: link state changed to DOWN Jun 10 13:28:05 pfsense01 kernel: vip3: MASTER -> BACKUP (more frequent advertisement received) Jun 10 13:28:05 pfsense01 kernel: vip3: link state changed to DOWN Jun 10 13:28:05 pfsense01 kernel: vip11: MASTER -> BACKUP (more frequent advertisement received) Jun 10 13:28:05 pfsense01 kernel: vip11: link state changed to DOWN Jun 10 13:28:05 pfsense01 kernel: vip2: MASTER -> BACKUP (more frequent advertisement received) Jun 10 13:28:05 pfsense01 kernel: vip2: link state changed to DOWN Jun 10 13:28:08 pfsense01 kernel: bge0: watchdog timeout -- resetting Jun 10 13:28:08 pfsense01 check_reload_status: Linkup starting bge0 Jun 10 13:28:08 pfsense01 kernel: vip1: INIT -> BACKUP Jun 10 13:28:08 pfsense01 kernel: vip5: INIT -> BACKUP Jun 10 13:28:08 pfsense01 kernel: vip6: INIT -> BACKUP Jun 10 13:28:08 pfsense01 kernel: vip7: INIT -> BACKUP Jun 10 13:28:08 pfsense01 kernel: vip8: INIT -> BACKUP Jun 10 13:28:08 pfsense01 kernel: vip9: INIT -> BACKUP Jun 10 13:28:08 pfsense01 kernel: vip12: INIT -> BACKUP Jun 10 13:28:08 pfsense01 kernel: vip14: INIT -> BACKUP Jun 10 13:28:08 pfsense01 kernel: vip15: INIT -> BACKUP Jun 10 13:28:08 pfsense01 kernel: vip16: INIT -> BACKUP Jun 10 13:28:08 pfsense01 kernel: vip17: INIT -> BACKUP Jun 10 13:28:08 pfsense01 kernel: vip18: INIT -> BACKUP Jun 10 13:28:08 pfsense01 kernel: vip20: INIT -> BACKUP Jun 10 13:28:08 pfsense01 kernel: vip21: INIT -> BACKUP Jun 10 13:28:08 pfsense01 kernel: vip22: INIT -> BACKUP Jun 10 13:28:08 pfsense01 kernel: vip23: INIT -> BACKUP Jun 10 13:28:08 pfsense01 kernel: vip24: INIT -> BACKUP Jun 10 13:28:08 pfsense01 kernel: vip25: INIT -> BACKUP Jun 10 13:28:08 pfsense01 kernel: vip26: INIT -> BACKUP Jun 10 13:28:08 pfsense01 kernel: vip27: INIT -> BACKUP Jun 10 13:28:08 pfsense01 kernel: vip28: INIT -> BACKUP Jun 10 13:28:08 pfsense01 kernel: vip29: INIT -> BACKUP Jun 10 13:28:08 pfsense01 kernel: vip31: INIT -> BACKUP Jun 10 13:28:08 pfsense01 kernel: vip32: INIT -> BACKUP Jun 10 13:28:08 pfsense01 kernel: vip33: INIT -> BACKUP Jun 10 13:28:08 pfsense01 kernel: vip34: INIT -> BACKUP Jun 10 13:28:08 pfsense01 kernel: vip35: INIT -> BACKUP Jun 10 13:28:08 pfsense01 kernel: vip37: INIT -> BACKUP Jun 10 13:28:08 pfsense01 kernel: vip38: INIT -> BACKUP
Any ideas? I know watchdog timeouts are about as nebulous as ever on FreeBSD but I'm hoping there's a sysctl or something that I can tweak to help prevent this. I've already verified hw.bge.allow_asf is set to 0:
[2.0.3-RELEASE][admin@pfsense01]/var/log(38): sysctl hw.bge hw.bge.allow_asf: 0 [2.0.3-RELEASE][admin@pfsense01]/var/log(35): sysctl -a | grep bge <5>bge1: link state changed to UP <5>bge1_vlan1: link state changed to UP <5>bge1_vlan100: link state changed to UP <5>bge1_vlan20: link state changed to UP <5>bge1_vlan10: link state changed to UP bge0: watchdog timeout -- resetting bge0: 2 link states coalesced <5>bge0: link state changed to UP bge1: watchdog timeout -- resetting <5>bge1: link state changed to DOWN <5>bge1_vlan1: link state changed to DOWN <5>bge1_vlan100: link state changed to DOWN <5>bge1_vlan20: link state changed to DOWN <5>bge1_vlan10: link state changed to DOWN <5>bge1: link state changed to UP <5>bge1_vlan1: link state changed to UP <5>bge1_vlan100: link state changed to UP <5>bge1_vlan20: link state changed to UP <5>bge1_vlan10: link state changed to UP bge0: watchdog timeout -- resetting bge0: 2 link states coalesced <5>bge0: link state changed to UP bge1: watchdog timeout -- resetting <5>bge1: link state changed to DOWN <5>bge1_vlan1: link state changed to DOWN <5>bge1_vlan100: link state changed to DOWN <5>bge1_vlan20: link state changed to DOWN <5>bge1_vlan10: link state changed to DOWN <5>bge1: link state changed to UP <5>bge1_vlan1: link state changed to UP <5>bge1_vlan100: link state changed to UP <5>bge1_vlan20: link state changed to UP <5>bge1_vlan10: link state changed to UP bge1: watchdog timeout -- resetting <5>bge1: link state changed to DOWN <5>bge1_vlan1: link state changed to DOWN <5>bge1_vlan100: link state changed to DOWN <5>bge1_vlan20: link state changed to DOWN <5>bge1_vlan10: link state changed to DOWN <5>bge1: link state changed to UP <5>bge1_vlan1: link state changed to UP <5>bge1_vlan100: link state changed to UP <5>bge1_vlan20: link state changed to UP <5>bge1_vlan10: link state changed to UP bge1: watchdog timeout -- resetting <5>bge1: link state changed to DOWN <5>bge1_vlan1: link state changed to DOWN <5>bge1_vlan100: link state changed to DOWN <5>bge1_vlan20: link state changed to DOWN <5>bge1_vlan10: link state changed to DOWN <5>bge1: link state changed to UP <5>bge1_vlan1: link state changed to UP <5>bge1_vlan100: link state changed to UP <5>bge1_vlan20: link state changed to UP <5>bge1_vlan10: link state changed to UP bge1: watchdog timeout -- resetting <5>bge1: link state changed to DOWN <5>bge1_vlan1: link state changed to DOWN <5>bge1_vlan100: link state changed to DOWN <5>bge1_vlan20: link state changed to DOWN <5>bge1_vlan10: link state changed to DOWN <5>bge1: link state changed to UP <5>bge1_vlan1: link state changed to UP <5>bge1_vlan100: link state changed to UP <5>bge1_vlan20: link state changed to UP <5>bge1_vlan10: link state changed to UP <6>arp: 192.168.1.64 moved from 1c:e6:2b:5f:e2:dd to 20:16:d8:5d:c2:34 on bge1_vlan1 <6>arp: 192.168.1.64 moved from 20:16:d8:5d:c2:34 to 1c:e6:2b:5f:e2:dd on bge1_vlan1 <6>arp: 192.168.1.38 moved from 74:2f:68:e9:0b:ba to 54:04:a6:4a:38:b5 on bge1_vlan1 <6>arp: 192.168.1.78 moved from 8c:70:5a:7b:39:1c to c4:85:08:1b:3b:a4 on bge1_vlan1 <6>arp: 192.168.1.78 moved from c4:85:08:1b:3b:a4 to 8c:70:5a:7b:39:1c on bge1_vlan1 bge1: watchdog timeout -- resetting <5>bge1: link state changed to DOWN <5>bge1_vlan1: link state changed to DOWN <5>bge1_vlan100: link state changed to DOWN <5>bge1_vlan20: link state changed to DOWN <5>bge1_vlan10: link state changed to DOWN <5>bge1: link state changed to UP <5>bge1_vlan1: link state changed to UP <5>bge1_vlan100: link state changed to UP <5>bge1_vlan20: link state changed to UP <5>bge1_vlan10: link state changed to UP bge1: watchdog timeout -- resetting <5>bge1: link state changed to DOWN <5>bge1_vlan1: link state changed to DOWN <5>bge1_vlan100: link state changed to DOWN <5>bge1_vlan20: link state changed to DOWN <5>bge1_vlan10: link state changed to DOWN <5>bge1: link state changed to UP <5>bge1_vlan1: link state changed to UP <5>bge1_vlan100: link state changed to UP <5>bge1_vlan20: link state changed to UP <5>bge1_vlan10: link state changed to UP bge0: watchdog timeout -- resetting bge0: 2 link states coalesced <5>bge0: link state changed to UP <6>arp: 192.168.1.78 moved from 8c:70:5a:7b:39:1c to c4:85:08:1b:3b:a4 on bge1_vlan1 <6>arp: 192.168.1.78 moved from c4:85:08:1b:3b:a4 to 8c:70:5a:7b:39:1c on bge1_vlan1 bge0: watchdog timeout -- resetting bge0: 2 link states coalesced <5>bge0: link state changed to UP hw.bge.allow_asf: 0 dev.bge.0.%desc: Broadcom NetXtreme Gigabit Ethernet Controller, ASIC rev. 0x002002 dev.bge.0.%driver: bge dev.bge.0.%location: slot=0 function=0 dev.bge.0.%pnpinfo: vendor=0x14e4 device=0x1648 subvendor=0x1028 subdevice=0x014a class=0x020000 dev.bge.0.%parent: pci2 dev.bge.0.forced_collapse: 0 dev.bge.0.stats.FramesDroppedDueToFilters: 0 dev.bge.0.stats.DmaWriteQueueFull: 0 dev.bge.0.stats.DmaWriteHighPriQueueFull: 0 dev.bge.0.stats.NoMoreRxBDs: 0 dev.bge.0.stats.InputDiscards: 0 dev.bge.0.stats.InputErrors: 1 dev.bge.0.stats.RecvThresholdHit: 325594 dev.bge.0.stats.DmaReadQueueFull: 505 dev.bge.0.stats.DmaReadHighPriQueueFull: 0 dev.bge.0.stats.SendDataCompQueueFull: 428 dev.bge.0.stats.RingSetSendProdIndex: 250750 dev.bge.0.stats.RingStatusUpdate: 467063 dev.bge.0.stats.Interrupts: 467063 dev.bge.0.stats.AvoidedInterrupts: 0 dev.bge.0.stats.SendThresholdHit: 0 dev.bge.0.stats.rx.Octets: 77286738 dev.bge.0.stats.rx.Fragments: 0 dev.bge.0.stats.rx.UcastPkts: 250315 dev.bge.0.stats.rx.MulticastPkts: 324548 dev.bge.0.stats.rx.FCSErrors: 1 dev.bge.0.stats.rx.AlignmentErrors: 0 dev.bge.0.stats.rx.xonPauseFramesReceived: 0 dev.bge.0.stats.rx.xoffPauseFramesReceived: 0 dev.bge.0.stats.rx.ControlFramesReceived: 0 dev.bge.0.stats.rx.xoffStateEntered: 0 dev.bge.0.stats.rx.FramesTooLong: 0 dev.bge.0.stats.rx.Jabbers: 0 dev.bge.0.stats.rx.UndersizePkts: 0 dev.bge.0.stats.rx.inRangeLengthError: 0 dev.bge.0.stats.rx.outRangeLengthError: 0 dev.bge.0.stats.tx.Octets: 21715826 dev.bge.0.stats.tx.Collisions: 0 dev.bge.0.stats.tx.XonSent: 0 dev.bge.0.stats.tx.XoffSent: 0 dev.bge.0.stats.tx.flowControlDone: 0 dev.bge.0.stats.tx.InternalMacTransmitErrors: 0 dev.bge.0.stats.tx.SingleCollisionFrames: 0 dev.bge.0.stats.tx.MultipleCollisionFrames: 0 dev.bge.0.stats.tx.DeferredTransmissions: 0 dev.bge.0.stats.tx.ExcessiveCollisions: 0 dev.bge.0.stats.tx.LateCollisions: 0 dev.bge.0.stats.tx.UcastPkts: 251179 dev.bge.0.stats.tx.MulticastPkts: 88 dev.bge.0.stats.tx.BroadcastPkts: 14 dev.bge.0.stats.tx.CarrierSenseErrors: 0 dev.bge.0.stats.tx.Discards: 0 dev.bge.0.stats.tx.Errors: 0 dev.bge.1.%desc: Broadcom NetXtreme Gigabit Ethernet Controller, ASIC rev. 0x002002 dev.bge.1.%driver: bge dev.bge.1.%location: slot=0 function=1 dev.bge.1.%pnpinfo: vendor=0x14e4 device=0x1648 subvendor=0x1028 subdevice=0x014a class=0x020000 dev.bge.1.%parent: pci2 dev.bge.1.forced_collapse: 0 dev.bge.1.stats.FramesDroppedDueToFilters: 0 dev.bge.1.stats.DmaWriteQueueFull: 288263 dev.bge.1.stats.DmaWriteHighPriQueueFull: 0 dev.bge.1.stats.NoMoreRxBDs: 0 dev.bge.1.stats.InputDiscards: 5229 dev.bge.1.stats.InputErrors: 647 dev.bge.1.stats.RecvThresholdHit: 494125434 dev.bge.1.stats.DmaReadQueueFull: 37978 dev.bge.1.stats.DmaReadHighPriQueueFull: 2 dev.bge.1.stats.SendDataCompQueueFull: 21 dev.bge.1.stats.RingSetSendProdIndex: 860426035 dev.bge.1.stats.RingStatusUpdate: 783311166 dev.bge.1.stats.Interrupts: 783311166 dev.bge.1.stats.AvoidedInterrupts: 0 dev.bge.1.stats.SendThresholdHit: 0 dev.bge.1.stats.rx.Octets: 1583585081 dev.bge.1.stats.rx.Fragments: 73 dev.bge.1.stats.rx.UcastPkts: 786142776 dev.bge.1.stats.rx.MulticastPkts: 395883 dev.bge.1.stats.rx.FCSErrors: 577 dev.bge.1.stats.rx.AlignmentErrors: 0 dev.bge.1.stats.rx.xonPauseFramesReceived: 3438 dev.bge.1.stats.rx.xoffPauseFramesReceived: 147597 dev.bge.1.stats.rx.ControlFramesReceived: 0 dev.bge.1.stats.rx.xoffStateEntered: 0 dev.bge.1.stats.rx.FramesTooLong: 0 dev.bge.1.stats.rx.Jabbers: 0 dev.bge.1.stats.rx.UndersizePkts: 0 dev.bge.1.stats.rx.inRangeLengthError: 0 dev.bge.1.stats.rx.outRangeLengthError: 0 dev.bge.1.stats.tx.Octets: 181809985 dev.bge.1.stats.tx.Collisions: 0 dev.bge.1.stats.tx.XonSent: 0 dev.bge.1.stats.tx.XoffSent: 0 dev.bge.1.stats.tx.flowControlDone: 0 dev.bge.1.stats.tx.InternalMacTransmitErrors: 0 dev.bge.1.stats.tx.SingleCollisionFrames: 0 dev.bge.1.stats.tx.MultipleCollisionFrames: 0 dev.bge.1.stats.tx.DeferredTransmissions: 0 dev.bge.1.stats.tx.ExcessiveCollisions: 0 dev.bge.1.stats.tx.LateCollisions: 0 dev.bge.1.stats.tx.UcastPkts: 857839350 dev.bge.1.stats.tx.MulticastPkts: 2032515 dev.bge.1.stats.tx.BroadcastPkts: 21415 dev.bge.1.stats.tx.CarrierSenseErrors: 0 dev.bge.1.stats.tx.Discards: 0 dev.bge.1.stats.tx.Errors: 0 dev.miibus.0.%parent: bge0 dev.miibus.1.%parent: bge1
J

Can't get simple VIP to work

• • Jeremy11one

1

0
Votes

1
Posts

1.4k
Views

No one has replied
C

Can't get DHCP failover to work [pictures of setup/config]

• • cmcdonald

3

0
Votes

3
Posts

1.9k
Views

J

@vbman213:

Okay, I think I figured this out based on a really old post from 2011. Apparently, when enabling DHCP Failover, you only specify the failover IP on the first interface performing DHCP, not every single interface. Is this correct?

You must specify a failover IP on every interface that does DHCP, on both the master and slave.
G

[SOLVED] CARP announces fancy IP addresses

• • galphanet

4

0
Votes

4
Posts

1.7k
Views

G

Ok, many thanks for your explanation. :)
H

Firewall: Virtual IP Address: Edit -> Base not config restored on 2.0.3

• • hcoin

1

0
Votes

1
Posts

1.7k
Views

No one has replied
D

Using CARP with Proxy-ARP and 1:1 NAT

• • Dregnus

2

0
Votes

2
Posts

1.8k
Views

D

Okay, after doing a bit more searching it looks like in a redundant firewall setup you cannot use Proxy-ARP, for exactly the reason I was thinking, that you will both firewalls responding to the request thereby creating issues.

It seems the solution is to create a CARP interface for each 1:1 NAT. Additionally, in 2.x you can create an IP Alias and tie that alias to the primary CARP interface. This is what I was trying to do with the Proxy-ARP configuration but couldn't.

I'll give this setup a test and let everyone know if it works out.
M

Error when trying to configure CARP settings

• • mhofmann

2

0
Votes

2
Posts

1.0k
Views

M

I can reproduce it. After I had removed the squid3 package the error occured. Reinstall the package - it work's, remove it - error and so on.

It works for me…

Bye
C

VIP redirects to pfsense login after changes are made

• • claystuckey

3

0
Votes

3
Posts

2.0k
Views

P

This has happened to me to in 2.1x.
M

CARP with Ipsec VPN problem

• • mmc18

2

0
Votes

2
Posts

2.0k
Views

M

Hi mmc18,

I have exactly the same problem. Have you fixed it yet? Maybe we need to change outbound NAT rules?
F

No NAT for Backup FW's webUI?

• • Fr3d

2

0
Votes

2
Posts

1.3k
Views

C

generally that's because you're NATing the secondary firewall's Internet traffic to a CARP IP. Not clear from that screenshot which rule would be doing that given I'm not sure what the WAN IP is, but just make sure you're not NATing traffic sourced from either firewall's WAN IP.
G

CARP WAN Gateway Up for Box 1 but down for Box 2

• • gabrielpc1190

3

0
Votes

3
Posts

2.0k
Views

J

The docs recommend setting (or, rather, not changing) the source away from the specific networks.

Using "any" as a source for outbound NAT is almost always going to have bad and/or unintended results, with or without CARP, but especially with CARP.

If you want to get away with only using one outbound NAT rule, make an alias containing your local/internal networks and use that as the source, not 'any'.
M

Automation Update?

• • mebdnaconet

1

0
Votes

1
Posts

1.2k
Views

No one has replied
G

SOLVED - Questions about blocked IP setup with CRAP

• • galphanet

4

0
Votes

4
Posts

1.6k
Views

S

@galphanet:

BUT this give the active gateway access to Internet, not the secondary one.
[/quote
correct, because the secondary doesn't have the CARP addresss. You would need internet access for the secondary addresses if you need both firewalls on the internet all the time.
J

CARP, 1% CPU Usage, Web Interface Very Slow

• • jamesb

4

0
Votes

4
Posts

2.4k
Views

J

I have fixed my weird web interface time outs now. It seems just one was being particularly slow, the BACKUP CARP device, the MASTER was OK.

I took the compact flash card out of the BACKUP ALIX board and put it in a third board of the exact same spec, still slow. I then wiped and re-flashed the card, placed it back into it's original BACKUP ALIX board, and restored the config from which I had backed up to my laptop. Now its as fast as the MASTER and everything is fine, so perhaps it was a bad flash.

That may seem a bit obvious, and a waist of time posting here, but we have 40+ ALIX & WRAP boards, this has happened before.

Hopefully that will help someone else!

Cheers,
JamesB.
G

Setting up CARP with 2 WAN upinks

• • glennonline

5

0
Votes

5
Posts

2.2k
Views

J

It's best to have both ISPs connected to both units, that way you get the benefit of Multi-WAN redundancy and CARP. If both IPs are dynamic, that's not necessarily fatal if you're willing to put up with double NAT. You can put the ISP modems into router mode, setup a "DMZ" in them to point all traffic at the CARP address you make for that WAN, and so long as easy ISP modem/router is using a different subnet and a separate switch/VLAN, and you set a monitor IP on each to something on the Internet somewhere, it can still work.

Not as pretty as having a /29 to use on each WAN, but it would get the job done.

You could use a separate ISP on each CARP node, but you wouldn't get Multi-WAN failover. If the ISP on WAN1 failed, it wouldn't make the cluster fail to the secondary node unless you power off the modem there manually.
E

Sync inconsistencies seen in backup files 2.0.3

• • eduardr

2

0
Votes

2
Posts

1.4k
Views

J

Nearly all of those are cases of settings that do not sync.

Only the specific settings listed in the CARP/HA sync options will sync, and that does not include anything in System > General, System > Advanced, interface settings, and so on.

Make sure you have all of the areas checked that you want to sync, or they won't sync.
N

How many pfSense can I apply CARP to

• • nicolas010

3

0
Votes

3
Posts

1.8k
Views

R

@nicolas010:

Hi all, I have this problem, I already have 2 pfSense in CARP with 4 interfaces each machine, but I need to apply CARP to other 4 pfSense with 3 interfaces each, but they are not phisically near the first 2. And these other 4 should be master as well […]

If CARP interfaces don't match in same network segments it make no sense to "carp" over both networks ?

I can guess what you want - as our 2 office setup it would be nice to configure with 1 Master "configuration" machine both places with each master/slave failover.
But on the other hand it's better to have both places their own CARP regulation because you can have only up to 256 CARP addresses per Setup => on /24 and your finished ;)

I've found here in forum some weeks ago a hint that there is a plan (sometimes in this century? ;)) to deploy a "cluster"-configuration tool for pfsense so that many pfsense clusters can managed at one place… that would be great ;)
B

Ping VIP without using 1:1 NAT ?

• • brasilnut

3

0
Votes

3
Posts

1.7k
Views

B

@cmb:

1:1 NAT doesn't expose everything, only what your firewall rules permit. With CARP or IP alias VIPs you can let the firewall respond to pings on them.

:o
PERFECT !

Works great.
::) I just assumed that 1:1 opened everything… ::)

Thank You so much...
:D

88 / 129