STP and network
-
The inside addresses just need to be routed to the CARP VIP and not to one of the interface addresses.
-
Should I define a LACP-lag on the switch for each server or isn't that needed at all?
And as far I can tell, it works without LACP and I can remove one link and traffic still happens (but maybe switches get confused? Even though it says switch independent). But if it can creates weird situations, I wouldn't want to keep it that way.
From the Wiki for Centos, it seems like mode 4=802.3ad is the switch dependent mode on Linux: https://wiki.centos.org/TipsAndTricks/BondingInterfaces - but it will only have one active connection at a time.
But don't know what is best to choose in my case.
-
Should I define a LACP-lag on the switch for each server or isn't that needed at all?
I would. What happens when an entire switch fails. Understand that the LACP is Layer 2 redundancy, not layer 3. For layer 3 redundancy you don't need the LACP at all.
And as far I can tell, it works without LACP and I can remove one link and traffic still happens (but maybe switches get confused? Even though it says switch independent). But if it can creates weird situations, I wouldn't want to keep it that way.
When you remove what link?
From the Wiki for Centos, it seems like mode 4=802.3ad is the switch dependent mode on Linux: https://wiki.centos.org/TipsAndTricks/BondingInterfaces - but it will only have one active connection at a time.
Each side of an LACP link generally has a method of deciding what traffic it sends over what link. A combination of MAC address, IP address, and sometimes even port.
But don't know what is best to choose in my case.
Nor do I really.
-
I'm only using layer2 on this network.
"I would. What happens when an entire switch fails."
I don't mean that I should only connect to one switch, just if I should set up lag (either LACP or static) BOTH on switch and on server. There is a limit to number of LACP-team on the switch and a lot of administration to keep track of this. It is faster to just use the vendor-indepenendent solution, like mode=6 (balance-alb) on that wiki-page. I can connect each of the network-ports to different switch. "The interfaces are bonded in the Adaptive Load Balancing mode which supports both outgoing and recieving load balancing as well as failure support. This mode does not require any special switch support and is said to achieve load balancing by ARP negotiation."
"When you remove what link?"
One of the two network connections on the server.
-
Note that the switches now are in cluster, with 10G SPF+ between them.
I do see it might be an issue with having the non-switch configured method and not using the 802.3-method when there are one server against two different switches. This is how I have it today basically (but I don't have any major issues, at least none I can pinpoint to this).
I find a lot 802.3 post where it is first said it is switch-spesific, but once they see documentation, that says it is vendor-indepenendent, they are more open up for it. So it seems like a lot of opinions on this. Some think the active-backup method is better in that cases…
For me, it looks like the non-802.3 method kind of works like a failover HA. Maybe it could cause problems even in switch-cluster.
https://serverfault.com/questions/406672/link-bonding-across-multiple-switches
http://useopensource.blogspot.no/2010/02/linux-nic-teaming-recommendations.html
-
Well, your server issues are not pfSense issues. Need to do whatever it is that they support.
