pfSense cluster and 2 switches
-
Hi,
i have a 2 node pf sense cluster and 2 HP Procurve 2824 Switches.
The switches are not fully stacked and configured with RSTP. (works greate so far).
The pfSense Node are a apu4c4 board, the interfaces assigned as follows:
igb0 -> wan
igb1-2 - lagg (failover)
igb3 -> ha sync portthe pfsense igb1 is connected to switch1 port 1 and igb2 is connected to switch2 port 1,
so the second node is connected so switch1 port2 and switch2 port2.There are several carps configured and sometimes i can't access the web interface of the backup node or very laggy.
Is that a problem with my pfsense lagg config and the connection to the switches?
Regards David
-
Never used those switches but generally to configure a lagg using member ports on two different switches they either need to be stacked or need to implement something usually called Multi-Chassis Trunking or similar.
Ah - failover mode not LACP... In failover mode it might or might not work. Hmm. You might need to dig a little deeper into what exactly is happening when the connectivity issues are occurring.
-
thanks for your answer,
hm, i dont know how i can dig deeper.
Have pfsense a STP functionality for a lagg interface?Regards
-
No. It does not need STP because it will not forward traffic received on one member out another member so it cannot create a loop.
The switches should never block one of the ports going to pfSense since it should never receive a BPDU from them.
Does it work fine with one of the failover links disconnected?
-
@godav said in pfSense cluster and 2 switches:
Procurve 2824 Switches
Any specific reason you're not stacking them?
-
@NogBadTheBad this model only does a configuration stack, not a full logical stack.
-
@Derelict if i disable one failover port (at switch site), the behavior is a little bit better but still laggy or sometimes there happens nothing in de web ui.
-
the problem is strange, if i open a single browser windows to my backup node, all is working great.
if i open a second browser window to my master node the backup node is laggy.i have two carps on that lagg.
One carp on the lagg0, 192.168.12.254 <-- web ui access
and another carp on the lagg0.100, 192.168.11.254 <-- vlan 100 -
i have two new cisco sg500 in a logical stack and connected the two firewalls with a lacp lagg. But my problem still be there. I cant figure it out where the problem is. Sometimes the gui appears and sometime the gui is loading and loading and nothing happens.
-
What is a "logical stack" in this case? Can you LACP to both switches on one LAGG there? You can usually only do that with a physical stack or something like Multi-Chassis Trunking (MCT).
How does it perform if you disconnect one of the LACP member links?
-
It's a physical stack, i have the pfsense-master lagg0 connected to switch port 1/1/1 and 1/1/2, the pfsense-slave is connected to 2/1/1 and 2/1/2. So yes one LAGG per firewall to both switches.
If i disconnect a lacp member link the issue is still the same.
Regards David
-
You should be connecting to 1/1/1 and 2/1/1 to the primary and 1/1/2 and 2/1/2 to the secondary so a switching failure does not blow up the routing cluster.
Really hard to say what you are seeing. You might have to pcap to see who is not responding to whom.
-
@Derelict Is cabled as you say, mentiont it false. :)
i've done a pcap when i can't connect to the gui:
07:25:33.069437 00:0d:b9:50:61:e5 > 00:0d:b9:50:61:c9, ethertype IPv4 (0x0800), length 96: (tos 0x0, ttl 127, id 13064, offset 0, flags [DF], proto TCP (6), length 82) 192.168.11.149.60257 > 192.168.12.2.10443: Flags [P.], cksum 0x943f (correct), seq 1918627193:1918627235, ack 3166548767, win 1026, length 42 07:25:33.075572 00:0d:b9:50:61:e5 > 00:0d:b9:50:61:c9, ethertype IPv4 (0x0800), length 124: (tos 0x0, ttl 127, id 13065, offset 0, flags [DF], proto TCP (6), length 110) 192.168.11.149.60257 > 192.168.12.2.10443: Flags [P.], cksum 0xdfba (correct), seq 42:112, ack 1, win 1026, length 70 07:25:36.764479 00:0d:b9:50:61:e5 > 00:0d:b9:50:61:c9, ethertype IPv4 (0x0800), length 96: (tos 0x0, ttl 127, id 13066, offset 0, flags [DF], proto TCP (6), length 82) 192.168.11.149.60257 > 192.168.12.2.10443: Flags [P.], cksum 0xa55a (correct), seq 112:154, ack 1, win 1026, length 42 07:25:36.773192 00:0d:b9:50:61:e5 > 00:0d:b9:50:61:c9, ethertype IPv4 (0x0800), length 124: (tos 0x0, ttl 127, id 13067, offset 0, flags [DF], proto TCP (6), length 110) 192.168.11.149.60257 > 192.168.12.2.10443: Flags [P.], cksum 0x7e1c (correct), seq 154:224, ack 1, win 1026, length 70 07:25:43.113812 00:0d:b9:50:61:e5 > 00:0d:b9:50:61:c9, ethertype IPv4 (0x0800), length 96: (tos 0x0, ttl 127, id 13068, offset 0, flags [DF], proto TCP (6), length 82) 192.168.11.149.60257 > 192.168.12.2.10443: Flags [P.], cksum 0xcfe5 (correct), seq 224:266, ack 1, win 1026, length 42 07:25:43.124429 00:0d:b9:50:61:e5 > 00:0d:b9:50:61:c9, ethertype IPv4 (0x0800), length 124: (tos 0x0, ttl 127, id 13069, offset 0, flags [DF], proto TCP (6), length 110) 192.168.11.149.60257 > 192.168.12.2.10443: Flags [P.], cksum 0xfd1b (correct), seq 266:336, ack 1, win 1026, length 70 07:25:46.490591 00:0d:b9:50:61:e5 > 00:0d:b9:50:61:c9, ethertype IPv4 (0x0800), length 96: (tos 0x0, ttl 127, id 13070, offset 0, flags [DF], proto TCP (6), length 82) 192.168.11.149.60257 > 192.168.12.2.10443: Flags [P.], cksum 0xc0f6 (correct), seq 336:378, ack 1, win 1026, length 42 07:25:46.502714 00:0d:b9:50:61:e5 > 00:0d:b9:50:61:c9, ethertype IPv4 (0x0800), length 124: (tos 0x0, ttl 127, id 13071, offset 0, flags [DF], proto TCP (6), length 110) 192.168.11.149.60257 > 192.168.12.2.10443: Flags [P.], cksum 0x19c6 (correct), seq 378:448, ack 1, win 1026, length 70 07:25:46.889703 00:0d:b9:50:61:e5 > 00:0d:b9:50:61:c9, ethertype IPv4 (0x0800), length 96: (tos 0x0, ttl 127, id 13072, offset 0, flags [DF], proto TCP (6), length 82) 192.168.11.149.60257 > 192.168.12.2.10443: Flags [P.], cksum 0x2177 (correct), seq 448:490, ack 1, win 1026, length 42 07:25:46.901803 00:0d:b9:50:61:e5 > 00:0d:b9:50:61:c9, ethertype IPv4 (0x0800), length 124: (tos 0x0, ttl 127, id 13073, offset 0, flags [DF], proto TCP (6), length 110) 192.168.11.149.60257 > 192.168.12.2.10443: Flags [P.], cksum 0x071f (correct), seq 490:560, ack 1, win 1026, length 70 07:25:50.268866 00:0d:b9:50:61:e5 > 00:0d:b9:50:61:c9, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 127, id 13074, offset 0, flags [DF], proto TCP (6), length 52) 192.168.11.149.60296 > 192.168.12.2.10443: Flags [S], cksum 0x69f0 (correct), seq 3722673935, win 64240, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0 07:25:50.270682 00:0d:b9:50:61:e5 > 00:0d:b9:50:61:c9, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 127, id 13075, offset 0, flags [DF], proto TCP (6), length 40) 192.168.11.149.60296 > 192.168.12.2.10443: Flags [.], cksum 0x155d (correct), seq 3722673936, ack 695034584, win 1026, length 0 07:25:50.271608 00:0d:b9:50:61:e5 > 00:0d:b9:50:61:c9, ethertype IPv4 (0x0800), length 571: (tos 0x0, ttl 127, id 13076, offset 0, flags [DF], proto TCP (6), length 557) 192.168.11.149.60296 > 192.168.12.2.10443: Flags [P.], cksum 0x9f78 (correct), seq 0:517, ack 1, win 1026, length 517 07:25:50.282487 00:0d:b9:50:61:e5 > 00:0d:b9:50:61:c9, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 127, id 13077, offset 0, flags [DF], proto TCP (6), length 40) 192.168.11.149.60296 > 192.168.12.2.10443: Flags [.], cksum 0x0d09 (correct), seq 517, ack 1616, win 1026, length 0 07:25:50.284896 00:0d:b9:50:61:e5 > 00:0d:b9:50:61:c9, ethertype IPv4 (0x0800), length 180: (tos 0x0, ttl 127, id 13078, offset 0, flags [DF], proto TCP (6), length 166) 192.168.11.149.60296 > 192.168.12.2.10443: Flags [P.], cksum 0x6be7 (correct), seq 517:643, ack 1616, win 1026, length 126 07:25:50.285757 00:0d:b9:50:61:e5 > 00:0d:b9:50:61:c9, ethertype IPv4 (0x0800), length 231: (tos 0x0, ttl 127, id 13079, offset 0, flags [DF], proto TCP (6), length 217) 192.168.11.149.60296 > 192.168.12.2.10443: Flags [P.], cksum 0x9029 (correct), seq 643:820, ack 1616, win 1026, length 177 07:25:50.285830 00:0d:b9:50:61:e5 > 00:0d:b9:50:61:c9, ethertype IPv4 (0x0800), length 317: (tos 0x0, ttl 127, id 13080, offset 0, flags [DF], proto TCP (6), length 303) 192.168.11.149.60296 > 192.168.12.2.10443: Flags [P.], cksum 0x5555 (correct), seq 820:1083, ack 1616, win 1026, length 263 07:25:50.288860 00:0d:b9:50:61:e5 > 00:0d:b9:50:61:c9, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 127, id 13081, offset 0, flags [DF], proto TCP (6), length 40) 192.168.11.149.60296 > 192.168.12.2.10443: Flags [.], cksum 0x0a52 (correct), seq 1083, ack 1745, win 1026, length 0 07:25:50.288896 00:0d:b9:50:61:e5 > 00:0d:b9:50:61:c9, ethertype IPv4 (0x0800), length 92: (tos 0x0, ttl 127, id 13082, offset 0, flags [DF], proto TCP (6), length 78) 192.168.11.149.60296 > 192.168.12.2.10443: Flags [P.], cksum 0x6f7a (correct), seq 1083:1121, ack 1745, win 1026, length 38 07:25:50.324940 00:0d:b9:50:61:e5 > 00:0d:b9:50:61:c9, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 127, id 13083, offset 0, flags [DF], proto TCP (6), length 40) 192.168.11.149.60296 > 192.168.12.2.10443: Flags [.], cksum 0xfaf8 (correct), seq 1121, ack 5636, win 1026, length 0 07:25:50.351593 00:0d:b9:50:61:e5 > 00:0d:b9:50:61:c9, ethertype IPv4 (0x0800), length 219: (tos 0x0, ttl 127, id 13084, offset 0, flags [DF], proto TCP (6), length 205) 192.168.11.149.60296 > 192.168.12.2.10443: Flags [P.], cksum 0xa79c (correct), seq 1121:1286, ack 5636, win 1026, length 165 07:25:50.351846 00:0d:b9:50:61:e5 > 00:0d:b9:50:61:c9, ethertype IPv4 (0x0800), length 143: (tos 0x0, ttl 127, id 13085, offset 0, flags [DF], proto TCP (6), length 129) 192.168.11.149.60296 > 192.168.12.2.10443: Flags [P.], cksum 0x9be4 (correct), seq 1286:1375, ack 5636, win 1026, length 89 07:25:50.351890 00:0d:b9:50:61:e5 > 00:0d:b9:50:61:c9, ethertype IPv4 (0x0800), length 142: (tos 0x0, ttl 127, id 13086, offset 0, flags [DF], proto TCP (6), length 128) 192.168.11.149.60296 > 192.168.12.2.10443: Flags [P.], cksum 0xf9d3 (correct), seq 1375:1463, ack 5636, win 1026, length 88 07:25:50.352155 00:0d:b9:50:61:e5 > 00:0d:b9:50:61:c9, ethertype IPv4 (0x0800), length 159: (tos 0x0, ttl 127, id 13087, offset 0, flags [DF], proto TCP (6), length 145) 192.168.11.149.60296 > 192.168.12.2.10443: Flags [P.], cksum 0x5eaa (correct), seq 1463:1568, ack 5636, win 1026, length 105 07:25:50.352757 00:0d:b9:50:61:e5 > 00:0d:b9:50:61:c9, ethertype IPv4 (0x0800), length 159: (tos 0x0, ttl 127, id 13088, offset 0, flags [DF], proto TCP (6), length 145) 192.168.11.149.60296 > 192.168.12.2.10443: Flags [P.], cksum 0xc624 (correct), seq 1568:1673, ack 5636, win 1026, length 105 07:25:50.352828 00:0d:b9:50:61:e5 > 00:0d:b9:50:61:c9, ethertype IPv4 (0x0800), length 144: (tos 0x0, ttl 127, id 13089, offset 0, flags [DF], proto TCP (6), length 130) 192.168.11.149.60296 > 192.168.12.2.10443: Flags [P.], cksum 0xb4fc (correct), seq 1673:1763, ack 5636, win 1026, length 90 07:25:50.358834 00:0d:b9:50:61:e5 > 00:0d:b9:50:61:c9, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 127, id 13090, offset 0, flags [DF], proto TCP (6), length 40) 192.168.11.149.60296 > 192.168.12.2.10443: Flags [.], cksum 0xd63e (correct), seq 1763, ack 14396, win 1026, length 0 07:25:50.359477 00:0d:b9:50:61:e5 > 00:0d:b9:50:61:c9, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 127, id 13091, offset 0, flags [DF], proto TCP (6), length 40) 192.168.11.149.60296 > 192.168.12.2.10443: Flags [.], cksum 0xbf6e (correct), seq 1763, ack 20236, win 1026, length 0 07:25:50.361409 00:0d:b9:50:61:e5 > 00:0d:b9:50:61:c9, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 127, id 13092, offset 0, flags [DF], proto TCP (6), length 40) 192.168.11.149.60296 > 192.168.12.2.10443: Flags [.], cksum 0x9d36 (correct), seq 1763, ack 28996, win 1026, length 0 07:25:50.361925 00:0d:b9:50:61:e5 > 00:0d:b9:50:61:c9, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 127, id 13093, offset 0, flags [DF], proto TCP (6), length 40) 192.168.11.149.60296 > 192.168.12.2.10443: Flags [.], cksum 0x8666 (correct), seq 1763, ack 34836, win 1026, length 0 07:25:50.364071 00:0d:b9:50:61:e5 > 00:0d:b9:50:61:c9, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 127, id 13094, offset 0, flags [DF], proto TCP (6), length 40) 192.168.11.149.60296 > 192.168.12.2.10443: Flags [.], cksum 0x642e (correct), seq 1763, ack 43596, win 1026, length 0 07:25:50.364418 00:0d:b9:50:61:e5 > 00:0d:b9:50:61:c9, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 127, id 13095, offset 0, flags [DF], proto TCP (6), length 40) 192.168.11.149.60296 > 192.168.12.2.10443: Flags [.], cksum 0x47aa (correct), seq 1763, ack 50896, win 1026, length 0 07:25:50.366374 00:0d:b9:50:61:e5 > 00:0d:b9:50:61:c9, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 127, id 13096, offset 0, flags [DF], proto TCP (6), length 40) 192.168.11.149.60296 > 192.168.12.2.10443: Flags [.], cksum 0x368e (correct), seq 1763, ack 55276, win 1026, length 0 07:25:50.366848 00:0d:b9:50:61:e5 > 00:0d:b9:50:61:c9, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 127, id 13097, offset 0, flags [DF], proto TCP (6), length 40) 192.168.11.149.60296 > 192.168.12.2.10443: Flags [.], cksum 0x08ee (correct), seq 1763, ack 66956, win 1026, length 0 07:25:50.367715 00:0d:b9:50:61:e5 > 00:0d:b9:50:61:c9, ethertype IPv4 (0x0800), length 172: (tos 0x0, ttl 127, id 13098, offset 0, flags [DF], proto TCP (6), length 158) 192.168.11.149.60296 > 192.168.12.2.10443: Flags [P.], cksum 0xb149 (correct), seq 1763:1881, ack 66956, win 1026, length 118 07:25:50.368467 00:0d:b9:50:61:e5 > 00:0d:b9:50:61:c9, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 127, id 13099, offset 0, flags [DF], proto TCP (6), length 40) 192.168.11.149.60296 > 192.168.12.2.10443: Flags [.], cksum 0xe63f (correct), seq 1881, ack 75716, win 1026, length 0 07:25:50.370736 00:0d:b9:50:61:e5 > 00:0d:b9:50:61:c9, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 127, id 13100, offset 0, flags [DF], proto TCP (6), length 40) 192.168.11.149.60296 > 192.168.12.2.10443: Flags [.], cksum 0xc9bb (correct), seq 1881, ack 83016, win 1026, length 0 07:25:50.370924 00:0d:b9:50:61:e5 > 00:0d:b9:50:61:c9, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 127, id 13101, offset 0, flags [DF], proto TCP (6), length 40) 192.168.11.149.60296 > 192.168.12.2.10443: Flags [.], cksum 0xa1cf (correct), seq 1881, ack 93236, win 1026, length 0 07:25:50.373197 00:0d:b9:50:61:e5 > 00:0d:b9:50:61:c9, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 127, id 13102, offset 0, flags [DF], proto TCP (6), length 40) 192.168.11.149.60296 > 192.168.12.2.10443: Flags [.], cksum 0x5d5f (correct), seq 1881, ack 110756, win 1026, length 0 07:25:50.376322 00:0d:b9:50:61:e5 > 00:0d:b9:50:61:c9, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 127, id 13103, offset 0, flags [DF], proto TCP (6), length 40) 192.168.11.149.60296 > 192.168.12.2.10443: Flags [.], cksum 0x18ef (correct), seq 1881, ack 128276, win 1026, length 0 07:25:50.418359 00:0d:b9:50:61:e5 > 00:0d:b9:50:61:c9, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 127, id 13104, offset 0, flags [DF], proto TCP (6), length 40) 192.168.11.149.60296 > 192.168.12.2.10443: Flags [.], cksum 0x133b (correct), seq 1881, ack 129736, win 1026, length 0 07:25:53.417041 00:0d:b9:50:61:e5 > 00:0d:b9:50:61:c9, ethertype IPv4 (0x0800), length 97: (tos 0x0, ttl 127, id 13105, offset 0, flags [DF], proto TCP (6), length 83) 192.168.11.149.60296 > 192.168.12.2.10443: Flags [P.], cksum 0x5085 (correct), seq 1881:1924, ack 129736, win 1026, length 43 07:25:53.417277 00:0d:b9:50:61:e5 > 00:0d:b9:50:61:c9, ethertype IPv4 (0x0800), length 97: (tos 0x0, ttl 127, id 13106, offset 0, flags [DF], proto TCP (6), length 83) 192.168.11.149.60296 > 192.168.12.2.10443: Flags [P.], cksum 0xcf05 (correct), seq 1924:1967, ack 129736, win 1026, length 43 07:25:53.417315 00:0d:b9:50:61:e5 > 00:0d:b9:50:61:c9, ethertype IPv4 (0x0800), length 97: (tos 0x0, ttl 127, id 13107, offset 0, flags [DF], proto TCP (6), length 83) 192.168.11.149.60296 > 192.168.12.2.10443: Flags [P.], cksum 0x12b3 (correct), seq 1967:2010, ack 129736, win 1026, length 43 07:25:53.417344 00:0d:b9:50:61:e5 > 00:0d:b9:50:61:c9, ethertype IPv4 (0x0800), length 97: (tos 0x0, ttl 127, id 13108, offset 0, flags [DF], proto TCP (6), length 83) 192.168.11.149.60296 > 192.168.12.2.10443: Flags [P.], cksum 0xdfb8 (correct), seq 2010:2053, ack 129736, win 1026, length 43
-
@Derelict i think i found a solution. Disabled hardware checksum offload and all is running smooth now!
-
That is an odd thing to have to do using physical nodes but glad you found it.