[SOLVED] Unable To Reach Second pfSense Firewall On LAN
-
I'm in the process of overhauling my company's LAN+WAN setup to accommodate a HA/CARP setup using 2x XG-7100-1U routers. Initially we were unable to continue the setup process as when connecting the secondary firewall to the local network, an apparent switching loop would occur, taking down the entire local network. The only way to stop this loop would be to take the second firewall back down.
Today we were able to solve the switching loop with some switch configurations to put the ports connecting directly to either of the two firewalls (XG-7100s) in "untagged" mode, and the switch ports that connect to other switch ports in "tagged" mode.
After making these changes, we were able to power on the second firewall and not have the local network get taken down. At the moment we are unable to reach the second firewall's LAN IP address from the local network, making us unable to continue the setup process.
Here is a diagram of the LAN switching configuration:
So currently everything is running correctly and there's no switching loops, however we are unable to reach the second firewall's LAN port from the LAN, or even through a VPN.
-
@postables said in Unable To Reach Second pfSense Firewall On LAN:
Today we were able to solve the switching loop with some switch configurations
???
Do the switches not support spanning tree? If not, you have no business using them in a complex network.
-
According to the spec sheet for a gs110tp it does
IEEE 802.1w Rapid Spanning Tree Protocol (RSTP)
IEEE 802.1s Multiple Spanning Trees Protocol (MSTP)Maybe they didn't have it enabled?
But looks like they might have some downstream switches - is the drawing showing multiple connections to the upstream switches - maybe the downstream switches don't support stp? Maybe they are just dumb switches?
The carp info is going to be multicast right - so maybe there is an issue with that? Depending on the switch configs, and the downstream switches, etc.
I don't think those gs110 stack.. Stackable switches prob be a better solution I would think? Hard to tell without more details of the environment and needs and configurations, etc.
-
@johnpoz said in Unable To Reach Second pfSense Firewall On LAN:
maybe the downstream switches don't support stp? Maybe they are just dumb switches?
If so, they shouldn't be used anywhere the potential for loops exists.
-
Yup completely concur.. Sure shouldn't be running multiple links from that that is for damn sure.
-
How are the on-board switches in the XG-7100s configured?
I assume that is a typo and the upstream ports are using ix0 and ix1 since ix2 is an internal port.
Yeah, are those switches stacked?
What VLANs are you using? It sounds like you just created separate layer 2 segments and that prevented the loop. But it would also disconnect the LAN from the firewalls unless those switches are layer 3.
Steve
I would think you are relying entirely on STP here to prevent loops with all the switches connected together.
-
@stephenw10 said in Unable To Reach Second pfSense Firewall On LAN:
How are the on-board switches in the XG-7100s configured?
I assume that is a typo and the upstream ports are using ix0 and ix1 since ix2 is an internal port.
Yeah, are those switches stacked?
What VLANs are you using? It sounds like you just created separate layer 2 segments and that prevented the loop. But it would also disconnect the LAN from the firewalls unless those switches are layer 3.
Steve
I would think you are relying entirely on STP here to prevent loops with all the switches connected together.
I haven't done any special configurations to the on-board switches for the XG-7100s. No ix0+ix1 are being dedicated to pfSync usage, and I'm using the default VLAN for pfSense LANs.
I'll take another crack at configuring STP, and replacing the two switches with ones that can do STP.
@johnpoz said in Unable To Reach Second pfSense Firewall On LAN:
According to the spec sheet for a gs110tp it does
IEEE 802.1w Rapid Spanning Tree Protocol (RSTP)
IEEE 802.1s Multiple Spanning Trees Protocol (MSTP)Maybe they didn't have it enabled?
But looks like they might have some downstream switches - is the drawing showing multiple connections to the upstream switches - maybe the downstream switches don't support stp? Maybe they are just dumb switches?
The carp info is going to be multicast right - so maybe there is an issue with that? Depending on the switch configs, and the downstream switches, etc.
I don't think those gs110 stack.. Stackable switches prob be a better solution I would think? Hard to tell without more details of the environment and needs and configurations, etc.
I tried setting up RSTP on the
netgear-gs-110tp-[1-3]and that didn't seem to solve the switching loop that was happening on the LAN. The only thing that solved the issue was setting the switch ports on the gs110tp's that connected to the switch ports of the XG-7100-1U's.However as stephenw10 pointed out, I think what that did was just create different layer 2 segments.
@JKnott said in Unable To Reach Second pfSense Firewall On LAN:
@postables said in Unable To Reach Second pfSense Firewall On LAN:
Today we were able to solve the switching loop with some switch configurations
???
Do the switches not support spanning tree? If not, you have no business using them in a complex network.
The
netgear-gs-110tp-[1-3], so does thedelta-cisco-2960s-1. I thinknetgear-0[1,2]do not support it. -
@postables said in Unable To Reach Second pfSense Firewall On LAN:
netgear-0[1,2] do not support it.
Are there any loops with those switches?
-
Mmm, those probably are not a problem if they each only have a single connection to one of the GS110s.
From your diagram it looks like you should have switch loops between both firewalls and the GS110s. Two loops on each side.
Steve
-
Yeah looks from his drawing to me that he has 2 lines coming up from those downstream.. You only need 1 loop and the whole thing can come down.. Especially with multiple carps - be a fair amount of multicast being sent out. And lots not forget the amount of broadcast and multicast even a single windows machine can put on the network..
Loops are Very Bad! We had a customer were they would have these idiot users that use to plug the phone in twice.. You know how you can bridge say a pc off a phone.. Well they would have a phone in a conference room and some user would get the smart idea that may it needs both connections plugged in ;)
Dumb switches really shouldn't be used in a work setup, other than maybe a few extra ports on some users desk because they are doing some special project or something.
An intelligent man is sometimes forced to be drunk to spend time with his fools
If you get confused: Listen to the Music Play
Please don't Chat/PM me for help, unless mod related
SG-4860 24.11 | Lab VMs 2.8, 24.11 -
@johnpoz said in Unable To Reach Second pfSense Firewall On LAN:
Dumb switches really shouldn't be used in a work setup, other than maybe a few extra ports on some users desk because they are doing some special project or something.
They still manage to create loops there.
Or they kick out the plug and the help desk phone rings.
LAYER 8 -
@johnpoz said in Unable To Reach Second pfSense Firewall On LAN:
Dumb switches really shouldn't be used in a work setup
I have a Cisco unmanaged switch that supports spanning tree.
-
Then I guess it's not a "dumb switch."
-
@Derelict said in Unable To Reach Second pfSense Firewall On LAN:
Then I guess it's not a "dumb switch."
It's certainly not managed. There's nothing to configure on it. Spanning tree is always on.
-
And what is the make and model of this switch? spanning tree without the ability to "configure" it not all that useful.
I just looked at specs for old sd2005 model and their 110 line - I don't see any spanning tree in the spec sheets.
An intelligent man is sometimes forced to be drunk to spend time with his fools
If you get confused: Listen to the Music Play
Please don't Chat/PM me for help, unless mod related
SG-4860 24.11 | Lab VMs 2.8, 24.11 -
@johnpoz said in Unable To Reach Second pfSense Firewall On LAN:
And what is the make and model of this switch? spanning tree without the ability to "configure" it not all that useful.
Geez. You made me go digging through my junk closet. Almost needed an archaeologist.
It appears I was thinking of another switch. This one is a Cisco SD216.
-
@JKnott said in Unable To Reach Second pfSense Firewall On LAN:
SD216.
That doesn't show any stp support per the spec sheets I can find.
-
@johnpoz said in Unable To Reach Second pfSense Firewall On LAN:
That doesn't show any stp support per the spec sheets I can find.
As I said in my previous post, I must have been thinking of another switch.
-
oh will that makes more sense - some smart/managed switch ;)
-
@johnpoz said in Unable To Reach Second pfSense Firewall On LAN:
spanning tree without the ability to "configure" it not all that useful.
Actually it is, for it's intended purpose of preventing loops. Spanning tree goes all the way back to 1985, which predates switches. Back then, bridges were used to extend coax based networks. There's not much that needs to be configured for basic spanning tree operation. Of course, with the managed switches used these days, things like priority and VLANs have to be configured, but those aren't necessary for a basic LAN.
