Best simple network
-
@dobby_ These switches you link to wouldn't help me with 10 Gbps SPF+ that is my main goal for entire network.
I already have D-link stacked managed switches with 4x SPF+ 10 Gbps and 48 ports 1Gbps. And they do the job, it is not really huge traffic normally. But the goal is to replace 1 Gbps ports with something with maybe 15-20 spf+ 10 Gbps ports instead, to speed up backups and just have everything running on one single type of cables.
This is a single rack in a data center where I run important servers, I can't have any downtime at all.
I have a flat network with no vlans, no segmentation and don't need any routing in my network. pfSense handles everything I need. The goal is not throughput (that's why I don't need LACP lag between each server and switch), but redundancy or at least fail-over that kicks in within a minute automatically. And before you say it, it is bad practice to not segment network, but I do have a valid reason.
With STP protection instead of LACP/lag on everything, I don't need to set up dedicated ports on the switch like I do now for lot of equipment (I setup LACP team on the Stacked Switch stack and then the same on each server - one by one).
All gear has STP protection, but their LAG-support is pretty bad on some of it (with bad, I mean complicated and often ends up with errors because they handle it differently than most modern equipment do). Also, if I replace some gear and plug into wrong port on switch, it is bad. So I wonder if I would get less administration with pure STP instead of the port config I deal with today.
Goal is less admin, more unity (only two switches), redundancy and higher bandwidth (1Gbps->10Gbps) is also a plus. Redundancy and ease of administration is more important than getting for instance 2x20Gbps that I could do in a LACP team.
-
@fireix To be clear, 10G SFP+ switches are popular, and extremely easy to find. What you are trying to find is something in your budget which might be difficult to do. As I stated already, if you want a 48 port 10/25/40/100G capable Layer 2 switch that can operate as a Leaf in a spine/leaf topology, run MC-LAG, or other advanced features then you need to pay.
You already stated that its a few blades. And as i already started off the thread im not sure and you havent stated why you even need so many switches. Maybe there is more to the topology than what you have given.In either case, to be honest, you should probably look at hiring a consultant or speaking to a VAR to find out whats best for your environment. It doesnt seem you have a grasp on the fundamentals here of building a network. Please dont take this personally but you should understand the requirements (budget, bandwidth needs, security,etc.).
-
I know exactly my budget (2x600 USD), the bandwidth needs (1 Gbps is more than enough per port for most, but since I have most stuff with 10 Gbps nic card by now (SPF+), it makes sense to base it on that - and security needs (no changes needed) :)
I have run servers in 3 data centers for 20 years redundant setup and almost no downtime on my part, so I think I can do a few more :) The thing is that once things are running, it requires very little maintenance in such a simple environment. So tend to forget a lot of the basics. I have learned myself enough to set it up and done it one way, but the way I'm now thinking about (make the network more switch-port independent) makes sense to me.
Even Aruba Networks writes in their blog that stacking isn't necessarily the most robust way of doing things (https://blogs.arubanetworks.com/solutions/stacking-network-switches-why-and-why-not/). For instance can upgrading a single switch be impossible without downtime and re-joining can be like auto-magic that is not very transparent.
Do you feel that two switches in a network really are a lot? The only reason for two switches is if one switch is damaged or fell out, I don't want to jump into my car and drive for 20 minutes. Let's say I will use 15-20 servers (all with 10 Gbps SFP+ NIC) to each switch. I have experienced one switch breaking in my 20 years, so it doesn't happen often, but still I can't risk it. Just like my drawing.
Note that running servers like this in a data center is totally different from hostile networks with guest users, DHCP-servers, access points, printers, random PC's/servers and segmentation on IP and VLANs. So it is like a single /24 across entire broadcast domain and none of the host trust each other. That is why I mean by simple. Two switches, one pfSense (with one as backup), one /24. I think you misunderstand what I'm asking for. In a normal office network, I would have have some stuff on VLAN, different networks/subnets for the different departments and real IP-segmentation. This is not what I need. Totally everybody can see everybody-network.
The question is: Why won't running a network with two switches and one pfSense not work with spanning tree protocol enabled? All my research says that it will work. Stacking and the recommended method is the current setup I have today - and provides single management - but I do see possible administration benefits from going away from this model. I don't know why layer2 and layer3 is brought into this. My current switches are L2+L3, but I didn't need any modifications other than setting up LACP-teams on both sides. With my easier setup, I would break the LACP-setup on both sides. Since I don't have multiple networks, I shouldn't need all the features in an expensive switch? Everybody in my network can talk to everybody - and that is what I want it to be.
-
@fireix since your switches are not stacked - my guess on this would be you would setup a lagg but use the failover mode of setup, so only really the active port would be used.. But if that port failed/went down then the other port in the lagg would be used. In this mode you don't need to setup any lacp on the switch side of the connection.
I would for sure test this, and might not account for all kinds of failure modes. But should allow for say if one of the switches going dark..
-
@johnpoz said in Best simple network:
@fireix since your switches are not stacked - my guess on this would be you would setup a lagg but use the failover mode of setup, so only really the active port would be used.. But if that port failed/went down then the other port in the lagg would be used. In this mode you don't need to setup any lacp on the switch side of the connection.
I would for sure test this, and might not account for all kinds of failure modes. But should allow for say if one of the switches going dark..
I assume you mean to setup both the ports on each switch and the device almost like LACP (just changing LAG mode to active-backup), that kind of defeat my ideal purpose I hope for :) My goal would be to not have to configure the switch ports at all.
I'm thinking of this featured called switch independent lag in Windows (there is similar in Linux). It requires no switch setup/port-setup at all on the switch. It is only using one of the ports at a time and if one loose connection, it will activate the other one. It can be two totally different switches and based on what I have read - it has not a requirement on pre-configuration of the switch itself at all.
I'm aware that I would "miss" the nice feature of having 2 Gbps or 20 Gbps when both are active in a LACP team, but it would be easier to manage/cable and to be honest it would care if the speed is doubled.
When I have been lazy and couldn't figure out to setup LACP team due to old hardware, I have actually run with this mode and haven't experienced any issue so far. When I for instance has deactivated the port in the switch, the other one has started sending traffic and work. But not sure if this is luck since the rest of my network is LACP or it is how it actually supposed to work. I assume Windows (in this case) simply is pinging the gw I have set up on the team connection and if ping stops, it sends over the other one. Should be very easy to code. This will also ensure that two nics on same server never is active at same time, so wouldn't even trigger any shutdown of links by using STP (I assume).
".. the NIC Team works independently of the switch; no additional configuration of network hardware is needed. If this mode is on, you can connect different network adapters to different switches to improve fault tolerance (protection against switch failure"
https://4sysops.com/archives/nic-teaming-in-windows-server-2022/
-
@fireix said in Best simple network:
each switch and the device almost like LACP
no that would be the lag setup on pfsense, would put it failover mode.
https://docs.netgate.com/pfsense/en/latest/interfaces/lagg.html
How else would pfsense see traffic on multiple interface, or the possibility of traffic on multiple interfaces that were 1 interface to pfsense. You could setup a bridge I guess on pfsense. And just hope the switch shuts down one of the interfaces..
-
@johnpoz Ah, you talked about the pfSense setup. Yes, there I agree, they must be in brige (used that setup before) or LAG (use now against D-links stacked switches).
But on the other end, between the two unconfigured switches (maybe with exception of lag setup against pfSense) and servers: Windows will ensure only one NIC is enabled at any time, so shouldn't ever be a situation where one server sends from both their nics at the same time.
-
Even Unify migth be a good option it seems.
Just a tiny but above budget, but at least easy management.
https://eu.store.ui.com/collections/unifi-network-routing-switching/products/unifi-switch-aggregation-pro
-
- 2 x Netgear XS708E
For 500 € used over eBay - 3 x HPE 561T V2 Dual Port 10G RJ45 (X540)
For 120 € each, refurbed from server store - 10 x CAT6A FTP/S (PIMF) 1200MHz patch cables
To ensure the full 10 GBit/s are given
All in all for 860 € plus fresh cables. No problems
with optics.If you only flash the newest firmware on the Netgear switches and don´t configure them, they will act as an
dump switch you don´t have to configure anything then!Server port 1 - 192.168.2.0/24
Server port 2 - 192.168.1.0/24No STP will be needed and all is done in pure routing.
- 2 x Netgear XS708E
-
@dobby_ I already have invested and put in SPF+ fiber cards and for blade servers - I can't switch out a card. Also 8 ports are about 8 ports to little :) Also, I would prefer a modern equipment, it is an important reason also. My current D-Link stackable is just a bit newer than these and firmware updates are ended long time ago.
