Do I need 1 or 2 Smart Switches?

mifronte

First, I hope I am posting in the correct section. I chose this section because this pertains to altering my WiFi network to support wired devices. Please move to a more appropriate section if needed.

Do I need 1 or 2 smart switches given my current configuration and my desired goals?

Current Configuration:
pfSense ---LAN Interface (subnet 1)--->Unmanaged Switch--->Trusted network
    |
    |
    |---Opt1 Interface -------->UniFi AP---->Untag (subnet 2)---> Trusted WiFi
                                      |
                                      |
                                      |-------->Tag VLAN 20 (subnet 3) --->Untrusted WiFi

Goals:
1. Use PoE+ to support more PoE devices on any part of my network.
2. I would like to add wired devices to Untrusted WiFi. Right now I can't since the UniFi AP is connected to pfSense directly.
3. I would like to replace the unmanaged switch with a managed switch, for now link aggregation. Future, more segmentation of LAN.

Questions:
A. Can one layer 3 switch with PoE+ handle everything or do I need 2 switches, one for LAN and one for Opt1?

B. Ideally 16-24 gigabit ports total with maybe 5-8 PoE+ ports. Fan less is preferred to avoid rewiring, but not absolutely required. I was thinking of the Cisco SG-300 line of routers. Are there better alternatives?

C. More follow-up questions once I identify a switch or switches to use.

Thanks.

jahonix

@mifronte:

A. Can one layer 3 switch with PoE+ handle everything or do I need 2 switches, one for LAN and one for Opt1?

Layer2 vs L3 is a different beast. It's management capabilitie is what counts here.
You can get this working with a single managed switch. Probably even easier than 2 devices, once you understand VLANs. Keeps it simple.

@mifronte:

B. Ideally 16-24 gigabit ports total with maybe 5-8 PoE+ ports. Fan less is preferred to avoid rewiring, but not absolutely required. I was thinking of the Cisco SG-300 line of routers. Are there better alternatives?

There is no "better" or "worse", only "different" in many aspects.
Cisco's SG-300 line is great with limitations (and will be superseded by SG-350 in the foreseeable future).
SG-300 PoE switches with >21 ports will have a fan, the SG300-10 models (P/PP/MP) have a hated power brick PSU, not an internal one with IEC connector. But therefore no fan…

So much for the switch side of your post.
Your thread will be about VLANs pretty soon which would usually post at General Questions.

edit: link repaired

johnpoz

Good post by jahonix there..

I concur you really only need 1 switch here. You might want more than 1 with 1 being for your poe devices. Make sure what unifi you have supports cisco poe. Some of their models use a unifi brand of poe ;) Keep mind how many poe devices are you going to have? If just a handful use of just power injectors works just fine vs having to spend extra on poe switch.

He makes a great point about the 350, while they are out - the price point is still quite high on them.. I saw 350-10 over at amazon for 400. While the sg-300 was down to 130 at one point, new I should of picked up an extra then. I got mine for 180 while back. But seems they are over 200 again. Stock prob running low as 350 start to take over, and prob not making any more of the 300's

One point I want to bring up in your post "for now link aggregation"

Why exactly are you wanting this? I find that many people, even in the field don't quite get actual real world use for aggregation. In a small home it most likely would never make sense to waste the ports. Unless your really really worried about failover on failed port?? 1+1 does not = 2 with link aggregation. So if your thinking your going to get 2gig vs 1 gig by aggregating some ports.. Unless you have lots of sessions to different places your not going to split your connections across those links.. Your file copy speed from 1 server to your clients, etc.. is not going to see 2 gig.. Its only ever going to go over 1 of the links. Even if you use smb3 with multichannel the way the switches determine what traffic goes over what link, same IP talking to same IP, same mac talking to same mac all those sessions would most likely only be over the 1 link.. So your going to be seeing no more than the 1 gig, etc. If you want more than 1 gig then need to go fiber or 10ge..

Also I don't see why you would want/need layer 3 switch. While great that the sg300 and 350 lines can be layer 3 for future use/learning. To be honest your using pfsense right - it routes and firewalls in much easier fashion than the ACLs and ACE's you could setup on your layer 3 switch. Now you have a downstream router and pfsense - this just makes for a complicated network that prob has no use in the home setup expect for learning.

Unless your pfsense box was unable to route traffic at the speed you need, ie say it can't do gig between your segments. And you really need to move files at gig speed between your segments then sure you might want to route those at the layer 3 switch vs pfsense, etc.

mifronte

Thanks jahonix & johnpoz.

Here are the main reasons why I am looking to add a Layer 3 switch wit PoE+:

1. Right now I am unable to add wired devices to my untrusted VLAN since it is WiFi only.
2. To learn more about networking.
3. Avoid having to maintain static routes with Layer 2 switch.
4. Maintain gigabit speed between all segments, including WAN (need to test with my A1SRi-2758F pfSense box).
5. Ability to add PoE devices like cameras.

Regarding link aggregation, I have a 40 TB all purpose server (i.e. windows vm for dvr, unifi controller docker, SMB file/media server) with dual NIC. I was hoping to utilize both NIC to better serve multiple simultaneous streaming and copying. I had planned on using LACP. I assumed with a Layer 3 switch, I will no longer be using my Opt1 interface on my pfSense. So I was planning to setup all the interface on my pfSense (LAN, Opt1, Opt2) as a LAG to provide more trunks to the Internet for all the devices (I have gigabit WAN).

Now that I know I only need one switch, I am researching how to implement such a switch in my current setup.

Derelict

This is a diagram of doing it with one interface. I prefer to have WAN and LAN on different interfaces. Hopefully it helps.

VLAN-pfSense.png_thumb

johnpoz

"Avoid having to maintain static routes with Layer 2 switch."

huh? That makes no sense at all.. There are not routes on a layer 2 switch.

"1. Right now I am unable to add wired devices to my untrusted VLAN since it is WiFi only."

Again huh - what does that have to do with layer 3 switch?

"Maintain gigabit speed between all segments"

That really has nothing to do with layer 3 switch either. If your pfsense box can route at gig speeds then there you go gig speeds. Only if your pfsense is not capable of routing at those speeds would moving to a layer 3 switch that can. Better option would be to update pfsense box.

" (I have gigabit WAN)."

So then what is the point of having more than 1 gig into psense if it can only go out at 1 gig.. I don't see what lag buys you or gets you. How many clients do you have that will be making connections that would go through pfsense in your downstream switch setup only traffic going to pfsense would be internet, your internet is only 1 gig. So even if lag was 1+2=2 what is the point of having 2 inbound to pfsense if its outbound is only 1 gig..

mifronte

Please keep in mind anything beyond an unmanaged switch is new to me and one of the main reason I would like to use a managed switch is to learn and experiment more about networking. I will be experimenting in a home environment, but I would be practicing for larger environments.

@derelict
Thank you! A picture is truly worth 1000 words. I plan on WAN and LAN being separate interface like I have it now.

@johnpoz

I apologize for not making sense. Please excuse my ignorance since I am still trying to understand and learn more about networking.

Layer 3 Switch:
I am not trying to justify that I need a Layer3 switch. I believe I need a managed switch for the reasons I gave because I can't get it done with my unmanaged switch. Since I like to experiment and learn more about networking, I plan on getting a Layer 3 switch and need help on figuring out how to correctly implement it with pfSense.

I could get a Layer 2 switch or even a smart switch if all I cared about was just to fulfill my current needs. However, I figured if I have to purchase a managed switch, why not just purchase a Layer 3 switch to learn about Layer 2 and Layer 3 switches in networking. For LAG. I don't need it, but if all the hardware supports it, what's the harm experimenting with it to gain some knowledge? The same goes for pfSense, I don't need pfSense, but using pfSense has taught me a lot about networking that I would not be able to learn using regular consumer routers. It is one thing to read about networking in textbooks, but hands on practice puts all the pieces together.

Static Routes:
I tried setting up a VLAN using a dd-wrt AP, but without running NAT on the dd-wrt router, my pfSense did not know about the subnets on the dd-wrt router. It only knew about the primary untagged subnet. I tried and research if I can get rid of the second NAT layer and concluded that I may need static routes if I wanted the pfSense to know how to route to the subnets defined on the dd-wrt AP. I just wanted to avoid this situation and may correctly or incorrectly deduced I needed a Layer 3 switch. Anyway, it sounds like I may not, but who cares, I want a Layer 3 switch over a Layer 2 switch or a smart (whatever that means) switch.

The big question I have in my mind is what is the best practice to implement a Layer 3 switch behind a pfSense router. For example, do I plug both my LAN and Opt1 interface into the managed switch? The more I research and get great feedback, the more I realized how some of my previous comments don't make too much sense. Again I apologize for not being sensible.

Derelict

Should not be using your wireless device as a layer 3 device but a layer 2 bridge. There are countless threads on that very subject and this:

https://doc.pfsense.org/index.php/Use_an_existing_wireless_router_with_pfSense

Your blue plastic router was probably doing NAT so routing its private LAN subnet to it will do you no good.

http://www.tcpipguide.com/free/index.htm

https://www.amazon.com/s/ref=nb_sb_noss_2?url=search-alias%3Daps&field-keywords=tcp%2Fip+networking

Comer's Internetworking with TCP/IP is generally considered definitive and has been for decades.

mifronte

Thanks Derelict for the references.

Regarding my dd-wrt wireless device:
It was being used as an access point (layer 2 bridge) until I decided I needed a separate wireless network for untrusted devices. To accomplish this on the dd-wrt wireless device, I had to create a virtual wireless interface in its own subnet. Now I was stuck with the problem of how to give devices on the virtual wireless interface access to the Internet. dd-wrt's solution is to enable NAT for all traffic coming out of the access point, which worked, but I did not like having a second NAT layer on my network. It is at this point that johnpoz recommended UniFi AP with VLAN tagging.

Now I have a need to also add wired devices to the untrusted VLAN so that lead me to this path of needing a managed switch.

johnpoz

Yes you need a switch that supports vlan tags. It does not need to support the full enterprise feature set of say a nexus 7k from cisco ;) Ie fully managed.. It does not need to support layer 3 routing.. This is going to be easier for you to do at pfsense for sure. You could pick up a $40 "smart" switch that would allow you vlan with tags..

Now I am a huge fan of the cisco sg300 and its replacement the 350 is prob even better? These are normally very good price point. They have a very rich feature set, and allow for both cli and have a nice gui for the new user, etc. You can use them in layer 3 mode if you so desire.

What I would suggest is you set it up layer 2 to start with.. Once you have everything working the way you want - if you want to play/learn about downstream routers in your network and the use of transit networks, etc. etc. Then sure by all means change it to layer 3 and have fun.

But that sort of setup is way more complex than you need in some home setup that is for sure. Now if you want to firewall between your networks your going to have to do it at your layer 3 switch(router) and then all your other firewall rules going to the internet are going to have to be done at pfsense. While depending on size and setup of your network sure it makes sense to use layer 3 switches.. But for a couple of vlans it is just so much easier to let pfsense do your firewalling and routing at the edge..

So I have been in the field for many years. I do mostly routing and switching for my job.. And don't run layer 3 switch in my some would say way over complicated home network ;) I have multiple vlans and multiple ssids on different vlans.. And I don't run my sg300 in layer 3 because it makes no sense to do so.. It would just make my network harder to manage. Since now I would have to do all the ACL's and ACE's at that devices vs just easy to use pfsense..

jahonix

@johnpoz:

… set it up layer 2 to start with ... change it to layer 3 and have fun.

@mifronte: Just a reminder that the Cisco SG-300 switches need a completely new config when the fun begins. After switching to L3 mode the old L2 config is gone. You have been warned. ;-)

I completely agree with what Derelict and johnpoz posted so far.

jahonix

@mifronte:

Please keep in mind anything beyond an unmanaged switch is new to me and one of the main reason I would like to use a managed switch is to learn and experiment more about networking.

Well, the part of a managed switch you'll be using probably comes down to VLAN separations with tagged and untagged ports and a trunk.

It's really not that hard, just think of multiple separate switches within one case where ports and dependencies can be freely configured.
A trunk-port just squeezes the traffic of multiple separate devices through a single port, still keeping traffic separated. The port's speed is what limits the combined traffic.

Since better AccessPoints can be fed a trunk as well you can have VLAN A for traffic bridged to LAN, VLAN B for separated/guest WiFi, VLAN C for IoT devices, … etc. That can't be done with separate switches.

mifronte

Thank you for the great advice!

The more I read up on VLAN and refresh my readings on TCP/IP the more I realized that I will just start using a managed switch as a L2 device and let pfSense handle all the routing. I too am leaning toward the Cisco SMB SG300/350 line and operate it in L2 mode until a need arises for converting it to L3 mode.

Looks like it is not a good time to be looking for a Cisco SMB SG300/350 switch since the SG300 stock level is low and hence prices are going up. The SG350 are not widely available yet and so their prices are high too.

I do have a network design question and I hope I am phrasing it correctly:

Let's say I assign a port on a managed switch to a VLAN, but the port will be an untagged member. Can I attach an unmanaged switch to that port so that all devices connected to the unmanaged switch will be part of the port's assigned VLAN?

If I can reuse my unmanaged switch, I may be able to get away with just the 10-port managed switch.

jahonix

@mifronte:

Let's say I assign a port on a managed switch to a VLAN, but the port will be an untagged member.

An untagged switch-port is pretty much exactly what an unmanaged switch is on all ports. Except you cannot assign internal belongings.
(with limitations [CDP et al], I know, but that's not relevant here.)
Yes, that would work. But it's not exactly a straight forward design.

@mifronte:

If I can reuse my unmanaged switch, I may be able to get away with just the 10-port managed switch.

That'll hardly work. Been there, done that.
You always need one more port. If not now then tomorrow or next month.
Your money is spent once you buy gear. You cannot reuse it to buy bigger gear (except for selling the old one on eBay and such).
Have a look at the SG300-20. No PoE but no fan either. Might fit your project well.

dennypage

If you search for SG 300 advice, you will find many experienced networking folk say things like "The first thing I always do with an SG 300 is put them in layer 3 mode." Like many others, I didn't listen the first time… a painful lesson. :)

@jahonix:

Just a reminder that the Cisco SG-300 switches need a completely new config when the fun begins. After switching to L3 mode the old L2 config is gone. You have been warned. ;-)

johnpoz

So what if the config is gone? He is moving from layer 2 to 3 so why would his previous config be relevant at all?

He can save his config, move to l3 - do the play he wants, and then move back and reload his config. No harm no foul.. Few minutes of copy of a config.

Why would you put the device in layer 3 mode if your not going to be using layer 3?? That seems like just horrible advice from someone that has no clue to be honest..

Yeah the price point on the older 300-10 is spiked currently, you might find better pricing on the 20 for sure. Need to wait a few months to let magic of supply and demand work its magic ;) When the price comes down going to get a 350 and move my 300 into my av cab.. That shitty little $40 smart I have in their while it works, I just miss being able to monitor it via snmp and all the other bells and whistles that come with the sg3xx series..

dennypage

It's good advice from people with experience. I've done a few of these now, and I agree with them. There really isn't any benefit to leaving the SG 300 in mode 2. As to why it's painful, you loose basic configuration that has nothing to do with mode 2/3 like certificates, users, logging, channel groups, port identification, etc. These are things you really don't want to loose, and really shouldn't loose, in a mode switch. And no, you can't reload the prior config. You have to hand edit a new config, which requires a good knowledge of what has to be changed between the modes. And if you have certificates in the config? Ouch.

johnpoz

If you have someone that is unsure of use of the switch, the differences between layer 3 and layer 2.. I don't agree with putting it in layer 3 mode and only using it as layer 2. You would be better off leaving it layer 2 only if that is what your learning/playing with.

The move to layer 3 if that is what you want to do and the having to put back all your common info like users and certs or ssh keys, etc. Would all be good practice for the new user ;)

I don't see how home/lab use of this device be layer 2 or layer 3 moving back and forth should be that big of a deal. If your using it in a actual production setup with a complex setup you should know how your going to use it out of the gate anyway ;)

Derelict

The problem is all your layer2 config. VLANs, tagged/untagged ports are blown out. You should be able to restore a layer 2 config to a layer 3 switch. Layer 3 to layer 2 probably not so much, but I would expect Cisco to disregard the config it doesn't understand and honor what it does. That's what you pay for.

I set my recently-acquired SG300 to L3 out-of-the-box for just this reason.

johnpoz

So what if they are blown out, just put them back.. You can paste in the part of the config that is pertinent.. If your trying to restore from the gui you might have some issues? But you can always copy and paste the good stuff from your config via the cli without issue.

You put it in layer 3 mode out of the box for what reason - your using it as layer 3?? Do you see a need in the future to use it as layer 3?