Do I need 1 or 2 Smart Switches?



  • First, I hope I am posting in the correct section.  I chose this section because this pertains to altering my WiFi network to support wired devices.  Please move to a more appropriate section if needed.

    Do I need 1 or 2 smart switches given my current configuration and my desired goals?

    Current Configuration:
    pfSense ---LAN Interface (subnet 1)--->Unmanaged Switch--->Trusted network
        |
        |
        |---Opt1 Interface -------->UniFi AP---->Untag (subnet 2)---> Trusted WiFi
                                          |
                                          |
                                          |-------->Tag VLAN 20 (subnet 3) --->Untrusted WiFi
    
    

    Goals:
    1. Use PoE+ to support more PoE devices on any part of my network.
    2. I would like to add wired devices to Untrusted WiFi.  Right now I can't since the UniFi AP is connected to pfSense directly.
    3. I would like to replace the unmanaged switch with a managed switch, for now link aggregation.  Future, more segmentation of LAN.

    Questions:
    A. Can one layer 3 switch with PoE+ handle everything or do I need 2 switches, one for LAN and one for Opt1?

    B. Ideally 16-24 gigabit ports total with maybe 5-8 PoE+ ports.  Fan less is preferred to avoid rewiring, but not absolutely required.  I was thinking of the Cisco SG-300 line of routers.  Are there better alternatives?

    C. More follow-up questions once I identify a switch or switches to use.

    Thanks.



  • @mifronte:

    A. Can one layer 3 switch with PoE+ handle everything or do I need 2 switches, one for LAN and one for Opt1?

    Layer2 vs L3 is a different beast. It's management capabilitie is what counts here.
    You can get this working with a single managed switch. Probably even easier than 2 devices, once you understand VLANs. Keeps it simple.

    @mifronte:

    B. Ideally 16-24 gigabit ports total with maybe 5-8 PoE+ ports.  Fan less is preferred to avoid rewiring, but not absolutely required.  I was thinking of the Cisco SG-300 line of routers.  Are there better alternatives?

    There is no "better" or "worse", only "different" in many aspects.
    Cisco's SG-300 line is great with limitations (and will be superseded by SG-350 in the foreseeable future).
    SG-300 PoE switches with >21 ports will have a fan, the SG300-10 models (P/PP/MP) have a hated power brick PSU, not an internal one with IEC connector. But therefore no fan…

    So much for the switch side of your post.
    Your thread will be about VLANs pretty soon which would usually post at General Questions.

    edit: link repaired


  • LAYER 8 Global Moderator

    Good post by jahonix there..

    I concur you really only need 1 switch here. You might want more than 1 with 1 being for your poe devices.  Make sure what unifi you have supports cisco poe.  Some of their models use a unifi brand of poe ;)  Keep mind how many poe devices are you going to have?  If just a handful use of just power injectors works just fine vs having to spend extra on poe switch.

    He makes a great point about the 350, while they are out - the price point is still quite high on them.. I saw 350-10 over at amazon for 400.  While the sg-300 was down to 130 at one point, new I should of picked up an extra then.  I got mine for 180 while back.  But seems they are over 200 again.  Stock prob running low as 350 start to take over, and prob not making any more of the 300's

    One point I want to bring up in your post "for now link aggregation"

    Why exactly are you wanting this?  I find that many people, even in the field don't quite get actual real world use for aggregation.  In a small home it most likely would never make sense to waste the ports.  Unless your really really worried about failover on failed port??  1+1 does not = 2 with link aggregation.  So if your thinking your going to get 2gig vs 1 gig by aggregating some ports.. Unless you have lots of sessions to different places your not going to split your connections across those links..  Your file copy speed from 1 server to your clients, etc.. is not going to see 2 gig..  Its only ever going to go over 1 of the links.  Even if you use smb3 with multichannel the way the switches determine what traffic goes over what link, same IP talking to same IP, same mac talking to same mac all those sessions would most likely only be over the 1 link.. So your going to be seeing no more than the 1 gig, etc.  If you want more than 1 gig then need to go fiber or 10ge..

    Also I don't see why you would want/need layer 3 switch.  While great that the sg300 and 350 lines can be layer 3 for future use/learning.  To be honest your using pfsense right - it routes and firewalls in much easier fashion than the ACLs and ACE's you could setup on your layer 3 switch.  Now you have a downstream router and pfsense - this just makes for a complicated network that prob has no use in the home setup expect for learning.

    Unless your pfsense box was unable to route traffic at the speed you need, ie say it can't do gig between your segments.  And you really need to move files at gig speed between your segments then sure you might want to route those at the layer 3 switch vs pfsense, etc.



  • Thanks jahonix & johnpoz.

    Here are the main reasons why I am looking to add a Layer 3 switch wit PoE+:

    1. Right now I am unable to add wired devices to my untrusted VLAN since it is WiFi only.
    2. To learn more about networking.
    3. Avoid having to maintain static routes with Layer 2 switch.
    4. Maintain gigabit speed between all segments, including WAN (need to test with my A1SRi-2758F pfSense box).
    5. Ability to add PoE devices like cameras.

    Regarding link aggregation, I have a 40 TB all purpose server (i.e. windows vm for dvr, unifi controller docker, SMB file/media server) with dual NIC.  I was hoping to utilize both NIC to better serve multiple simultaneous streaming and copying.  I had planned on using LACP.  I assumed with a Layer 3 switch, I will no longer be using my Opt1 interface on my pfSense.  So I was planning to setup all the interface on my pfSense (LAN, Opt1, Opt2) as a LAG to provide more trunks to the Internet for all the devices (I have gigabit WAN).

    Now that I know I only need one switch, I am researching how to implement such a switch in my current setup.


  • LAYER 8 Netgate

    This is a diagram of doing it with one interface. I prefer to have WAN and LAN on different interfaces. Hopefully it helps.



  • LAYER 8 Global Moderator

    "Avoid having to maintain static routes with Layer 2 switch."

    huh?  That makes no sense at all.. There are not routes on a layer 2 switch.

    "1. Right now I am unable to add wired devices to my untrusted VLAN since it is WiFi only."

    Again huh - what does that have to do with layer 3 switch?

    "Maintain gigabit speed between all segments"

    That really has nothing to do with layer 3 switch either.  If your pfsense box can route at gig speeds then there you go gig speeds.  Only if your pfsense is not capable of routing at those speeds would moving to a layer 3 switch that can.  Better option would be to update pfsense box.

    " (I have gigabit WAN)."

    So then what is the point of having more than 1 gig into psense if it can only go out at 1 gig.. I don't see what lag buys you or gets you.  How many clients do you have that will be making connections that would go through pfsense in your downstream switch setup only traffic going to pfsense would be internet, your internet is only 1 gig.  So even if lag was 1+2=2 what is the point of having 2 inbound to pfsense if its outbound is only 1 gig..



  • Please keep in mind anything beyond an unmanaged switch is new to me and one of the main reason I would like to use a managed switch is to learn and experiment more about networking.  I will be experimenting in a home environment, but I would be practicing for larger environments.

    @derelict
    Thank you! A picture is truly worth 1000 words.  I plan on WAN and LAN being separate interface like I have it now.

    @johnpoz

    I apologize for not making sense.  Please excuse my ignorance since I am still trying to understand and learn more about networking.

    Layer 3 Switch:
    I am not trying to justify that I need a Layer3 switch.  I believe I need a managed switch for the reasons I gave because I can't get it done with my unmanaged switch.  Since I like to experiment and learn more about networking, I plan on getting a Layer 3 switch and need help on figuring out how to correctly implement it with pfSense.

    I could get a Layer 2 switch or even a smart switch if all I cared about was just to fulfill my current needs.  However, I figured if I have to purchase a managed switch, why not just purchase a Layer 3 switch to learn about Layer 2 and Layer 3 switches in networking.  For LAG.  I don't need it, but if all the hardware supports it, what's the harm experimenting with it to gain some knowledge?  The same goes for pfSense, I don't need pfSense, but using pfSense has taught me a lot about networking that I would not be able to learn using regular consumer routers.  It is one thing to read about networking in textbooks, but hands on practice puts all the pieces together.

    Static Routes:
    I tried setting up a VLAN using a dd-wrt AP, but without running NAT on the dd-wrt router, my pfSense did not know about the subnets on the dd-wrt router.  It only knew about the primary untagged subnet.  I tried and research if I can get rid of the second NAT layer and concluded that I may need static routes if I wanted the pfSense to know how to route to the subnets defined on the dd-wrt AP.  I just wanted to avoid this situation and may correctly or incorrectly deduced I needed a Layer 3 switch.  Anyway, it sounds like I may not, but who cares, I want a Layer 3 switch over a Layer 2 switch or a smart (whatever that means) switch.

    The big question I have in my mind is what is the best practice to implement a Layer 3 switch behind a pfSense router.  For example, do I plug both my LAN and Opt1 interface into the managed switch?  The more I research and get great feedback, the more I realized how some of my previous comments don't make too much sense.  Again I apologize for not being sensible.


  • LAYER 8 Netgate

    Should not be using your wireless device as a layer 3 device but a layer 2 bridge. There are countless threads on that very subject and this:

    https://doc.pfsense.org/index.php/Use_an_existing_wireless_router_with_pfSense

    Your blue plastic router was probably doing NAT so routing its private LAN subnet to it will do you no good.

    http://www.tcpipguide.com/free/index.htm

    https://www.amazon.com/s/ref=nb_sb_noss_2?url=search-alias%3Daps&field-keywords=tcp%2Fip+networking

    Comer's Internetworking with TCP/IP is generally considered definitive and has been for decades.



  • Thanks Derelict for the references.

    Regarding my dd-wrt wireless device:
    It was being used as an access point (layer 2 bridge) until I decided I needed a separate wireless network for untrusted devices.  To accomplish this on the dd-wrt wireless device, I had to create a virtual wireless interface in its own subnet.  Now I was stuck with the problem of how to give devices on the virtual wireless interface access to the Internet.  dd-wrt's solution is to enable NAT for all traffic coming out of the access point, which worked, but I did not like having a second NAT layer on my network.  It is at this point that johnpoz recommended UniFi AP with VLAN tagging.

    Now I have a need to also add wired devices to the untrusted VLAN so that lead me to this path of needing a managed switch.


  • LAYER 8 Global Moderator

    Yes you need a switch that supports vlan tags.  It does not need to support the full enterprise feature set of say a nexus 7k from cisco ;)  Ie fully managed..  It does not need to support layer 3 routing.. This is going to be easier for you to do at pfsense for sure.  You could pick up a $40 "smart" switch that would allow you vlan with tags..

    Now I am a huge fan of the cisco sg300 and its replacement the 350 is prob even better?  These are normally very good price point.  They have a very rich feature set, and allow for both cli and have a nice gui for the new user, etc.  You can use them in layer 3 mode if you so desire.

    What I would suggest is you set it up layer 2 to start with.. Once you have everything working the way you want - if you want to play/learn about downstream routers in your network and the use of transit networks, etc. etc.  Then sure by all means change it to layer 3 and have fun.

    But that sort of setup is way more complex than you need in some home setup that is for sure.  Now if you want to firewall between your networks your going to have to do it at your layer 3 switch(router) and then all your other firewall rules going to the internet are going to have to be done at pfsense.  While depending on size and setup of your network sure it makes sense to use layer 3 switches..  But for a couple of vlans it is just so much easier to let pfsense do your firewalling and routing at the edge..

    So I have been in the field for many years.  I do mostly routing and switching for my job..  And don't run layer 3 switch in my some would say way over complicated home network ;)  I have multiple vlans and multiple ssids on different vlans..  And I don't run my sg300 in layer 3 because it makes no sense to do so..  It would just make my network harder to manage.  Since now I would have to do all the ACL's and ACE's at that devices vs just easy to use pfsense..



  • @johnpoz:

    … set it up layer 2 to start with ... change it to layer 3 and have fun.

    @mifronte: Just a reminder that the Cisco SG-300 switches need a completely new config when the fun begins. After switching to L3 mode the old L2 config is gone. You have been warned.  ;-)

    I completely agree with what Derelict and johnpoz posted so far.



  • @mifronte:

    Please keep in mind anything beyond an unmanaged switch is new to me and one of the main reason I would like to use a managed switch is to learn and experiment more about networking.

    Well, the part of a managed switch you'll be using probably comes down to VLAN separations with tagged and untagged ports and a trunk.

    It's really not that hard, just think of multiple separate switches within one case where ports and dependencies can be freely configured.
    A trunk-port just squeezes the traffic of multiple separate devices through a single port, still keeping traffic separated. The port's speed is what limits the combined traffic.

    Since better AccessPoints can be fed a trunk as well you can have VLAN A for traffic bridged to LAN, VLAN B for separated/guest WiFi, VLAN C for IoT devices, … etc. That can't be done with separate switches.



  • Thank you for the great advice!

    The more I read up on VLAN and refresh my readings on TCP/IP the more I realized that I will just start using a managed switch as a L2 device and let pfSense handle all the routing.  I too am leaning toward the Cisco SMB SG300/350 line and operate it in L2 mode until a need arises for converting it to L3 mode.

    Looks like it is not a good time to be looking for a Cisco SMB SG300/350 switch since the SG300 stock level is low and hence prices are going up.  The SG350 are not widely available yet and so their prices are high too.

    I do have a network design question and I hope I am phrasing it correctly:

    Let's say I assign a port on a managed switch to a VLAN, but the port will be an untagged member.  Can I attach an unmanaged switch to that port so that all devices connected to the unmanaged switch will be part of the port's assigned VLAN?

    If I can reuse my unmanaged switch, I may be able to get away with just the 10-port managed switch.



  • @mifronte:

    Let's say I assign a port on a managed switch to a VLAN, but the port will be an untagged member.

    An untagged switch-port is pretty much exactly what an unmanaged switch is on all ports. Except you cannot assign internal belongings.
    (with limitations [CDP et al], I know, but that's not relevant here.)
    Yes, that would work. But it's not exactly a straight forward design.

    @mifronte:

    If I can reuse my unmanaged switch, I may be able to get away with just the 10-port managed switch.

    That'll hardly work. Been there, done that.
    You always need one more port. If not now then tomorrow or next month.
    Your money is spent once you buy gear. You cannot reuse it to buy bigger gear (except for selling the old one on eBay and such).
    Have a look at the SG300-20. No PoE but no fan either. Might fit your project well.



  • If you search for SG 300 advice, you will find many experienced networking folk say things like "The first thing I always do with an SG 300 is put them in layer 3 mode." Like many others, I didn't listen the first time… a painful lesson. :)

    @jahonix:

    Just a reminder that the Cisco SG-300 switches need a completely new config when the fun begins. After switching to L3 mode the old L2 config is gone. You have been warned.  ;-)


  • LAYER 8 Global Moderator

    So what if the config is gone?  He is moving from layer 2 to 3 so why would his previous config be relevant at all?

    He can save his config, move to l3 - do the play he wants, and then move back and reload his config. No harm no foul.. Few minutes of copy of a config.

    Why would you put the device in layer 3 mode if your not going to be using layer 3??  That seems like just horrible advice from someone that has no clue to be honest..

    Yeah the price point on the older 300-10 is spiked currently, you might find better pricing on the 20 for sure.  Need to wait a few months to let magic of supply and demand work its magic ;)  When the price comes down going to get a 350 and move my 300 into my av cab..  That shitty little $40 smart I have in their while it works, I just miss being able to monitor it via snmp and all the other bells and whistles that come with the sg3xx series..



  • It's good advice from people with experience. I've done a few of these now, and I agree with them. There really isn't any benefit to leaving the SG 300 in mode 2. As to why it's painful, you loose basic configuration that has nothing to do with mode 2/3 like certificates, users, logging, channel groups, port identification, etc. These are things you really don't want to loose, and really shouldn't loose, in a mode switch. And no, you can't reload the prior config. You have to hand edit a new config, which requires a good knowledge of what has to be changed between the modes. And if you have certificates in the config? Ouch.


  • LAYER 8 Global Moderator

    If you have someone that is unsure of use of the switch, the differences between layer 3 and layer 2..  I don't agree with putting it in layer 3 mode and only using it as layer 2.  You would be better off leaving it layer 2 only if that is what your learning/playing with.

    The move to layer 3 if that is what you want to do and the having to put back all your common info like users and certs or ssh keys, etc.  Would all be good practice for the new user ;)

    I don't see how home/lab use of this device be layer 2 or layer 3 moving back and forth should be that big of a deal.  If your using it in a actual production setup with a complex setup you should know how your going to use it out of the gate anyway ;)


  • LAYER 8 Netgate

    The problem is all your layer2 config. VLANs, tagged/untagged ports are blown out. You should be able to restore a layer 2 config to a layer 3 switch. Layer 3 to layer 2 probably not so much, but I would expect Cisco to disregard the config it doesn't understand and honor what it does. That's what you pay for.

    I set my recently-acquired SG300 to L3 out-of-the-box for just this reason.


  • LAYER 8 Global Moderator

    So what if they are blown out, just put them back.. You can paste in the part of the config that is pertinent..  If your trying to restore from the gui you might have some issues?  But you can always copy and paste the good stuff from your config via the cli without issue.

    You put it in layer 3 mode out of the box for what reason - your using it as layer 3??  Do you see a need in the future to use it as layer 3?



  • You can't just paste the pertinent stuff back. It ends up being more complicated than that. And by the time you have the experience necessary to hand edit the config, you have learned enough to know that you want preserve your options and put the unit in mode 3 from the beginning. Even if you have no immediate need for layer 3 routing, there is no advantage to leaving it in mode 2.


  • LAYER 8 Global Moderator

    Other than less overhead of something your not using..

    I am fairly freaking sure I can past my port configs and what vlans they are in back in..  Not like the syntax of commands change for gosh sake..

    I half mind to switch it layer 3 just to prove my point ;)



  • In the for what it's worth category, I installed my first SG350 (replacement for the SG300) yesterday. They have completely done away with the system mode setting. :)



  • You wanna say it's always in L3 mode?

    BTW: Can you tell differences between 300 and 350 series? I find them extremely hard to find on Cisco's pages…  :(


  • LAYER 8 Global Moderator



  • The differences between SG300 and SG350.
    Like compare SG300-10 and SG300-20 features and technical data. I know those tables.

    Or any other document which describes what's new in 350 series.
    Basically I don't want to campare the complete feature sets myself, only the diffs.



  • @jahonix:

    You wanna say it's always in L3 mode?

    BTW: Can you tell differences between 300 and 350 series? I find them extremely hard to find on Cisco's pages…  :(

    I wasn't able to find any meaningful comparison of the two series on Cisco's site either. I'm guessing that is intentional.

    Yes, the main unit itself is always L3. The biggest change that I've noticed is that the 350 has true IPv6 support, and actually appears to route IPv6 at "wire speed". Other things of note include double the TCAM entries (includes IPv6), sFlow support, remote SPAN support for interface or vlan mirroring, L2/L3 on a per interface basis, policy based routing, and port flap monitoring. It also purports to be truly stackable.

    There is a new UI. Mostly good. The built in help effectively replaces the Admin Guide and is pretty good. Couple of annoying things: it seems to ignore the idle timer settings and logs you out every two minutes (I assume this is a bug); it has a basic/advanced display mode which would be fine except that basic mode hides almost all IPv6 settings. It also has SNA (Smart Network Application) which I do not have a good use for yet.

    One thing that I thought was rather nice is that the 10 port version can itself be powered by upstream POE ports with pass power through to downstream ports. It can also be used as a backup power to an AC adapter. Kinda sweet.

    The big disappointment (for me) is that gigabit port to port latency has not improved. Still 2450ns for idle, 3200ns under load for the 10 port unit. The 28 port should offer slightly better numbers (200-300ns), but I haven't tested it yet.


  • LAYER 8 Global Moderator

    "It also purports to be truly stackable."

    Huh where are you seeing that you can stack it?  The SG500 series is stackable.. Maybe your thinking the 350X ?



  • I'm aware that it's not in the data sheet John. It's in the UI help and in the configs. I don't have a second 350 unit to confirm or deny, hence the term "purports."


  • LAYER 8 Netgate

    I thought the stackable (max 4) part was the 350X.


  • LAYER 8 Global Moderator

    And does it have a stack port on the back of it?  How exactly are you suppose to stack it?

    Looking online for images of the sg350 and do not see any place that would allow to connect the stack..



  • Perhaps I wasn't clear enough. I only have the 10 port sg350 at this time. Stacking, if it exists on the 350, would be with the 28 or 48 port models, which I have not yet had my hands on. The admin guide directly implies that the 350 series is stackable, and stack information is kicked out in the saved configs, even for the 10 port. Of course, this may be an unintended consequence of Cisco choosing to use the same firmware in the sg350 and sg350x series. I honestly don't know. If the stackable distinction is important to you, I would recommend reaching out to Cisco. Btw, there is no separate physical stack port. According to the admin guide, the standard uplink ports are used for stacking.


  • Galactic Empire



  • Gentlemen, have a look yourself:
    Cisco Small Business Online Device Emulators

    IIRC, you can even get an SG300-10 in a "stacked mode" from CLI. Well…



  • BTW, 300-series switches are not even end-of-sale yet.



  • LAYER 8 Global Moderator

    dude is there a port to stack it with or not?

    As to end of sale.. Pretty sure this looks like end of sale for some of them..

    http://www.cisco.com/c/en/us/products/collateral/switches/small-business-stackable-managed-switches/eos-eol-notice-c51-733213.pdf

    But looks like its just model number of specific sg300's  There clearly could of been a misread there.. I thought it funny that I thought mine was end of sw support back in april but one came out in may, and then another just last month.. So that is clearly a good thing.

    If you can stack it there has to be a port to stack with.. So can you send picture of yours that has this port..




  • @johnpoz:

    dude is there a port to stack it with or not?

    Don't shoot me, I'm the messanger only…
    IIRC, SG500 uses SFP+ ports for stacking with up to 5Gbps, don't they? Is that considered dedicated?

    @johnpoz:

    As to end of sale.. Pretty sure this looks like end of sale for some of them..

    But looks like its just model number of specific sg300's

    While the headline reads
    End-of-Sale and End-of-Life Announcement for the Cisco Small Business Stackable Managed Switches
    it is obvious that it should be "some" not "the". Even in that list are lots of SG300 switches and Cisco itself calls them "Small Business Stackable" for whatever reason.

    BTW, johnpoz, did you find any documents or other references explainig the diffs between SG300 and SG350 switches?


  • LAYER 8 Global Moderator

    You are correct, upon looking deeper the 500 series uses specific sfp ports, or specific normal ports, etc..

    http://www.cisco.com/c/dam/en/us/td/docs/switches/lan/csbms/Sx500/quick_start/en/500_Series_QSG_En.pdf
    The default stack ports on the 500X are XG3/S1 and XG4/S2. If the correct module is plugged into XG3/S1 and XG4/S2, the switch should be able to detect the connection and configure the speed according to the module capability without any manual configuration. The 5G/S1 and 5G/S2 interfaces on the 500X need to be configured manually via the CLI or web-based interface in order to utilize these ports as stack ports

    But if you ask me this is not a real stack ;)  Isn't the stack bandwidth with like a 3750 and stackwise cables like 64gbps - this seems like nothing more than a fancy daisychain ;)  If your going to be limited to say 10ge or 5 or 1gpbs..


Log in to reply