One L3 per VLAN across 2+ interfaces
-
We just had a lengthy discussion about this two weeks or so ago.
Interfaces in pfSense are router ports, what you need are switch ports.You cannot and don't want to use the same VLANs on multiple interfaces in pfSense. Distribute VLANs with a core switch where it belongs.
Creating a software bridge is far from being two switch ports. -
I thought it sounded familiar... What @jahonix said..
You distribute your vlans via your switching environment.. Pfsense is not a "switch" If you need a switch and want it to sit in the same box as your pfsense.. Then get say the sg3100 that comes with 4 port switch built in. Or say the XG-7100 which as 8 port switch built in.
And adding a core or distribution layer switch DOES NOT add a "hop"
Looks like vlan 50 is pretty much isolated to rack 2 switch 2 Why not uplink that on its own to pfsense interface. Then run a connection for vlan 100 to your rack 1, and your other rack 3 switch could just connect into rack 1 switch.
If you want to leverage your router ports - then your vlans that have larger intervlan traffic should sit on those physical interfaces.. Or do a lacp setup from your 2 interfaces you have to your rack 1 switch for all your vlans other than 50.
-
@jahonix @johnpoz Thank you both for your input, from reading and testing I thought this might be the solution.
I'm inheriting this network, it's currently using no VLAN's and has 50+ devices over 10+ switches (mix of desktop and rack, although has the correct patch capabilities) in a single broadcast domain. The only thing I was going to keep was the pfSense firewall (re-configured its perfect for them), I believe its on netgate hardware but I need to verify what exactly.
I'm trying to keep the cost reasonable to not scare off the bean counters. Typically I'd have a firewall cluster and stacked switches, justifying it for this business is tricky - although I really hear what you're saying! Perhaps bazaar logic, we may have found the one redeeming quality for juniper's SRX appliances (I still hate working on them though) :P
You're completely right regarding the 'hop' - Its L2 only. What I really meant was that if that first switch or the firewall dies, so do others . Whereas if the firewall is trunked to each switch, only the Firewall failing would bring all of them down.
I'l give Rack Switch 2 an uplink from pfSense, and 'daisy chain' 1 & 3. There's some more complicated stuff going on with the servers (hyper-visors and container clusters) but the advice stands true.
I could have a use case for both XG-7100 and SG3100, out of curiosity what would the configuration steps look like to achieve what i'm looking for?
Thank you both for your help, I truly appreciate it.
-
@braveben said in One L3 per VLAN across 2+ interfaces:
we may have found the one redeeming quality for juniper’s SRX appliances
The fact that you can live without the functionality of a core switch? Come on, that's chicken shit.
@braveben said in One L3 per VLAN across 2+ interfaces:
out of curiosity what would the configuration steps look like to achieve what i’m looking for
Sorry, I can't follow you here. Can you re-phrase the question, please.
(I never had one of those units so my input might be limited. johnpoz IMHO has such a device around) -
@jahonix said in One L3 per VLAN across 2+ interfaces:
The fact that you can live without the functionality of a core switch? Come on, that’s chicken shit.
C'mon, where's your sense of humour? There's no way to justify a 4th 'enterprise' switch just to distribute to 3 other switches in this setup. There are use-cases for assigning Virtual L3 interfaces to a VLAN for tagging across interfaces, especially in smaller networks.
Justifying to finance two boxes for the same or similar cost to 1 box that achieves the same is harder than it may sound, no matter how much of a PITA that second blue box is to configure.
@jahonix said in One L3 per VLAN across 2+ interfaces:
Sorry, I can’t follow you here. Can you re-phrase the question, please.
I'm curious on what configuration steps would be recommended to achieve the setup in my topology, using the devices mentioned:
@johnpoz said in One L3 per VLAN across 2+ interfaces:
If you need a switch and want it to sit in the same box as your pfsense… Then get say the sg3100 that comes with 4 port switch built in. Or say the XG-7100 which as 8 port switch built in.
Essentially then, should redundancy become an issue: "Buy 3x switches + firewall" is a much easier sell than "buy 4x switches + firewall".
The following still stands true, your conformation that this should at least be the interim solution is greatly appreciated.
@braveben said in One L3 per VLAN across 2+ interfaces:
It may be easier to re-spec “Rack Switch 1” to handle the minimal patching required and distribution - only
it introduces another hop &its demise would have greater impactThanks again
-
@braveben said in One L3 per VLAN across 2+ interfaces:
There’s no way to justify a 4th ‘enterprise’ switch just to distribute to 3 other switches in this setup.
If the investment for a $40 or so smart managed switch is too much then ... you can still get an SG-3100 or an XG-7100 which have managed switches built-in. No additional box required...
-
I wouldn't say they have managed switches.. While there is a 4 port switch in the sg-3100 - and you can put an SVI on and vlan tags, etc. Its feature set doesn't compare to a full blown managed switch.. even a small business line like the cisco sg300.
As to the configuration steps... You create your vlans on the switch and plug in your downstream switches
If your all worried about failure - your racks still could fail.. I don't see how using one as your core and 1 switch downstream of it or even 2 is a problem.
What switches are you using now, what is the port density.. 24 ports, 48? Any fiber connections? 1 gig, 10gig?
if your going to redo it all - what is the budget?
-
@jahonix said in One L3 per VLAN across 2+ interfaces:
If the investment for a $40 or so smart managed switch is too much then ... you can still get an SG-3100 or an XG-7100 which have managed switches built-in. No additional box required...
This is half of it, i'm trying to get them away from from just ramming another switch wherever. The other half is trying to keep consistent manageable (supportable) hardware, I don't think Cisco/Brocade do a CLI only L2 1Gbit managed switch for <$100 ;) If the netgate's can do the switchport magic - Id rather get new firewall hardware with 'switch' capabilities and access switches than just new switches.
@johnpoz I'm not sure if this weighs in at all - I'm not looking to do Inter-vlan switching before L3 routing, In fact the aim is for that only to occur should Firewall/routing allow for it. Pseudo diagram hopefully makes sense, its the simplest use case I can think of:
@johnpoz said in One L3 per VLAN across 2+ interfaces:
and you can put an SVI on and vlan tags, etc
I can't find any reference to this - Is there an added CLI/GUI tool specifically for 'switchport' netgate devices?
@johnpoz said in One L3 per VLAN across 2+ interfaces:
What switches are you using now,
Currently Its a mess of un-managed Dlink, TPlink, HP and god knows what else. There's a 24 port switch sat between two 5 port desktop switches..... this is a much simplified example of whats going on:
I hope you're not suggesting I don't burn this to the ground ;P
- Fix L2
- Fix L3
- Firewall everything.
- Add much desired Domain Controller/Servers
@johnpoz said in One L3 per VLAN across 2+ interfaces:
what is the port density… 24 ports, 48? Any fiber connections? 1 gig, 10gig?
Currently its looking like:
1x 48 Gbit Port (~300w POE)+ 2x 1GB SFP
1x 16 Gbit Port (~300w POE)+ 2x 1GB SFP
1x 16 Gbit Port + 2x 1GB SFPIts all copper bar the 1 fibre up-link between building 1 & 2 - SFP's are most welcome in this case (10Gbit is un-necessary).
Most likely to be the L2 Cisco 2950 Series. With the lifetime warranty/next day replacement we can get thrown in outweighs the value of the SG300/500 & SF350 - although i''m happy to be proven wrong on this.
Most of the heavy-bandwidth actions will be contained within the servers, Mostly PXE booting Virtual Nodes for test environments.@johnpoz said in One L3 per VLAN across 2+ interfaces:
if your going to redo it all - what is the budget?
Build a strong business case. They've been burned and are uneasy without clear design and business value/impact. Enterprises with 'growing pains' often don't understand (without some help) their work network cannot be like their home network anymore.
I can make a very strong business case with being able to say 'Buy one or the other of this firewall + X amount of (Cisco 2950 with N ports + N Poe)' in these medium/small size networks (Until redundancy or X amount of switches requirements change). It also makes it super easy to 'mimic' the environment at scale for satellite offices.
This may sound a little over-enthusiastic, but this really level's the playing field for pfSense/Netgate against some of the more 'established enterprise' products. "Can pfSense do everything this blue box can do?" or "pfSense isn't really an enterprise ready appliance!" is getting easier to overcome.
Buying say a Super-Micro server with Switch ports and installing pfSense adds another layer of logistics/costs organisations don't want to deal with. Sure, this may seem like a bit of an edge-case; it's getting more common as security is being taken more seriously and becoming more accessible.Appreciate you both
-
@braveben said in One L3 per VLAN across 2+ interfaces:
Most likely to be the L2 Cisco 2950 Series
You mean the series that has been EOL for years??
https://www.cisco.com/c/en/us/obsolete/switches/cisco-catalyst-2950-series-switches.html
Yeah I would burn that old shit down to the ground.. But not sure why you think you would have a hard time selling anyone the need for proper L2 switching environment and then L3 routing/firewalling..
Yes you can manage the switch on the sg-3100 and I would assume the newer xg-7100 but since I do not have that model I can not speak from experience on that one. Via cli {etherswitchcfg} and or gui.. I have not done much with it - but it does not have all the full featured fully managed features that the sg300 or 350 or 500 have.
https://www.freebsd.org/cgi/man.cgi?query=etherswitchcfg&sektion=8&apropos=0&manpath=FreeBSD%2B10.0-RELEASE
I would prob look at the SG500 which you can stack, if not for growth... What specific switches are you looking at - it can not be the 2950 line?? Also the smb line can do layer 3 if need be.. Again allows for growth from the company side even if explosive, etc.
While you can get a router/firewall with switch ports - they are never going to be as full featured as a real switch.. Your talking nickels and dimes to save what 2u in your rack?
-
@johnpoz said in One L3 per VLAN across 2+ interfaces:
I would prob look at the SG500 which you can stack,
I don't want to be that guy, but both the SG300 and SG500 series of switches are EOL :
SG300 - https://www.cisco.com/c/en/us/products/collateral/switches/small-business-300-series-managed-switches/eos-eol-notice-c51-740542.html
SG500 - https://www.cisco.com/c/en/us/products/switches/small-business-500-series-stackable-managed-switches/index.htmlThe replacements would be:
350x - https://www.cisco.com/c/en/us/products/switches/350x-series-stackable-managed-switches/index.html
550x - https://www.cisco.com/c/en/us/products/switches/550x-series-stackable-managed-switches/index.html@johnpoz said in One L3 per VLAN across 2+ interfaces:
You mean the series that has been EOL for years??
My mistake, correct answer:
(Probable) 2960-L range : https://www.cisco.com/c/en/us/products/switches/catalyst-2960-l-series-switches/index.html
(Likely un-necissary)2960-x/r for all the POE/L3: https://www.cisco.com/c/en/us/products/collateral/switches/catalyst-2960-x-series-switches/datasheet_c78-728232.htmlAnyways, the exact switches don't really matter here:
'Enterprise' package / CLI based
Some POE
16/24/42 Gbit port options
L2 only
1Gbit SFP Uplinks are nice to haveIf brocade weren't comparably priced to the Cisco kit i'd have no issue persuading them used instead. I will re-evaluate the 550x series though.
EDIT:
The SG350X-48P would probably be perfect as the 'largest' switch - although it not having full IOS may be a draw back... If they spring for 2960-L's we'd be living the dream.@johnpoz said in One L3 per VLAN across 2+ interfaces:
Yes you can manage the switch on the sg-3100 and I would assume the newer xg-7100 but since I do not have that model I can not speak from experience on that one. Via cli {etherswitchcfg} and or gui… I have not done much with it - but it does not have all the full featured fully managed features that the sg300 or 350 or 500 have.
Honestly, If that's the case i'l look into the XG-7100 for onsite, and the SG-3100 for remote offices. I'd much rather not build a case between a blue/red box VS pfSense + Distribution switch for these environment(s). Especially if i'm only loosing features that I wouldn't use anyway.
Could you confirm that the diagram below would work with the XG-7100/SG-3100 devices, or would I be best contacting pre-sales/getting one to play with? This is the 'refined' question that sparked this thread.
@braveben said in One L3 per VLAN across 2+ interfaces:
-
yeah with the switch models you can do that where with your vlan 10 and vlan 20 sure.
BTW the SG500 is still shipping until July 12, 2018 ;) So tech another month heheheeh
-
@johnpoz said in One L3 per VLAN across 2+ interfaces:
yeah with the switch models you can do that where with your vlan 10 and vlan 20 sure.
Awesome! Consider this solved/done/resolved/completed. Thank you for all your help and feedback
BTW the SG500 is still shipping until July 12, 2018 ;) So tech another month heheheeh
-
Everybody uses a core/distribution switch for this. Why are you insisting on being different?
The absolute cheapest managed switch (Think D-Link DGS-1100 as the starting point - $30) will do this job better than a pfSense software bridge.
An SG-3100 or SG-7100 would be a far better choice for this but I still don't see a reason why you would want to go that route (pun intended) instead of just getting another switch.
Yes, the switch chip in either the SG-3100 or the XG-7100 could do that. Depending on bandwidth, I would probably suggest the 7100 due to your attempting IDS/IPS and, especially, anti-virus. Sales would know more.
-
@derelict said in One L3 per VLAN across 2+ interfaces:
Everybody uses a core/distribution switch for this. Why are you insisting on being different?
Respectfully, no not everybody has an environment where dedicated distribution is needed. I don't have Lots of (access) switches, redundancy requirement or high speed intervlan-switching (before firewalling). And @johnpoz extrapolated, really its only 2 switches that will share VLAN Trunks - why shouldn't this be distributed from the Firewall if all traffic should hit it first? I understand this is partially a hardware issue, I'm not trying to say 'well if this box can do it, why can't you' just that there are use-cases, and other vendor's facilitate this:
This is for their switches, but I've done this on their firewalls:
https://kb.juniper.net/InfoCenter/index?page=content&id=KB10972&cat=SWITCH_PRODUCTS&actp=LISTWatchguard can also do it...
https://www.watchguard.com/help/docs/fireware/12/en-US/Content/en-US/networksetup/vlan_example_1vlan_2switches_c.html@derelict said in One L3 per VLAN across 2+ interfaces:
The absolute cheapest managed switch (Think D-Link DGS-1100 as the starting point - $30) will do this job better than a pfSense software bridge.
I get what you're saying. With an SG-3100/SG-7100 using switch ports I wouldn't be creating a software bridge though, right? I'd be able to switch the tagged packets to an L3 interface?
This isn't my home network, distributing a network between a ~$1k Firewall and a ~$1K enterprise switch with a $30eur D-Link isn't gonna cut it ;P not to mention dem 10Gbit SFP uplinks....Speaking of those sexy XG-7100's 10Gbit SFPs, how would they respond to being a trunk with the same VLAN's as one of the switch ports? Could I still share a L3 interface/IP on the single VLAN?
@derelict said in One L3 per VLAN across 2+ interfaces:
Depending on bandwidth, I would probably suggest the 7100 due to your attempting IDS/IPS and, especially, anti-virus.
The 7100 would be a perfect fit, I spent some time with a UTM/NGF vendor in support - without any [Inspect any <> any] rules, it should be pretty future proof for us.
Thank you for all your input
-
@braveben said in One L3 per VLAN across 2+ interfaces:
This is for their switches, but I’ve done this on their firewalls:
https://kb.juniper.net/InfoCenter/index?page=content&id=KB10972&cat=SWITCH_PRODUCTS&actp=LIST
Watchguard can also do it…
https://www.watchguard.com/help/docs/fireware/12/en-US/Content/en-US/networksetup/vlan_example_1vlan_2switches_c.htmlwhat everyone has been trying to tell you isn't that you 'CANT' do it ..... you just SHOULDNT.
but honestly, you do what you wanna do .... nobody will lose sleep over it either way.
-
@braveben said in One L3 per VLAN across 2+ interfaces:
Speaking of those sexy XG-7100’s 10Gbit SFPs, how would they respond to being a trunk with the same VLAN’s as one of the switch ports? Could I still share a L3 interface/IP on the single VLAN?
You would, again, have to software bridge them. I would suggest using the 10G to an external switch instead.
