Converting from a Single Interface to a LAGG
-
@guardian said in Converting from a Single Interface to a LAGG:
Any comments/hints/suggestions are most appreciated.
Understand the tech your wanting use - make sure it works the way you think it does. And understand your network flow to how to best optimize it while allowing for the filtering you need/want through your firewall.
I see it all the time with hairpins and bad uplinking.. in line with Derelict's Two is one, lets also remember that 1 + 1 does not equal 2, it is just 1 and 1 ;)
You do not instantly get 2 gig when you lagg/etherchannel/portchannel/lacp whatever term you want to use two gig connections. Just means you have 2 1 gig connections.
-
@johnpoz said in Converting from a Single Interface to a LAGG:
Understand the tech your wanting use - make sure it works the way you think it does.
This is much easier said than done because of the way this stuff is taught. I've cherry picked though a few of online CCNA/Networking courses and they concentrate far too much on how to set something up as opposed to "why" set it up a particular way and don't teach enough first principles.
The courses tend to have way too much material that is too basic (this is binary, this is a subnet), and then often very quickly go way too complicated (multi-building with several layers of distribution), and concentrate on details on the negotiation (active/passive-which modes are compatible etc.) and the command line commands to setup LACP in all the different modes.
Details about how the traffic gets divided, when hairpinning would be an issue, and a lot of the "why" set things up a given way is skipped over.
[Possible suggestion for whoever creates the content for the Gold Membership:
----BEGIN SUGGESTION
Please consider this a feedback/a suggestion for possible improvement, not a complaint-I appreciate your subscribers could be mostly network engineers, if so my comments may not apply.There is a real need for some good tutorials why/how/gotcha's to watch out for on how to set up a system following best practices for the advanced home user/small business user (Think startup/small development shop with a couple of programmers who have a decent understanding of network basics, but no IT department/network engineer -- They need a good/secure network, but can't pay for support).
Case studies on how to set up a network with VOIP, Cameras, Media Players/TVs, PCs, OpenVPN LAN gateway (some traffic through the LAN, some thought the VPN), OpenVPN remote access server(s). There are setups for some of these individually, but often they don't explain enough to make it easy to combine the elements. It's often much easier to figure out how to remove an element from a more complex design than add one in - especially if it's documented.
Planning/setup/workflow when combining elements and applying good security practices and getting routing correct is very hard to figure out from scratch, but once you have a basic blueprint (and enough understanding of why it's done that way), it's not that bad. A visual model of the interaction between Interface Rules, Floating Rules, Policy routing/tagging and NAT would have saved me days of struggle (i'm still a bit unclear about some of it.)
Setting up remote access to view security cameras and to be able to access files (I assume this requires two separate remote access servers / firewall rules to do safely.) is very difficult to figure out based on what's out there online.
For my level of experience/needs I'm sad to say many time that Tom Lawrence's YouTube videos were more helpful than the Gold content I paid for. A deep dive/case study approach on 2, 3, 4 (whatever is needed to cover the topics) would be far more useful than the current book for many like myself. Case studies allow the right advanced topics to be covered and in sufficient detail to get the job done properly without having to become a CCIE! I don't know if the community thinks this is a good idea, I just know what would have saved me weeks.
----------------END SUGGESTION]I see it all the time with hairpins and bad uplinking.. in line with Derelict's Two is one, lets also remember that 1 + 1 does not equal 2, it is just 1 and 1 ;)
You do not instantly get 2 gig when you lagg/etherchannel/portchannel/lacp whatever term you want to use two gig connections. Just means you have 2 1 gig connections.
It all comes back to education and experience (don't know if you learned this from experience or formal training.)
This thread caused me to do a bit of research/review and for the benefit of others I thought I would put this out for the benefit of others/comments.
----REVISED----
I just came across these so I have edited this post to correct/improve it:
http://ieee802.org/3/hssg/public/apr07/frazier_01_0407.pdf
http://www.ieee802.org/1/files/public/docs2017/ax-rev-haddock-conversation-sensitive-collection-distribution-0517-v1.pdf
A LAGG bonds 2 (or more) links together into single interface with common ingress and egress queues so they act as one. Data goes into the ingress queue, then some sort of hash based on the Layer 2 header which determines which link get used, and then the packet comes out the egress queue on the other end.
What fields the hash is derived from is dependent on the vendor/age/quality of the hardware.
The important points are:
- As @johnpoz has suggested LAGG does not increase the bandwidth for a single conversation and in this case the only value is graceful failover in the event of a link failure.
- LAGG achieves high utilization only when carrying multiple simultaneous conversations since all packets associated with a given “conversation” are transmitted on the same link to prevent mis-ordering
- The load sharing is generally based on a hash derived from the Layer 2 header and the algorithm will vary with manufacturer/equipment. As a result it is difficult or impossible for a network administrator to predict or control which frames are transmitted on which links. The resulting load‐sharing between the links in the LAG may be very imbalanced.
Anyone have anything to add?
-
You are getting into the realm of "real" network design. If you want to do that you should hire someone who knows how to do it or know/learn how to do it yourself.
No documentation is going to acceptably cover every use case and the people who don't know how will need a step-by-step walkthrough for their specific circumstances and will likely be unable to adapt the docs to their set of criteria.
Those who know how to google can get all of the information from many sources because at the end of the day, we are just talking about IEEE 802.3ad.
All you do is create a LAGG, add the member interfaces, and assign it. That is all covered in the pfSense book. You asked for a specific case of migrating an existing connection to a lagg. You received, what I feel, is valid feedback for that scenario.
Whether or not to use a lagg is not a "pfSense" question. Entire books are already written on all of this. Industry certifications and high $ courses are available, etc.
-
@derelict said in Converting from a Single Interface to a LAGG:
You are getting into the realm of "real" network design. If you want to do that you should hire someone who knows how to do it or know/learn how to do it yourself.
I agree, and the point I am trying to make is there is a huge gap in the knowledge/training sources available out there. I want to learn enough to do the job - I have no interest in becoming a CCNA as I'm not working in the industry. The person who want a good/secure network for a home/small shop with a handful of PCs, a few phones, a few cameras, small file server and smart TV or two that needs to be kept under control has no need for 90-95% of the stuff that is taught in professional courses.
No documentation is going to acceptably cover every use case and the people who don't know how will need a step-by-step walkthrough for their specific circumstances and will likely be unable to adapt the docs to their set of criteria.
I agree that no documentation will cover every use case, but I disagree with your comment about adaptation to particular circumstances. Anyone who is going to set up pfSense has got to know networking basics
I would also suspect that someone who works in the field/knows it well would have a pretty good idea of a handful of reference designs that would fit 80-90% of the use cases with a bit of adaptation. Using Google for the last 10-20% is a whole lot more manageable than using Google for 80-90%!
I also suspect that a small collection of well documented/commented reference designs would improve pfSense Gold membership. Over the years I've dealt with a number of programming reference manuals - the ones that include a short but sensible/meaningful example of the command being describe are infinitely more valuable/useful than than line after line of text that didn't describe as well as a few lines of code. (I'm not suggesting getting rid of the reference manual - but in MY PERSONAL EXPERIENCE I've found the the GUI is so well done that it is almost self documenting. For the issues I had, the reference manual didn't add very much. If I couldn't figure it out from the GUI, the manual didn't get me there-I needed google for basic information (if I as missing a basic network concept) or I needed to ask a question in the forum (if it was more of a pfSense specific issue).
Those who know how to google can get all of the information from many sources because at the end of the day, we are just talking about IEEE 802.3ad.
Actually hasn't it changed to 802.1AX? 802.3ad got me close enough-Thanks for that comment, that gave me an idea of what to google to fill in a couple of missing bits so I'll edit my post above.
Let's face it if someone isn't willing to google for answers then clearly pfSense or any open source project isn't for them. Often when you don't know, you also don't know WHAT to google and that's where the fourm can help a lot.
All you do is create a LAGG, add the member interfaces, and assign it. That is all covered in the pfSense book. You asked for a specific case of migrating an existing connection to a lagg. You received, what I feel, is valid feedback for that scenario.
You are correct, I feel I received excellent feedback that was exceptionally helpful. If I was starting from scratch, I wouldn't have even asked the question. I found that the GUI was all I would need.
I am VERY grateful that @johnpoz injected the question why/are you sure you want to do that? I think based on his comment I'm going to leave things alone until I get time to go over my design in detail. I put LAGG on my FreeNAS, and it didn't seem to matter (and now I know why).
Whether or not to use a lagg is not a "pfSense" question. Entire books are already written on all of this. Industry certifications and high $ courses are available, etc.
I'm wandering a bit and my feedback/comments were more general comments that likely should be in a different sub-forum, but they would lose of of their significance if removed from the context of this discussion. I put the comment out there, since I see it as a need in the marketplace that Netgate might be able to take advantage of to increase their Gold adoption.
You are quite right Whether or not to use a lagg is not a "pfSense" question, and in the context of a good reference design for a small network it likely would not come up at all.
Again, thank for your input--it was the catalyst for filling the the missing pieces and correcting my earlier post.
-
@guardian said in Converting from a Single Interface to a LAGG:
Anyone who is going to set up pfSense has got to know networking basics
You would think huh ;) Clearly not the case...There are people that try and setup pfsense that don't understand why they can not have 192.168.1/24 on both their wan and lan networks, etc.
Another scenario that comes up ALL the time - is why can pfsense not filter traffic between devices on the lan.. So the concept of what a router/firewall is doesn't seem to click to why you can not filter traffic between 192.168.1.100 and .101 on the gateway device..
So basic network understanding is clearly not a requirement for trying to setup pfsense ;) The one that gets me the most is not understanding the basic difference between a resolver and a forwarder in dns.
Derelict's points are very valid though about "not a pfsense question" But thats what we have the forum for - to discuss and exchange information between people that know and have experience and people that do not.
I have been doing network design/support for years. I see many people with years and years in the biz still not grasp some concepts for good traffic flow.. And still think oh will just through a lagg on there when need more bandwidth. And then don't get why there is still issues with congestion, etc.
I still see home runs all the way back to the edge router vs doing routing internally to remove traffic from the uplinks, etc. etc. Teaching this sort of stuff is not something you could expect to get out of documentation for a specific product "pfsense"
Another problem I see is you have people that shouldn't be putting together guides - putting together guides. The amount of BAD information out there is horrible.. They try and latch onto pfsense to draw traffic to their site so they put together some guide on how to do XYZ, and its just horrible advice, etc..
I always attempt to ask the user why they want to do something, because more often then not they are going down the wrong path to the correct solution.. But they get something in their heads and its sometimes impossible to get them to listen they just want to know how to do X, when they should be doing Y. Bridging router interfaces vs a using s switch is a big one that comes up over and over and over again. They don't see anything wrong with it...
Hang out here and sure there will be many discussions on network design in general. But putting together a use case that goes into all the details of why you did this network decision vs another way is a lot of work, and will go WAY over the heads of the vast majority of users here anyway.
The vast majority of users here are just new users to pfsense asking questions - that tend to be the same questions over and over and over again. But there are a few here that know what they are talking about - and that is when it gets fun and interesting is when you get a discussion going where the person wanting to learn actually gets it - your ahah moment was fantastic to me knowing I help light the lightbulb above someones head.. Or discussion about the how's and why of something - Derelict and JimP are very very good just in IT in general and then when it comes to pfsense they are just gods amoung us.. So yes they can smack down some good wisdom, etc..
if you have a networking question that is design related - just ask and sure lots of people will chime in with different views on the subject. Design is not always black and white as you think it might be.. Example I see no problem with running a native untagged vlan with tagged traffic. While Derelict is of the mind that if going to running tag then all should be tagged, etc. Both work - nothing wrong with either way, etc.
But if I have a native interface with no vlans, and then I want vlan on that interface I am not going to change the first naked network to tagged - its just extra work ;) heheh
-
@johnpoz said in Converting from a Single Interface to a LAGG:
But if I have a native interface with no vlans, and then I want vlan on that interface I am not going to change the first naked network to tagged - its just extra work ;) heheh
I would never do that in the first place if it was something like a port to a managed switch. Tagged vlans all day every day no days off from the get-go. And if I was to make that change I would set a window to change the pfSense interface and the single(!) port it's talking to to tagged. It's worth it to do it right. It only takes a minute.
-
^ see my point exactly ROFL ;) His point is valid, but there is nothing wrong with have 1 untagged vlan on your trunk. It is a valid configuration - why they have the native vlan commands in cisco, and use of PVID etc. But Derelicts point is also valid - and if setting up a greenfield network that would prob be the best option.
There are many ways to skin a specific cat to be sure - while there might be a few ways to do something that are correct technically. It might go against standard best practice, and we learn bad habits and sometimes they are hard to shake.
Another example where me and Derelict conflict is using a not rule as more explicit allow in a firewall rule. Where he sees it as blocking with an allow. I don't see it this way, I just see it as more explicit allow, and you have to be very comfortable with your understanding and do valid testing of your rules. And yes above this not rule you should always put in place explicit blocks if that is what your looking to do, etc.. Its just another example where something can not always be black and white in a field that is pretty much always black and white ;)
If it was always the same questions over and over again - I wouldn't be here as long as I have. Now and then something new comes up, or a good discussion gets going on some good techie topic. Not always directly related to pfsense - but in general the board is here to help the pfsense user community and not really a general networking/IT discussion forum. There is the general section for those - and they added the vlan/layer2/switch section - which prob could of been added quite some time ago - but now that some of the appliances are coming with switch ports and not just router interfaces I think that section will see more and more threads going forward for sure. And in that section you are going to be very likely to get into discussion that go deeper into the lan network that moves away from actual pfsense stuff. How you handle uplinks of your vlans to either other switches or your router is not really a "pfsense" discussion - but can be very informative and fun to discuss.
-
Hi @johnpoz @derelict thanks for the replies - good discussion
@derelict said in Converting from a Single Interface to a LAGG:
@johnpoz said in Converting from a Single Interface to a LAGG:
But if I have a native interface with no vlans, and then I want vlan on that interface I am not going to change the first naked network to tagged - its just extra work ;) heheh
I would never do that in the first place if it was something like a port to a managed switch. Tagged vlans all day every day no days off from the get-go.
I agree your way is best practice, but other than sloppy what's the major concern? Security/VLAN hopping?
@johnpoz said in Converting from a Single Interface to a LAGG:
@guardian said in Converting from a Single Interface to a LAGG:
Anyone who is going to set up pfSense has got to know networking basics
You would think huh ;) Clearly not the case...There are people that try and setup pfsense that don't understand why they can not have 192.168.1/24 on both their wan and lan networks, etc.
Another scenario that comes up ALL the time - is why can pfsense not filter traffic between devices on the lan.. So the concept of what a router/firewall is doesn't seem to click to why you can not filter traffic between 192.168.1.100 and .101 on the gateway device..
@johnpoz You just made my point even clearer - that would be 2 points in the "gotchas" section for noobs. Having that type of material well organized would save a ton of time and heartache. I would not expect much greater detail either. The info would be laid out as "here's what works/must do/must not do (and in some cases could do/could not do where there are multiple choices). If you want to know more then google is your friend. They sell plans for "prefab" houses that show you how to build the design they have provided, but they don't teach plumbing and carpentry.
So basic network understanding is clearly not a requirement for trying to setup pfsense ;) The one that gets me the most is not understanding the basic difference between a resolver and a forwarder in dns.
Why is this coming up so much? Are people using a forwarder and wondering why pfBlocker DNS block isn't working?
Derelict's points are very valid though about "not a pfsense question" But thats what we have the forum for - to discuss and exchange information between people that know and have experience and people that do not.
Agreed, but wouldn't it be nice if we could raise the "staring level for the discussion."
I have been doing network design/support for years. I see many people with years and years in the biz still not grasp some concepts for good traffic flow.. And still think oh will just through a lagg on there when need more bandwidth. And then don't get why there is still issues with congestion, etc.
I think it goes back to how this stuff is taught. It's mostly about how to configure stuff, not why/why not do things a certain way. A lot of the training creates robots, not thinkers. A certain percentage will have a good mentor, or take a lot of time to really think about/experiment with things and really learn the craft. The vast majority will learn just what they need to get by and that's it.
I still see home runs all the way back to the edge router vs doing routing internally to remove traffic from the uplinks, etc. etc. Teaching this sort of stuff is not something you could expect to get out of documentation for a specific product "pfsense"
Again it wouldn't be teaching per se. I'd be calling is something like the pfSense Cookbook - Recipies for Home/Small Office Networks. Define the scope, and include the caveat to use at your own risk and google is your friend.
Again, provide what to do, and maybe include a paragraph in the "got ya section (or not)". Its about providing a blueprint. Either follow the blueprint or do some research to learn what you need to learn. It's often much easier/more fun to learn by doing.
Another problem I see is you have people that shouldn't be putting together guides - putting together guides. The amount of BAD information out there is horrible.. They try and latch onto pfsense to draw traffic to their site so they put together some guide on how to do XYZ, and its just horrible advice, etc..
Even more reason to have one set up by the community. I wouldn't have been looking for that type of stuff if I could have gotten it here.
I always attempt to ask the user why they want to do something, because more often then not they are going down the wrong path to the correct solution.. But they get something in their heads and its sometimes impossible to get them to listen they just want to know how to do X, when they should be doing Y. Bridging router interfaces vs a using s switch is a big one that comes up over and over and over again. They don't see anything wrong with it...
I have a teacher (in another field) who says "You can't fix stupid.", and this doesn't sound like something that would be in the scope of complexity
that I had imagined. Would this show up in a home or a small office (and I
don't mean a branch office)?Hang out here and sure there will be many discussions on network design in general. But putting together a use case that goes into all the details of why you did this network decision vs another way is a lot of work, and will go WAY over the heads of the vast majority of users here anyway.
And for what I was suggesting that isn't necessary. Present ONE/TWO WELL DESIGNED examples of a network that includes the most common things that a SOHO/Home user would need with a few examples of how to make common modifications.
The reason for the design(s) is(are) to illustrate how to set up firewall rules (interface/floating) and NAT when working with Multiple VLANs and OpenVPN clients/servers. I know of nowhere that is properly illustrated/explained - and that is pfSense not general networking.
What I find most difficult is visualizing how the various elements interconnect with pfSense. I got as far as diagramming the interface and floating rules. I still haven't got lo, NAT, and OpenVPN clear. This is what I did as a personal reference (I posted it, but the forum conversion has totally mangled it) Ascii art is a bit lacking for this type of task.
LAN ----*FW--(192.168. 1.1)--| | VLAN10 ----*FW--(192.168.10.1)--| | WAN VLAN20 ----*FW--(192.168.20.1)--| Interface Public or | IP Gateway IP VLAN30 ----*FW--(192.168.30.1)--@@@--NAT--(x.x.x.x)-FW*---WAN--- (p.p.p.p) | VLAN40 ----*FW--(192.168.40.1)--| | VLAN50 ----*FW--(192.168.50.1)--| | VLAN60 ----*FW--(192.168.60.1)--| Sample Firewall with Multiple VLANS and a Single Internet Gateway ----LOCAL I/F---- *FW ---GATEWAY I/F--- IN (I) OUT OUT (O) IN ---> ~F>R*********F<R< | >F>R*********F<R< <--- LAN ~L>U* --> *L<U< | >L>U* <-- *L<U< or ******~O>L* I/F *O<L<~@~~~~~~|~~~~~~@~>O>L* I/F *O<L<****** WAN VLANx ~A>E* RULES *A<E< | >A>E* RULES *A<E< ~T>S*********T<S< | >T>S*~~~~~~~~T<S< Detail of FW Blocks with Floating Rules DEFINITIONS IN/OUT: Direction is always with respect to the interface. SOURCE: The Initiator of the session to be blocked. DESTINATION: The recipient of the session to be blocked I/F RULES: Rules applied on the Firewall / Rules / (LAN,WAN etc.) Tab for a given interface. FLOAT RULES: Rules applied on the Firewall / Rules / Floating Tab. The IP for an interface sits inside the firewall block (FB) and access to it can be blocked with a firewall rule Floating Rules are processed before I/F rules, so they can be used to bypass I/F rules. Once the firewall block is opened for a session, the rules are no longer evaluated for that session. When creating rules that block an open traffic flow, the states associated with that traffic flow must be cleared before the new rule will be effective. So either the traffic flow must stop and the state must time out and clear by itself, or you must use thw WebGUI to either clear the chosen states, or to clear all states. (Clearing all states will disrupt streaming traffic and downloads.) Rules can only block traffic in the direction shown by the arrows shown with the particular rule. Interface/Normal rules can only block traffic going into that interface - either from the WAN into pfSense box, or from a (V)LAN into pfsense as shown. Floating rules may use one rule to appear on multiple interfaces simultaneously, and will work as if they were placed on the diagram as shown.Anyway that's my $0.02 for what it's worth.
-
@guardian said in Converting from a Single Interface to a LAGG:
pfSense Cookbook - Recipes for Home/Small Office Networks
If I wrote that I would sell it, not give it away. Just sayin'.
-
@derelict said in Converting from a Single Interface to a LAGG:
@guardian said in Converting from a Single Interface to a LAGG:
pfSense Cookbook - Recipes for Home/Small Office Networks
If I wrote that I would sell it, not give it away. Just sayin'.
Who said anything about giving it away?
I was suggesting that this is what would provide REAL value for Gold Membership for the hordes of neophytes that attempt to set up pfSense. I did a gold membership for awhile as a "thank you", but I didn't feel that I got much value out of it other than supporting the project.
It could also be a stand alone product - As I said the education out there sucks for someone who wants to learn enough to do good small networks, but not become a CCNA. I was suggesting Netgate might do something like to raise the profile of the project, and make a few $$. The big obstacle with setting up pfSense is the configuration. I would suspect a publication like that could bolster the sales of their appliances if promoted correctly. Think the person who knows a bit, is reasonably handy with computers, knows that even the high end consumer routers out there are untrustworthy/not updated etc, and need a hand to get to the next level.
I don't know if they sell their lower end appliances to "educated consumers" or just enterprise, but that's the market I see it serving. When I was getting started I would have paid $100 for that in a heartbeat - and depending on the contents, I might still be in the market.
