WAN Port Bonding / Link Aggregation Group / DOCSIS 3.1
-
Bare with me as I try to explain myself.
From what I have read DOCSIS 3.1 is capable of being full duplex of 10 Gbit/s Downstream and 10 Gbit/s Upstream, supports fully symmetrical speeds. That being said if my ISP would offer it to me or if I would even want to pay for that is a totally different question.
Currently my ISP (Cox) is only offering for their top tier 1 Gbit/s Downstream and 35 Mbit/s Upstream. I know that currently, some people that I know that have this service (depending on the strength of the signal their cable modem is receiving) are getting speeds of more or less around 930-975 Mbit/s Downstream and their Upstream is consistently at least 35-42 Mbit/s Upstream. They are all feeding their Router from a single 1ge ethernet port and no one is ever getting exactly 1 Gbit/s Downstream or more. (I have heard of people around the country with 1 Gbit/s tier internet packages and using lagg over 2 x 1ge ethernet ports getting consistently around 1-1.2 Gbit/s since their medium is capable of ~2 Gbit/s).
Previously they had an internet tier (which is currently my tier) that is 300 Mbit/s Downstream and 30 Mbit/s Upstream but get on average around 320-355 Mbit/s Downstream and 30-35 Mbit/s Upstream.
The reason I would want to use lagg is to make sure I had more than enough overhead to accomodate 1 Gbit/s since I would be paying for 1 Gbit/s I'd want to get that at minimum. The modem has this capability ( https://motorolamentor.zendesk.com/hc/en-us/articles/115010589128 ) and I would like to utilize it and not to mention be able to use the higher packages that would be offered in the future 2 Gbit/s etc. Theoretically, the Motorola MB8600 should be able to handle speeds up to 3.8 Gbit/s from the ISP if they offered a 4 Gbit/s tier, but due to the overhead I couldn't get exactly 4 Gbit/s, the same way I wouldn't be able to get the full 1 Gbit/s if I upgraded to the 1 Gbit/s tier using only 1 ethernet Gbit/s port.
I'm not sure what you are referring to with the 1+1 analogy? I thought that if you had setup 802.3ad LACP or lagg on two 1ge ports that they would be capable of higher throughput of something closer to 2ge?
Also, keep in mind that the 4ge Port Bonding is not yet available but will be in the future as stated by Motorola, to support higher throughput and internet speeds offered via DOCSIS 3.1.
-
"From what I have read DOCSIS 3.1 is capable of being full duplex of 10 Gbit/s Downstream and 10 Gbit/s Upstream"
No what you read is that "full duplex" docsis 3.1, which is an extension of docsis 3.1 is touting symmetrical 10/10
" I thought that if you had setup 802.3ad LACP or lagg on two 1ge ports that they would be capable of higher throughput of something closer to 2ge?"
It is with lots of clients and lots of destination macs.. Ie like over an uplink in a switching setup.. Not all laggs are created equal.. There is a HASHING method that determines what physical interface/wire the packets go out on, etc..
When you talking 1 mac talking to 1 mac.. Good luck seeing traffic load share across the physical connections. In the case this sort of connection.. What mac would be talking to what mac.. Your pfsense mac to your gateway mac..
So what method are they using with this lagg that is going to load share the connections over the pipe.. When you start moving traffic over different physical devices. The nic and the wire and then the other nics on the other end of your lagg. When you split up the gain more bandwidth you no introduce issues with out of order packets.. Than just cause slow down in the traffic, retrans, etc. etc. Buffering if trying to queue them and put them back in the correct order, etc..
If you have different sessions sending data, client A could use path 1, client B talking could use path 2 and you no don't introduce out of order packets, etc.
If they want to provide 10ge up and down - then the solution is 10gig interface.. Or higher even… Not putting multiple smaller connections together in a lagg or bond.. Not the correct solution!!
Here is good starting article on the whole 1+1 not = 2 thing... its from 2010... But it should give you the idea about how throwing more wires at the thing does not mean you get 1+1+1+1 etc.. for your pipe.. Its just you have 4 1ge pipes, not a 4ge pipe..
http://packetpushers.net/the-scaling-limitations-of-etherchannel-or-why-11-does-not-equal-2/So how exactly are they going to hash the traffic to send it across your 2 or 4 paths that allow for any client to see more than the limit of physical pipe.. And not run into any sort of packets out of order issue? Keep in mind then in normal ipv4 setup, especially in the home you will be behind a napt and source IP is always going to be the same.. For all your sessions, etc.
The point heper is trying to make I believe and I agree with is 1+1 does not equal 2 just because you lagg, etherchannel, port channel, nic bonding - whatever term you want to use, etc. So you thinking you are going to see something with this lagg is highly unlikely, even your service was more than 1+ up and down.. What are you clients connected at on the lan side btw? Are 10 or 802.3bz? What sort of traffic are you going to be doing that you will be exceeding more than 1 gig on the uplink? How many boxes will you have streaming netflix? ;)
edit:
Think of it this way.. You have a 4 lane highway.. Can your car drive in all 4 lanes at once? There is a speedlimit in this lane. Say you have 4 people and you all get in different cars and drive in different lanes.. Are you going to get there any faster? But now the traffic in these different lanes all have different amounts of cars in them.. So your 4 people don't get there in order when they are suppose to.. The person that left 3rd might actually get there before the person that left 1st.. The waiter not going to give you your table until your all there.. Shoot he might even say sorry come back again... So now you got to all leave the house again and hope you arrive at the same time in the correct order.So while you could all get in your cars and drive in the same lane right after each other, so now you get there in the proper order.. You have to obey the speedlimit of that 1 lane. But another party going somewhere else could drive in the other lane. So if you work out total bandwidth if you put lots parties all driving their group of cars staying in their own lanes, but using all 4 lanes then yeah you get 4 x the speedlimit.. But no party going to 1 dest even sees more than the speedlimit of the 1 lane.
-
I think I "semi-understand" some of what you just mentioned johnpoz. Please correct me if I understood incorrectly:
"I thought that if you had setup 802.3ad LACP or lagg on two 1ge ports that they would be capable of higher throughput of something closer to 2ge?"
"It is with lots of clients and lots of destination macs.. Ie like over an uplink in a switching setup.. Not all laggs are created equal.. There is a HASHING method that determines what physical interface/wire the packets go out on, etc.."So you are saying that even if the lagg is 1+1 "2ge" that it won't hit that speed or throughput mark unless there is enough traffic going through it that adds up to ~2ge? Even if this was the case I would still like to have the capability of doing it even if I never hit that mark with all my clients on from the LAN side, my traffic wouldn't be limited to ~1ge throughput but ~2ge throughput instead. To use an analogy I guess you can say that "I'd like a Ferrari and will most likely drive it around 65-80 miles an hour most of the time but would like to know that if I wanted to I can go 150 miles an hour and know that I can handle that speed, plus I have an extra $200,000 to buy a Ferrari and I can". Instead of being just at the limit of what I can handle I'd like to be able to have the capability of handling more even if currently most of the time I stick to my current limit. I hope I am making sense? lol ;)
"If they want to provide 10ge up and down - then the solution is 10gig interface.. Or higher even… Not putting multiple smaller connections together in a lagg or bond.. Not the correct solution!!"
I fully agree with you on the 10gig interface or higher part. But check out the Motorola MB8600 technical specifications, why would they even design it the way they did if it wasn't to be used in that way?
In the end, at the consumer level there are no routers that can do what I want or even work with the Motorola MB8600 to its capabilities. That is why building my own pfsense router to me makes the most sense. I'm not a network engineer but can configure and figure out any consumer level router to do what I want or what it is capable of no problem. I am eager to learn more on pfsense after building my own.
"What are you clients connected at on the lan side btw? Are 10 or 802.3bz? What sort of traffic are you going to be doing that you will be exceeding more than 1 gig on the uplink? How many boxes will you have streaming netflix? ;)"
My main client (PLEX server) is using TN9510 10GBase-T/NBASE-T Ethernet Adapter SPF+ ( https://www.amazon.com/gp/product/B00QX4XTE6/ref=oh_aui_search_detailpage?ie=UTF8&psc=1 ) and I know that I am not using anywhere near its full potential but that being said I know for sure that I have enough overhead to support multiple 4K stream concurrently ;D "set it and forget it type mentality".
-
what would be the point of this?
Educational purposes.
https://www.freebsd.org/doc/handbook/network-aggregation.html
:)
edit- Im using FEC on mine right now.. I am passing traffic on both ports. Whether its working "correctly" or not… ? I haven't added any latency to the connection, but in fact have lost 2ms on my WAN according to the graph since employing LAGG.
-
Did anyone in this thread try LAG with their Motorola MB8600? I have it and pfSense, too.
Really curious if any more progress has been made on this front, as I just signed up for gigabit with Comcast last night. :o
Thanks!
-
I have, it works. Enable Port Bonding on your MB8600. Then configure the pfSense to use a LACP LAGG to connect to ports 1,2 on the MB8600. Assign that LAG to your WAN interface, and you should be good to go.
-
I found this old topic searching after running into problems with LACP on my Arris SB8200.
I have gigabit service but have only been getting 750-800mbps and thought it might be an issue with a bottleneck somewhere in my hardware. (using a supermicro C2758 motherboard) Recently, a firmware update was pushed out to enable LAG in the SB8200. Figured I might as well. I set up an LACP under the LAGG interface and connected my second ethernet cable.
Results...I noticed no difference in speed. Of course this makes sense since my network is nowhere near being even close to saturated. What I did notice is a few errors, all corrected, on all of my downstream bonded channels. Also, in pfSense, several hundred errors in the "errors out" on my WAN interface. I ended up turning off the LAG and going back to a single interface and all the errors went away.
-
I too didn't quite understand how LAG worked and tried bonding my desktop to my switch in the hope that if one file transfer was hitting 1Gig, I could still access the other HDDs in the NAS at the same time (which has 10Gig). It didn't work reliably at all, just slowed down all file transfers!
Even maxing out 1Gig to my NAS and using the other link to access the Internet seemed sub-optimal, possible due to the second NIC being USB.
Clearly as mentioned above, LAG doesn't benefit a client, it benefits the server when multiple clients are hitting it at the same time. Or switch to switch uplinks where again it will load balance the clients, not the individual traffic from a single client.
-
I too had issues enabling LAG on my SB8200.
When enabled on the modem and using LACP for my lag interface in pfsense this is what i noticed.
1: lots of corrected errors. millions per second showing up on the modem status page
2: Router would cyclically lose internet on both ipv4 and ipv6 every 50 minutes.
3: When "enabled lag" was checked I could not access the spectrum analyzer on port 8080
I'm going to guess it's an issue with the sb8200 firmware and lag.
Or maybe it does not use LACP?The modem losing internet every 50 minutes would cause the alerts to start piling up in the pfsense dashboard.
-
Did anyone get running LAG with a Technicolor TC4400 cable modem? I got it running but the speed is much worse than without.
