Variable connection speed yet share what's available evenly per IP?
-
Our connection is getting more congested at peak times lately as our backhaul is oversubscribed. At peak congestion, the connection's max speed will hover between 512kbps-1Mbps down and can get quite choppy in terms of latency. When things are running smoothly the connection will be at about 3Mbps down (this is on ADSL1, presumably connected to a Conklin as sync is full speed). This makes it hard to set a traffic shaper as setting it to a very low speed means it would be very slow all the time and it still doesn't seem to give everyone a fair slice of the pie (one IP still seems to be able to monopolise the connection).
I've found running without the shaper works best most of the time unless anyone starts continuously streaming or downloading. If that's the case, that IP will completely dominate the external bandwidth to the point that even pings from other hosts to sites outside the network time out. Other tasks performed on the most active device will run, albeit very slowly. Just a couple of examples I've recently noticed were whilst updating via aptitude on Debian and grabbing the latest LTS Edubuntu ISO. Other people on the network cause the same thing with Youtube streaming and Skype for example.
The dashboard doesn't show any sensors as pegged and there's generally not more than a half-dozen devices online at once. This post suggesting that m0nowall does a good job at this has got me thinking of trying that but it would still seem to require specifying a fairly stable connection speed for the shaper to work from.
Is there anything out there that can poll what the max throughput on the WAN has been recently and adjust the available bandwidth and shape per IP accordingly? Or by any other mechanism?
-
I am looking for a variable traffic shaper as well. My understanding is that pfsense lacks such a feature. I would think the best way to implement a variable traffic shaper would be to use a limiter with time based adjustment cycles and the following:
A: A preset bandwidth which the limiter will use when initialized, this value will be the lower limit and the bandwidth value will not go below this value.
B: A max bandwidth value, which the limiter will never exceed.
C: A ping value of a specific IP address pinged every adjustment cycle (assumed to be default gateway.)
D: A lower ping value, assumed to represent the baseline acceptable ping value for C.
E: A upper ping value for C, this represents an unacceptable ping value.
F: A 'up' threshold integer variable to represent how much the limiter bandwidth should increase should C be less than D.
G: A 'down' threshold integer variable to represent how much the limiter bandwidth should decrease should C be greater than E.
H: A optional 'up' multiplier such that the difference between C and D is factored into an instantaneous up threshold. The instantaneous up threshold will be calculated by ((D-C)*H)*F.
I: A optional 'down' multiplier such that the difference between E and C is factored into an instantaneous down threshold. The instantaneous down threshold will be calculated by ((C-E)*I)*G. -
I believe this is impossible. It's only possible to shape traffic by dropping packets inside YOUR network. For that the shaper needs to know the maximum upload & download speeds. If the connection externally drops below the specified speeds then packets will start getting dropped by the ISP according to their shaping rules instead of yours.
-
I would also love to dream up a solution to this. I have some links where the contract provides a minimum upload/download speed (e.g 1Mbps/1Mbps) but the provider gives us more if it is available - e.g. during business hours we often see exactly the 1Mbps, but after hours a download might run at 4 to 8 Mbps.
If I implement a traffic shaper setup and tell it the WAN is 1Mbps, then I waste the potential of the extra bonus.
Conceptually, such software could have parameters so it would periodically let some extra out of the download queues to the LAN clients. Then it would monitor to see if that resulted in the rate of data coming in from WAN increasing (i.e. the client connections TCP window-size etc algorithms would get the idea and pickup speed). Then some monitoring also to see if any of the queues seem to be getting starved of their "chunk" of bandwidth, indicating that the available WAN bandwidth is not so much any more, and throttling back the download queues…
But there are lots of issues, parameters to think about in designing such a thing.
Any magic bullet out there? -
You can create a utility that measures the latency and converts that to bw value.
After that instrument the pipe of queue.Its doable but not in priority anyhow.
-
Gargoyle Router's Active Congestion Controller seems to be a good way of dealing with this situation, here's some more discussion on it.
-
In my cases in Nepal, on a quiet connection, the ping time to anywhere except the direct ISP device at the other end of the ADSL copper phone wire, is completely variable. Sometimes 50ms, sometimes 500ms and anywhere in between. If I run a speedtest I usually get my full paid-up bandwidth, even though the ping times are long (or short). The ISPs here seem to have all sorts of problems that come and go, routes go "via-the-moon" for a bit, then come good. Just because the ISP has routed the long way doesn't always mean they have slowed down the data transfer rate.
I wonder how any algorithm at the user end can effectively guess what the current max achievable link speed is?
(apart from the QoS software actually doing a download in the background every so often as a test and then adjusting its view of the link based on the results) -
QoS software actually doing a download in the background every so often as a test and then adjusting its view of the link based on the results….
Seems like for that to work well it would have to run often enough as to become a bandwidth hog its self?
-
QoS software actually doing a download in the background every so often as a test and then adjusting its view of the link based on the results….
Seems like for that to work well it would have to run often enough as to become a bandwidth hog its self?
Yes, it somewhat defeats the purpose!
The real solution to all these problems is that the ISP provides a manageable service where the customer can put QoS settings on the ISP router end. The ISP router knows how much bandwidth it is letting out to the customer from time-to-time, and can do the prioritizing according to the customer's QoS settings. Wouldn't it be nice if ISPs offered such things to a wide range of customers at a reasonable price. -
In my cases in Nepal, on a quiet connection, the ping time to anywhere except the direct ISP device at the other end of the ADSL copper phone wire, is completely variable.
I assume that's why the Gargoyle solution defaults to pinging the gateway on your WAN connection, to which pfSense's RRD WAN quality graphs also refer.