Help on funneling specific traffic to queues
-
Hey all,
I have been reading voraciously about traffic shaping, and though many aspects still seem a bit beyond my comprehension I think I have the basics of how to set the limits of a hsfc configuration. But what I'm still very fuzzy on, and can't seem to find much documentation regarding, is how to create rules to actually direct specific traffic to specific queues. Could someone point me to some documentation on that?
For more color on what I'm trying to accomplish (since it's pretty simple),
-
I'm trying to funnel Usenet downloads to the same queue as P2P (which I only have cause it was included in the setup wizard).
-
I also have a raspberry pi running Openhab that has a script that tests my internet speed every hour and updates a Google Drive spreadsheet. I would like to make sure this isn't slowed at all by the shaper
-
more generally (and seriously this is all I really want), I'd like My Wife and My personal computer and phones (4 devices) to always have priority over everything else. Then I'd like any streaming music or video (Netflix and Spotify primarily) to take as much bandwidth as they need without stuttering, and finally my P2P and Usenet downloads (which all come from a synology box) to saturate my bandwidth as long as nothing else needs it, but be subservient to all else.
-
-
You use firewall rules to assign traffic to queues. Unassigned traffic goes into the Default queue.
Regarding HFSC; just use link-share's m2 and maybe upper-limit's m2. Forget about m1 & d and real-time. That should keep your HFSC introduction reasonably straightforward.
-
I would seriously suggest you look at using PRIQ first and get the hang of that. It is MUCH simpler than HFSC, and it will give you the result you want far more predictably. It works purely on a priority basis, and you don't have to screw around with bandwidth allocations and HFSC parameters.
-
In the wizard I set NNTP to lower priority.
Once the wizard is done, go to rules->floating rules and find nntp near the bottom. I changed the port from 443 to 563 for the 2 nntp rules because I use 563 instead of 443. The wizard already places these into the queues.
My issue was that port 563 wasn't given full bandwidth when nothing else was going on, less than half my total. Also if you remove traffic shaper those 2 floating rules that were changed won't be removed. But its an easy way to try different setups in the wizard.
-
I changed the port from 443 to 563 for the 2 nntp rules…
NNTP uses tcp/119. Why would it have 443 there?
-
@KOM:
I changed the port from 443 to 563 for the 2 nntp rules…
NNTP uses tcp/119. Why would it have 443 there?
i think he meant he uses 563 instead of 443 for ssl usenet. Just a guess but I think he probably just skipped over that nntp initially had 119 as its port
My issue was that port 563 wasn't given full bandwidth when nothing else was going on, less than half my total. Also if you remove traffic shaper those 2 floating rules that were changed won't be removed. But its an easy way to try different setups in the wizard.
reassigning nntp is a good idea, but how did you solve this issue above?
-
Ah yes, thanks. My old-school brain totally forgot about NNTP/S on 563. Even more embarrassing is that I have a Giganews account, so you would think I would be a little more aware :-[
-
@KOM haha i think you're being a little hard on yourself :)
as to your suggestion to use PRIQ instead of HFSC…I did consider that, since my setup is so simple, but it seemed like it had a lot of drawbacks (hard limits and no borrowing of bandwidth, if I didn't oversimplify too much) and figured if I'm going to spend the time, might as well just go with the more robust/effective option. Though to implement this step by step approach, is there a way I can just disable a traffic shaper without deleting it? I've set up HSFC a few times and deleted it when it wasn't performing as I like, but its somewhat time consuming to rebuild it each time…
-
@KOM haha i think you're being a little hard on yourself :)
as to your suggestion to use PRIQ instead of HFSC…I did consider that, since my setup is so simple, but it seemed like it had a lot of drawbacks (hard limits and no borrowing of bandwidth, if I didn't oversimplify too much) and figured if I'm going to spend the time, might as well just go with the more robust/effective option. Though to implement this step by step approach, is there a way I can just disable a traffic shaper without deleting it? I've set up HSFC a few times and deleted it when it wasn't performing as I like, but its somewhat time consuming to rebuild it each time…
There should be an Enable/Disable toggle in the Traffic-shaper. "Enable/disable discipline and its children". Do that on the root interface queue and should disable everything below it. Do it for each interface.
-
but it seemed like it had a lot of drawbacks (hard limits and no borrowing of bandwidth
Eh? That's not how PRIQ works at all. There are no hard limits (other than max for your link, ~90-95% of your lowest measurable speed) and bandwidth is available based on priority. You set up your queues with simple priority via a number designation, and higher priority queues always take precedence over lower queues. The only drawback to PRIQ that I'm aware of is that if a higher priority queue fully-saturates your link, all lower queues will be starved.
-
@KOM:
but it seemed like it had a lot of drawbacks (hard limits and no borrowing of bandwidth
Eh? That's not how PRIQ works at all. There are no hard limits (other than max for your link, ~90-95% of your lowest measurable speed) and bandwidth is available based on priority. You set up your queues with simple priority via a number designation, and higher priority queues always take precedence over lower queues. The only drawback to PRIQ that I'm aware of is that if a higher priority queue fully-saturates your link, all lower queues will be starved.
Well, he kinda got the "no borrowing" part right. I think he understands it enough to know it will likely not be the best choice for his setup.
I agree with you though, PRIQ is a good place to start.
-
Does the concept of borrowing even apply? All PRIQ queues have full access to the bandwidth unless a higher-priority packet comes along. Borrowing isn't required because the queues don't have any maximums like UL with HFSC.
-
@KOM:
Does the concept of borrowing even apply? All PRIQ queues have full access to the bandwidth unless a higher-priority packet comes along. Borrowing isn't required because the queues don't have any maximums like UL with HFSC.
I'm unsure what the proper terminology would be, but PRIQ has fundamental deficiencies when multiple queues are requesting max bandwidth. (Lack of) "Borrowing" seems accurate enough.
Anyway, my point was primarily that I think OP has a reasonable grasp of PRIQ vs HFSC. His decision to avoid PRIQ makes sense.
-
@KOM:
Ah yes, thanks. My old-school brain totally forgot about NNTP/S on 563. Even more embarrassing is that I have a Giganews account, so you would think I would be a little more aware :-[
[/quote]Still a newb to this, but tried to read every thread I could find over the last couple years here on traffic shaping. My mind is completely garbled with ports. But yes change default 119 to 563 or whatever SSL port is being used. I figured usenet is an easy test with VOIP and games to see if its working. Which I got stuck on usenet not even using 50% of my bandwidth when nothing else was going on. Of course I get an A in bufferbloat from dslreports.
PRIQ didn't feel as smooth as HFSC while playing games. Just putting CODEL on WAN/LAN didn't work that well either. But I really want HFSC to give at least 80% of my bandwidth to the lower priority queue when nothing else is going on.
Could anyone give an example of qOthersLow queue settings along with what they input in the default wizard WAN and LAN numbers?
For example I put 7 and 300 for WAN/LAN at the start of the wizard.
qlink defaults to
bandwidth 20%qInternet
bandwidth 251658.24 Kbit/s
max bandwidth for queue upper limit checked 251658.24Kb
b/w share of queue checked 251658.24KbAnd lower queue defaults to
bandwidth 5%
B/W Share of backlogged queue checked with m2-5%Are the default settings on the right track, or does something need to be changed??? I am basically trying to do the exact same thing as OP.
-
Following up, it must have been my ISP as it now works. Of course after I decide to post and ask. Ping in game was 1000ms with usenet, now its 100ms. Good enough start maxing out the bandwidth.
edit: RED in/out, ECN, and CODEL on the WAN (qDefault, qGames, qOthersHigh) got me straight A's on dslreports. Ended up keeping usenet at default priority which worked better than putting NNTP under qOthersLow.
-
Anyway, my point was primarily that I think OP has a reasonable grasp of PRIQ vs HFSC. His decision to avoid PRIQ makes sense.
AGH, why was I not notified of new posts!?! :D So yeah, I think this summary of my understanding is valid, I know next to nothing and have a very easy setup, but do have the most basic of understanding and understand that PRIQ probably isn't my best bet but a good starting point.
With this understanding I DID go ahead and start the wizard to build a PRIQ setup. However, after it was done and I began looking at all the floating rules it created I started to get REALLY confused again. Perhaps if I just try to get help enacting the SIMPLEST of PRIQ setup I can use that understanding to get to my (most likely) end point of a working HFSC setup. From the wiki and these posts I learned of aliases and their usefulness in designing these floating rules. Luckily in the DHCP server I have given ALL of my devices static mappings and pre-sorted them into useful groups (i.e. 10.10.10.10-.19 are personal computers, 10.10.10.20-.29 are personal mobile devices etc). I then spent the morning creating aliases that I feel would make for very easy segmentation for a simple shaping setup. In the absolute most basic sense, what I want is this hierarchy, highest priority to lowest:
-
PrimaryDevices
-
HighPriorityDevices
-
HTPCs
-
default (just a catchall)
-
NAS (where all the downloading happens
The first 3 are already defined aliases of IP ranges, the 5th would just be the IP address of my Synology NAS that does all my torrenting and Usenet downloading and I hope the rest would fall into the 4th default queue. Now, I fully understand this is FAR from optimal, I'd prefer to more specifically filter traffic (i.e. not filter everything that goes to the NAS as lowest priority, just Usenet and Torrent activity OR give any type of video streaming the absolute highest priority, but not everything else that may be happening on my HTPC's), but if I can accomplish the above (shape by IP), I think the jump to full understanding would be much more manageable.
But looking at the Floating Rules section I am bit bewildered. I think i can manage to get a queue called qP2P be the lowest priority, but for the associated Floating Rule directing traffic there, would I change the source or the destination to the IP Address of my NAS? And I assume I would make 2 rules, one for TCP and one for UDP, like the wizard did, but would all the rules I make be done on WAN interface (as all the premade rules from the wizard appear to be)? I guess this question really highlights me misunderstanding the relationship between Floating Rules for the WAN and the LAN.
More broadly: am I just kidding myself by trying to do this "simpler" approach (filtering by client IP)? Since so many rules are already built involving ports, would it probably make more sense to just add a bit more specificity to them, such as an IP destination (still don't know whether its destination or source that I'd need to set)?
Sorry if these questions seem like I haven't done enough background reading, but I swear I have, it's just very confusing and the terminology always makes reading it a bit cumbersome. I think the answers to the above 2-3 questions will really help me "teach myself to fish" hereafter (for a least a little while;))
-
-
Maybe try making a floating rule using protocol -> any and put your destination using a single host or alias -> alias or IP address?
Usenet was the thing giving me problems. With the wizard, speedtests give me A's but not full download speed. But usenet goes full download speed. Switching some things around, I can get speedtests to give me full speed but not A's and usenet isn't close to my full speed. That's why I would figure out how to get one thing working against another computer and then make it more complicated with torrents, streaming, uploading and whatever else.
-
For a great explanation of QoS/traffic-shaping, read http://www.linksysinfo.org/index.php?threads/qos-tutorial.68795/
That tutorial explains the difference between download & upload very well.
-
For a great explanation of QoS/traffic-shaping, read http://www.linksysinfo.org/index.php?threads/qos-tutorial.68795/
That tutorial explains the difference between download & upload very well.
Thank you very much! That is quite long. I will def read this but in the meantime would it be at all possible to answer that one question? Or whether I'd want a source or destination IP Address if the floating rule is being applied to the WAN?
-
I will def read this …
well I did it! and you were right @Nullity , it made a lot of the concepts a LOT clearer. Not least of all that QOS really isn't an exact science. I think i'll need to ruminate on my new found knowledge a little bit, but I feel a bit more optimistic now.
Still seems like IP based prioritization might still be a reasonable first step, or maybe I should just skip straight to prioritizing http and let everything else go to default….