HFSC & Codel
-
Does anyone have a good multi LAN setup? If I understand past posts correctly, LAN sharing is one of the more difficult / impossible things to do… Any volunteers?
Option 1: Segregation
Give x-Mbps to LAN1, and x-Mbps to LAN2
Downside: unable to utilize full bandwidth if the other LAN(s) are quietOption 2: VLAN
Using a managed switch, tag each LAN as it's own VLAN. Funnel each VLAN into it's own queue. Prioritize each queue appropriately.
Downside: more complicated to set up. I think this may not prioritize within each VLAN (only between VLANs), but I could be mistaken. -
Tagged VLANS present as different interfaces in pfSense, so I'm unclear how option 2 differs from option 1. Something I don't understand?
-
Tagged VLANS present as different interfaces in pfSense, so I'm unclear how option 2 differs from option 1. Something I don't understand?
Using floating rules, you can funnel the VLANs into their own queues, so each queue (in HFSC) could utilize the full bandwidth if the other queue was quiet.
The segregated LANs can't do this to my understanding. If I'm mistaken, then you could probably do the same.
-
Animosity022
I see that you have set shaper limits to 290/290 (3.3% down from ISP limit) and speedtest result you provided is 278/278, this is about 4.1% loss?This is strange because I have 300/300 PPPoE and my bandwidth loss is about 3% down from limit set in shaper, when shaper enabled, but I have also limiters enabled, for equalizing LAN clients. For example if my limit is set to "ISP limit -8%", ex ~275Mbit/s, then maximum bandwidth I really have is 265 Mbit/s.
And just for information what is your speedtest results without shaper? What CPU load you have?
Thanks. -
@w0w:
Animosity022
I see that you have set shaper limits to 290/290 (3.3% down from ISP limit) and speedtest result you provided is 278/278, this is about 4.1% loss?This is strange because I have 300/300 PPPoE and my bandwidth loss is about 3% down from limit set in shaper, when shaper enabled, but I have also limiters enabled, for equalizing LAN clients. For example if my limit is set to "ISP limit -8%", ex ~275Mbit/s, then maximum bandwidth I really have is 265 Mbit/s.
And just for information what is your speedtest results without shaper? What CPU load you have?
Thanks.I'm a Verizon FIOS customer in s a smaller neighborhood that just went live maybe 6 months back so I can't imagine I have too many folks. I'm provisioned 300/300, but without the shaper I always test well above.
I use this for my router:
https://www.amazon.com/gp/product/B01MEGSMRZ/ref=oh_aui_detailpage_o05_s00?ie=UTF8&psc=1
When I push my link in both directions, I can tag around 30-40% CPU usage. I have telegraf installed so I can post any cpu/memory/network stats as well if you are curious.
I've found in more testing that I can really set to 300/300 safely and I've done more testing with FQ_Codel on some other distributions and I can get better bufferbloat.
At present, I'm running on FQ_Codel and 300/300:
CPU Load and Usage % from my speedtests this morning:
-
@w0w:
Animosity022
I see that you have set shaper limits to 290/290 (3.3% down from ISP limit) and speedtest result you provided is 278/278, this is about 4.1% loss?This is strange because I have 300/300 PPPoE and my bandwidth loss is about 3% down from limit set in shaper, when shaper enabled, but I have also limiters enabled, for equalizing LAN clients. For example if my limit is set to "ISP limit -8%", ex ~275Mbit/s, then maximum bandwidth I really have is 265 Mbit/s.
And just for information what is your speedtest results without shaper? What CPU load you have?
Thanks.I'm a Verizon FIOS customer in s a smaller neighborhood that just went live maybe 6 months back so I can't imagine I have too many folks. I'm provisioned 300/300, but without the shaper I always test well above.
I use this for my router:
When I push my link in both directions, I can tag around 30-40% CPU usage. I have telegraf installed so I can post any cpu/memory/network stats as well if you are curious.
I've found in more testing that I can really set to 300/300 safely and I've done more testing with FQ_Codel on some other distributions and I can get better bufferbloat.
At present, I'm running on FQ_Codel and 300/300:
CPU Load and Usage % from my speedtests this morning:
Thank you for information. I have the same CPU but on the other board, it is Asrock J1900D2Y.
If other distribution is FreeBSD based, I am pretty sure FQ_CODEL is used with dummynet and IPFW, so yes, it is a little bit better, then classic ALTQ shaper, I've tested this also.
I also was looking for some flexible shaper that can detect bufferbloat on ISP/upstreaming router and ajust bandwidth limit on the fly, but I have no success. -
My WAN and LAN queues are the same. I only posted my WAN. I just noticed I have dupe ICMP floating rules.
-
Thank you Harvy!
-
firewall software I ran codel and the wizard with HFSC and in DSLReports am now getting A+ for bufferbloat, this I can get only with pfsense other router never get these good results for me
-
Apologies for bumping an older thread, but I have a question directly related to Harvy66's reply.
I think I've more or less wrapped my head around how to have QoS working correctly, but I don't understand why Harvy has codel turned on for only some of his queue's… namely ACK and LowPri. Any guidance that someone could she would be appreciated.
Thanks!
~Spritz
-
Apologies for bumping an older thread, but I have a question directly related to Harvy66's reply.
I think I've more or less wrapped my head around how to have QoS working correctly, but I don't understand why Harvy has codel turned on for only some of his queue's… namely ACK and LowPri. Any guidance that someone could she would be appreciated.
Thanks!
~Spritz
An over-simplification would be that small queues (CoDel) drops packets in an effort to keep latency low while a large queue could eat random bursts while dropping no packets but latency would increase/fluctuate. Certain traffic like streaming or bulk downloads would probably prefer large buffers while VOIP or DNS would prefer small buffers.
You might try searching Google for "cisco buffer OR queue depth OR length OR limit". Cisco's documentation is sexy. You might also look up some generic network queueing/buffering wikipedia articles to see what situations call for buffers.
Really, CoDel should be safe to enable on any traffic type (except UDP?) but maybe there is certain traffic that you dislike and want to force an oversized buffer to discourage it rather than block it?
VOIP, for example, is usually very precise with the bandwidth it needs so you can precisely allocate that amount of bandwidth. In that case, VOIP probably would not benefit from CoDel.
-
Most protocols that use UDP are more sensitive to latency than loss. There are some TCP-like usages of UDP that responds to loss. There is very little reason to not use Codel. I pretty much only don't use Codel for situations where there is always going to be enough bandwidth in the queue and I don't want the "overhead" of Codel. My ACK queue has 20% bandwidth, which is complete overkill on my symmetrical connection. ACKs tend to only consume about 1/30th of your ingress, which can be an issue for asymetric connections.
One thing of note. Even ACK handle loss better than latency. If you have an asymmetric connection where you don't want to give ACKs too much bandwidth, you can use Codel or a smaller queue. This mostly applies to bulk transfers. Video games that use TCP may not like ACK loss.
-
Thank you for the replies and explanation.
So if I'm understanding you correctly, I believe I've set everything up correctly for my situation. I've used the wizard as a starting point, allowing it to set the service curves. I've then tweaked the following settings –>
- Set upload and download to 95% of measured max
- disabled Explicit Congestion Notification for all queue's
- enabled codel for all queue's, with the exception of VoIP and Ack (both inbound & outbound)
- set the queue limit for all queue's to 1024
Does this make sense?
Thanks again!
~Spritz
-
For those interested, in the end I ended up with a fairly simple configuration.
WAN Scheduler Type HFSC, Bandwidth 22.5Mb qAck Priority 6, Queue Limit 1000, Bandwith 20% qInternet Bandwith 80% qDefault Priority 2, Queue Limit 1000, Default Queue, ECN, Codel, Bandwith 75% qHigh Priority 4, Queue Limit 1000, ECN, Codel, Bandwith 10%, Link Share m2 10% qLow Priority 1, Queue Limit 1000, ECN, Codel, Bandwith 5%, Link Share m2 5% LAN Scheduler Type HFSC, Bandwidth 115Mb qAck Priority 6, Queue Limit 1000, Bandwith 20% qInternet Bandwith 80% qDefault Priority 2, Queue Limit 1000, Default Queue, ECN, Codel, Bandwith 75% qHigh Priority 4, Queue Limit 1000, ECN, Codel, Bandwith 10%, Link Share m2 10% qLow Priority 1, Queue Limit 1000, ECN, Codel, Bandwith 5%, Link Share m2 5%The bandwidth values for WAN and LAN are between 90% and 95% of peak available. I also have DMZ and GUEST which are configured identical to LAN except that they have smaller bandwidth specifications. I decided to live with a bit of conflict between LAN, DMZ and GUEST rather than trying to stand on my head and spin like a top. :)
Thank you all for your help. Additional comments or suggestions are welcomed.
-
For those interested, in the end I ended up with a fairly simple configuration.
WAN Scheduler Type HFSC, Bandwidth 22.5Mb qAck Priority 6, Queue Limit 1000, Bandwith 20% qInternet Bandwith 80% qDefault Priority 2, Queue Limit 1000, Default Queue, ECN, Codel, Bandwith 75% qHigh Priority 4, Queue Limit 1000, ECN, Codel, Bandwith 10%, Link Share m2 10% qLow Priority 1, Queue Limit 1000, ECN, Codel, Bandwith 5%, Link Share m2 5% LAN Scheduler Type HFSC, Bandwidth 115Mb qAck Priority 6, Queue Limit 1000, Bandwith 20% qInternet Bandwith 80% qDefault Priority 2, Queue Limit 1000, Default Queue, ECN, Codel, Bandwith 75% qHigh Priority 4, Queue Limit 1000, ECN, Codel, Bandwith 10%, Link Share m2 10% qLow Priority 1, Queue Limit 1000, ECN, Codel, Bandwith 5%, Link Share m2 5%The bandwidth values for WAN and LAN are between 90% and 95% of peak available. I also have DMZ and GUEST which are configured identical to LAN except that they have smaller bandwidth specifications. I decided to live with a bit of conflict between LAN, DMZ and GUEST rather than trying to stand on my head and spin like a top. :)
Thank you all for your help. Additional comments or suggestions are welcomed.
In theory, you can move the bandwidth limiter to the qInternet level, and have a qLink available, so inter-vlan-guest-dmz communications can be done at full speed
-
moscato359 makes a good point. In the case of multi-LAN, separating intra-LAN traffic from Intranet can be useful. If you go this route, I would recommend placing your ACK queue under qInternet.
More of a philosophical reason, but my default queue is qLow. I have a lot of normal traffic that is not low, but I don't care enough to add a rule. Because of this, I set my qLow pretty high, like 20% bandwidth. The reason for this is most traffic that is a "bandwidth hog" is also incredibly difficult if not impossible to classify.
-
In theory, you can move the bandwidth limiter to the qInternet level, and have a qLink available, so inter-vlan-guest-dmz communications can be done at full speed
Yes, the wizard does this. To make it work properly requires more firewall rules. In my case, it wasn't necessary because there is very little traffic between LAN and DMZ, and none at all between GUEST and LAN or DMZ.
-
Truth be told, there are actually two other interfaces that I didn't bother to mention, nor did I bother with shaping on them. Combined they average about 1Kb. :)
-
I have a question that you may be able to answer quickly, but if not I'd be glad to open a new post. I have my traffic shaping rules set up very much like yours, but I have asymmetric upload and download speeds, which caused me to question whether I'm matching on the correct interfaces and/or directions. As an example, I have a floating rule set to match inbound UDP traffic on the LAN interface whose destination is port 53 and assign it to a higher priority queue to prioritize DNS traffic. However, I think I recall that traffic gets assigned to the queue on which the match was made, so in this case I believe that I am erroneously assigning outbound DNS queries to a queue on my LAN interface (i.e. the download queue). I'm wondering if I should instead change this rule to match outbound on the WAN interface? If this isn't clear I'd be glad to provide more details. Or, if it's not too much trouble, could someone describe how to set up the match (on which interface(s) and direction(s)) in order to successfully assign download traffic to a queue on the LAN interface and upload traffic to a queue on the WAN interface? I think that's all I really need. Thanks in advance for any assistance.
-
I don't believe you need to have interface specific rules, just floating rules. See attached.
