HFSC & Codel

dennypage

Would anyone who is using HFSC or PRIQ with Codel care to share their actual configuration as an educational exercise for the less experienced?

Thanks in advance to any volunteers.

Animosity022

My config is pretty straight forward.

I have 300/300 Fios installed so I have my WAN/LAN queue configured at 290Mbits/s.

I have a "High/Default/Low" setup and break up my queues as noted in the image above.

The only thing I check in the queue is Codel Active Queue and I fill in the bandwidth numerically. If I put in 1000 queue limit, I get A+/A/A+ from speedtest. If I leave at the default 50 or blank, I get A+/A+/A since I drop packets and cause some retransmits.

For firewall rules, there are a few posts, but I just use "Floating" "Quick" "Pass" rules setup for my use as my brain processes top down first match better than anything else. This allows me to validate the queue is matching properly since I can click on the sessions and see whats in that rule:

I prioritize IMCP/my TCP ACKs and my VOIP/Gaming Traffic above anything else.

I have a Plex server that the 32400 is normal, but everything else gets nuked to my low queue as that's all my file transfer stuff.

It works well as I've done numerous tests saturating my down/up links to see how things played out.

Speedtest:

w0w

May be I can add some useful info too.
For example you have a couple of forwarded ports via NAT rules. Every port forwarding NAT rule creates also by default firewall rule on WAN.
You have choice, you can use those NAT automatically created rules for matching queues or the second choice — select 'none' for associated firewall rule in the NAT rule settings and create own floating rule with "PASS" and "QUICK" options, for me it looks like traffic assigned much faster to the desired queue, but may be it's just buggy /status_queues.php
I am not sure, do you really need to remove NAT created firewall rules, because according to pfSense docs floating rule is in the first place and with QUICK option must ignore other rules and active queue.
It looks a bit unclear in real life, because queue just works if it used in same floating rule where 'quick' option is ticked. I just preferred not to use automatically created firewall rules (by NAT) .

Thanks Animosity022 for pointing it out to me.

dennypage

@Animosity022:

The only thing I check in the queue is Codel Active Queue and I fill in the bandwidth numerically.

Why not enable ecn?

Nullity

Personally, I experienced a few websites becoming unreachable with forced, bidirectional ECN. I have had no obvious problems with the (Linux tcp_ecn = 2) default unidirectional ecn.

Animosity022

@dennypage:

@Animosity022:

The only thing I check in the queue is Codel Active Queue and I fill in the bandwidth numerically.

Why not enable ecn?

I was just using the keep it simple approach and only using what I needed.

I read a lot in terms of ECN getting mixed results and causing some issues so I just avoided it since I had the results I was looking for.

dennypage

I understand that there used to be an occasional problem with not being able to establish connections when using ecn, but I thought this was pretty much resolved. The shaper wizard enables it by default. Apple purportedly has it enabled for all connections in iOS 10.

dennypage

Does anyone have a good multi LAN setup? If I understand past posts correctly, LAN sharing is one of the more difficult / impossible things to do… Any volunteers?

Again, thanks in advance!

Nullity

@dennypage:

I understand that there used to be an occasional problem with not being able to establish connections when using ecn, but I thought this was pretty much resolved. The shaper wizard enables it by default. Apple purportedly has it enabled for all connections in iOS 10.

Enabling ECN support at the client is different than enabling it at the pfSense device. There should be practically no problems with enabling it at the pfSense device.

moikerz

@dennypage:

Does anyone have a good multi LAN setup? If I understand past posts correctly, LAN sharing is one of the more difficult / impossible things to do… Any volunteers?

Option 1: Segregation
Give x-Mbps to LAN1, and x-Mbps to LAN2
Downside: unable to utilize full bandwidth if the other LAN(s) are quiet

Option 2: VLAN
Using a managed switch, tag each LAN as it's own VLAN. Funnel each VLAN into it's own queue. Prioritize each queue appropriately.
Downside: more complicated to set up. I think this may not prioritize within each VLAN (only between VLANs), but I could be mistaken.

dennypage

Tagged VLANS present as different interfaces in pfSense, so I'm unclear how option 2 differs from option 1. Something I don't understand?

moikerz

@dennypage:

Tagged VLANS present as different interfaces in pfSense, so I'm unclear how option 2 differs from option 1. Something I don't understand?

Using floating rules, you can funnel the VLANs into their own queues, so each queue (in HFSC) could utilize the full bandwidth if the other queue was quiet.

The segregated LANs can't do this to my understanding. If I'm mistaken, then you could probably do the same.

w0w

Animosity022
I see that you have set shaper limits to 290/290 (3.3% down from ISP limit) and speedtest result you provided is 278/278, this is about 4.1% loss?

This is strange because I have 300/300 PPPoE and my bandwidth loss is about 3% down from limit set in shaper, when shaper enabled, but I have also limiters enabled, for equalizing LAN clients. For example if my limit is set to "ISP limit -8%", ex ~275Mbit/s, then maximum bandwidth I really have is 265 Mbit/s.

And just for information what is your speedtest results without shaper? What CPU load you have?
Thanks.

Animosity022

@w0w:

Animosity022
I see that you have set shaper limits to 290/290 (3.3% down from ISP limit) and speedtest result you provided is 278/278, this is about 4.1% loss?

This is strange because I have 300/300 PPPoE and my bandwidth loss is about 3% down from limit set in shaper, when shaper enabled, but I have also limiters enabled, for equalizing LAN clients. For example if my limit is set to "ISP limit -8%", ex ~275Mbit/s, then maximum bandwidth I really have is 265 Mbit/s.

And just for information what is your speedtest results without shaper? What CPU load you have?
Thanks.

I'm a Verizon FIOS customer in s a smaller neighborhood that just went live maybe 6 months back so I can't imagine I have too many folks. I'm provisioned 300/300, but without the shaper I always test well above.

I use this for my router:

https://www.amazon.com/gp/product/B01MEGSMRZ/ref=oh_aui_detailpage_o05_s00?ie=UTF8&psc=1

When I push my link in both directions, I can tag around 30-40% CPU usage. I have telegraf installed so I can post any cpu/memory/network stats as well if you are curious.

I've found in more testing that I can really set to 300/300 safely and I've done more testing with FQ_Codel on some other distributions and I can get better bufferbloat.

At present, I'm running on FQ_Codel and 300/300:

CPU Load and Usage % from my speedtests this morning:

w0w

@Animosity022:

@w0w:

Animosity022
I see that you have set shaper limits to 290/290 (3.3% down from ISP limit) and speedtest result you provided is 278/278, this is about 4.1% loss?

This is strange because I have 300/300 PPPoE and my bandwidth loss is about 3% down from limit set in shaper, when shaper enabled, but I have also limiters enabled, for equalizing LAN clients. For example if my limit is set to "ISP limit -8%", ex ~275Mbit/s, then maximum bandwidth I really have is 265 Mbit/s.

And just for information what is your speedtest results without shaper? What CPU load you have?
Thanks.

I'm a Verizon FIOS customer in s a smaller neighborhood that just went live maybe 6 months back so I can't imagine I have too many folks. I'm provisioned 300/300, but without the shaper I always test well above.

I use this for my router:

When I push my link in both directions, I can tag around 30-40% CPU usage. I have telegraf installed so I can post any cpu/memory/network stats as well if you are curious.

I've found in more testing that I can really set to 300/300 safely and I've done more testing with FQ_Codel on some other distributions and I can get better bufferbloat.

At present, I'm running on FQ_Codel and 300/300:

CPU Load and Usage % from my speedtests this morning:

Thank you for information. I have the same CPU but on the other board, it is Asrock J1900D2Y.
If other distribution is FreeBSD based, I am pretty sure FQ_CODEL is used with dummynet and IPFW, so yes, it is a little bit better, then classic ALTQ shaper, I've tested this also.
I also was looking for some flexible shaper that can detect bufferbloat on ISP/upstreaming router and ajust bandwidth limit on the fly, but I have no success.

Harvy66

My WAN and LAN queues are the same. I only posted my WAN. I just noticed I have dupe ICMP floating rules.

WAN.PNG_thumb

WAN-qACK.PNG_thumb

WAN-qUnclassified.PNG_thumb

WAN-qUnclassified-qUDP.PNG_thumb

WAN-qUnclassified-qDefault.PNG_thumb

WAN-qClassified.PNG_thumb

WAN-qClassified-qNormal.PNG_thumb

WAN-qClassified-qHigh.PNG_thumb

WAN-Rules.png_thumb

LAN-Rules.png_thumb

Floating-Rules.png_thumb

dennypage

Thank you Harvy!

Chrismallia

firewall software I ran codel and the wizard with HFSC and in DSLReports am now getting A+ for bufferbloat, this I can get only with pfsense other router never get these good results for me

http://www.dslreports.com/speedtest/11692051

Spritzup

Apologies for bumping an older thread, but I have a question directly related to Harvy66's reply.

I think I've more or less wrapped my head around how to have QoS working correctly, but I don't understand why Harvy has codel turned on for only some of his queue's… namely ACK and LowPri. Any guidance that someone could she would be appreciated.

Thanks!

~Spritz

Nullity

@Spritzup:

Apologies for bumping an older thread, but I have a question directly related to Harvy66's reply.

I think I've more or less wrapped my head around how to have QoS working correctly, but I don't understand why Harvy has codel turned on for only some of his queue's… namely ACK and LowPri. Any guidance that someone could she would be appreciated.

Thanks!

~Spritz

An over-simplification would be that small queues (CoDel) drops packets in an effort to keep latency low while a large queue could eat random bursts while dropping no packets but latency would increase/fluctuate. Certain traffic like streaming or bulk downloads would probably prefer large buffers while VOIP or DNS would prefer small buffers.

You might try searching Google for "cisco buffer OR queue depth OR length OR limit". Cisco's documentation is sexy. You might also look up some generic network queueing/buffering wikipedia articles to see what situations call for buffers.

Really, CoDel should be safe to enable on any traffic type (except UDP?) but maybe there is certain traffic that you dislike and want to force an oversized buffer to discourage it rather than block it?

VOIP, for example, is usually very precise with the bandwidth it needs so you can precisely allocate that amount of bandwidth. In that case, VOIP probably would not benefit from CoDel.