HFSC/CoDel for 40 devices

moikerz

That's good information, thanks Harvy66! Realtime makes perfect sense; it would be nice if the UI could just say what each block is relative to (parent/root).

So with the concept of "borrowing":

1. using Bandwidth=minimum, and Upperlimit=maximum, does that imply that the child can borrow from the parent until it reaches the child's Upperlimit? Nothing else to set?

2. using percentages, does the Upperlimit of Child1 + Child2 need to equal 100%, or can it be more?

Harvy66

@moikerz:

That's good information, thanks Harvy66! Realtime makes perfect sense; it would be nice if the UI could just say what each block is relative to (parent/root).

So with the concept of "borrowing":

1. using Bandwidth=minimum, and Upperlimit=maximum, does that imply that the child can borrow from the parent until it reaches the child's Upperlimit? Nothing else to set?

2. using percentages, does the Upperlimit of Child1 + Child2 need to equal 100%, or can it be more?

Correct. The percentage can be misleading, but it really is just a short-hand to place relative fixed bandwidth amounts, so it's pre-computed, not dynamic. As long as the child is not at it's upper-limit, it follows normal distribution rules.
Correct. Upper limits restrict, so they can be more than 100% among queues, but no more than 100% for a single queue

Interesting to note is that HFSC effectively distributes bandwidth in ratios. If I set one queue to 1% and another queue to 1%, and they both try to use all of the bandwidth, they will get a 50/50 split.

Nullity

Forget about real-time. Like Harvy66 said, ocus on link-share (and maybe upper-limit on download traffic). Link-share is just a ratio, not an absolute, so pay attention to the proportional relationships of the queues. Keep your rules and queues simple.

Sadly, upload & download need to be treated differently. I would not use upper-limit on upload, because the router can throttle the LAN clients almost immediately to make bandwidth available for VIP traffic.

Download needs to be preemptively throttled, sometimes as low as 60% to account for the exponentially larger delays between when you request a slower download and when the download actually slows down.

moikerz

So here's what I've currently got:

WAN, HFSC, Bandwidth: 5Mbps
- qInternet, CoDel, Bandwidth: 5Mbps
- qDefault, Default, CoDel, Bandwidth: 10%
- qOthersHigh, CoDel, Bandwidth: 20%
- qOthersLow, CoDel, Bandwidth: 5%

LAN1, HFSC, Bandwidth: 900Mbps
- qLink, Default, CoDel, Bandwidth: 895Mbps
- qInternet, CoDel, Bandwidth: 4Mbps, Upperlimit: 4Mbps
- qOthersHigh, CoDel, Bandwidth: 10%
- qOthersLow, CoDel, Bandwidth: 5%

LAN2, HFSC, Bandwidth: 900Mbps
- qLink, Default, Bandwidth: 895Mbps
- qInternet, Bandwidth: 1Mbps, Upperlimit: 1Mbps
- qOthersHigh, CoDel, Bandwidth: 10%
- qOthersLow, CoDel, Bandwidth: 5%

So, here's my questions and notes:
1. So I should remove the WAN-qInternet Upperlimit value? Not sure I understand your reasoning, Nullity.

2. For LAN1-qLink and LAN2-qLink, is this bandwidth correct? Or should it be my Upload bandwidth?

3. LAN1-qInternet + LAN2-qInternet = 5Mbps, is this the appropriate way to shape the LAN2 speed?

Harvy66

You don't need an upper limit on your WAN because the interface is already limited to 5Mb total, Your LAN interfaces are not.

2+3) Because PFSense does not allow sharing bandwidth among interfaces, you are correct that you need to split the bandwidth between LAN1 and LAN2 in fixed amounts. PFSense shapes bandwidth going out because technically you can't shape bandwidth coming in. Your LAN interfaces represent your download.

moikerz

I understand about have LAN1 and LAN2 Bandwidths set as my desired download speed.

But what does that mean for LAN1-qLink and LAN2-qLink? Since they are my default queues for LAN, it sounds like I should remove the qLink parents completely, and make new qDefaults as children of qInternet, correct? (With the obvious side-effect of limiting inter-vlan comms to whatever download speed I configure for the LANx parent)

WAN, HFSC, Bandwidth: 5Mbps
- qInternet, CoDel, Bandwidth: 5Mbps
- qDefault, Default, CoDel, Bandwidth: 10%
- qOthersHigh, CoDel, Bandwidth: 20%
- qOthersLow, CoDel, Bandwidth: 5%

LAN1, HFSC, Bandwidth: 4Mbps
- qLink, Default, CoDel, Bandwidth: ???Mbps
- qInternet, CoDel, Bandwidth: 4Mbps, Upperlimit: 4Mbps
- qOthersHigh, CoDel, Bandwidth: 10%
- qOthersLow, CoDel, Bandwidth: 5%

LAN2, HFSC, Bandwidth: 1Mbps
- qLink, Default, CoDel, Bandwidth: ???Mbps
- qInternet, Bandwidth: 1Mbps, Upperlimit: 1Mbps
- qOthersHigh, CoDel, Bandwidth: 10%
- qOthersLow, CoDel, Bandwidth: 5%

Harvy66

qLink is meant to be used for non-WAN related traffic, like inter-LAN or between PFSense and the LANs. That way all of that traffic can run full LAN speed and not affect the WAN traffic coming in.

moikerz

Yes that's what I understood qLink to be for too :D But I'm not understanding the purpose of setting LAN bandwidth. Does the bandwidth of the parent queues (qLink, qInternet) need to be equal/less than the interface bandwidth?

With the config I posted, is the correct approach (see bold):

LAN bandwidth = 900Mbps
- qLink bandwidth = 896Mbps
- qInternet bandwidth = 4Mpbs
- qOthersHigh, CoDel, Bandwidth: 10%
- qOthersLow, CoDel, Bandwidth: 5%
??

It seems like that should be correct, from how I interpret what you're saying.

Harvy66

@moikerz:

But I'm not understanding the purpose of setting LAN bandwidth.

You can only shape egress traffic. This means if you want to shape your download, you need to shape it as it leaves your LAN interface, not as it comes into your WAN interface. I like to shape my download so downloads don't make my ping jump high and reduces packet-loss.

moikerz

You've inadvertently answered my question in another thread yesterday ::)

For the sake of completion for this thread, I'll link it here:
https://forum.pfsense.org/index.php?topic=112038.msg623926#msg623926
@Harvy66:

Your LAN interface is set to 1Gb/s. Your traffic is probably going into the default queue of qLink, which is limited to….. 1Gb/s. If you want your traffic to be under your qInternet, you need to place it in there somewhere

P.S. Don't place any traffic directly in qInternet, you're only supposed to place traffic in a leaf queue with HFSC.

moikerz

Here's what I've currently got:

WAN, HFSC, Bandwidth: 5Mbps
- qInternet, CoDel, Bandwidth: 5Mbps
- qNormal, Default, CoDel, Bandwidth: 10%
- qHigh, CoDel, Bandwidth: 20%
- qLow, CoDel, Bandwidth: 5%

LAN1, HFSC, Bandwidth: 900Mbps
- qLink, CoDel, Bandwidth: 895Mbps
- qInternet, CoDel, Bandwidth: 4Mbps, Upperlimit: 4Mbps
- qHigh, CoDel, Bandwidth: 20%
- qNormal, Default, CoDel, Bandwidth: 10%
- qLow, CoDel, Bandwidth: 5%

LAN2, HFSC, Bandwidth: 900Mbps
- qLink, CoDel, Bandwidth: 895Mbps
- qInternet, CoDel, Bandwidth: 1Mbps, Upperlimit: 1Mbps
- qHigh, CoDel, Bandwidth: 20%
- qNormal, Default, CoDel, Bandwidth: 10%
- qLow, CoDel, Bandwidth: 5%

And my classification rules (attached).

This give me:

5Mbps max upload
4Mbps max download for LAN1
1Mbps max download for LAN2
qHigh traffic can use 100% if available, always guaranteed 20% of parent
qNormal traffic can use 100% if available, always guaranteed 10% of parent
qLow traffic can always use 100% if available, always guaranteed 5% of parent
All traffic defaults to qNormal
email ports are low priority (qLow, saves approx 10% of bandwidth)
DNS, private cloud and OpenVPN is high priority (qHigh)
Internal-to-Internal traffic is assigned to qLink, approx 900Mbps speed

Still to do:

push Pandora, Spotify traffic into qLow
push Skype, Hangouts into qHigh
consider making the defaults qLow, and prioritize back to qNormal

floating.jpg_thumb

Harvy66

@moikerz:

consider making the defaults qLow, and prioritize back to qNormal

That's what I did.

teladero

@Harvy66:

@moikerz:

consider making the defaults qLow, and prioritize back to qNormal

That's what I did.

Can you please show us your layout? I also have two LANs that I would do this on.

Is Codel what you would use?

Harvy66

I've posted many times int he past what my setup is. I'd have to go over my message history to find it.

I use Codel as a sub-discipline and HFSC as the shaper.

KOM

It would be great if Harvy, Nullty or sideout could draft a quickie guide to pfSense HFSC and provide some basic examples for common use cases. The pfSense Book is weak on HFSC, there hasn't been a Hangout on it, and every day there is someone new trying to wrap his head around it. I'd even cough in a few bucks if it was bounty-worthy.

Nullity

@KOM:

It would be great if Harvy, Nullty or sideout could draft a quickie guide to pfSense HFSC and provide some basic examples for common use cases. The pfSense Book is weak on HFSC, there hasn't been a Hangout on it, and every day there is someone new trying to wrap his head around it. I'd even cough in a few bucks if it was bounty-worthy.

I have tried to encourage myself to do precisely that, but the combination of documentation being so unrewarding mixed with HFSC being beyond my full comprehension makes the task very daunting.

Defining "common use cases" might be a good beginning.

KOM

Common HFSC Use Cases

1 WAN / 1 LAN - [LAN: VoIP phones, ACK, DNS, WWW]
1 WAN / 2 LAN - [LAN: VoIP phones, ACK, DNS, WWW] * [DMZ: WWW, MAIL]
2 WAN / 1 LAN - [LAN: VoIP phones, ACK, DNS, WWW]
2 WAN / 2 LAN - [LAN1: VoIP phones, ACK, DNS, WWW] * [LAN2: VoIP phones, ACK, DNS, WWW]
2 WAN / 3 LAN - [LAN1: VoIP phones, ACK, DNS, WWW] * [LAN2: VoIP phones, ACK, DNS, WWW] * [DMZ: WWW, MAIL]
Per-client shaping
VPN shaping

MaxPF

@Harvy66:

I've posted many times int he past what my setup is. I'd have to go over my message history to find it.

I use Codel as a sub-discipline and HFSC as the shaper.

Do you recommend setting the priority (0-7) in the child queues? Also if using Codel as sub-discipline, do you still check the "Explicit congestion notification" option?
Thanks.

moikerz

Sooo … my traffic shaper config for this thread is working well. EXCEPT for Windows10 updates. Holy moly, that brings my internet access to a crawl/stop. Yes, I have Windows10 configured to only get updates from Microsoft and local LAN.

But, when Win10 updates, the data ignores my HFSC rules and uses 100% of my bandwidth. Everything else obeys the rules, except these updates.

For those new to this thread, my rules are:

WAN, HFSC, Bandwidth: 5Mbps
- qInternet, CoDel, Bandwidth: 5Mbps
- qNormal, Default, CoDel, Bandwidth: 10%
- qHigh, CoDel, Bandwidth: 20%
- qLow, CoDel, Bandwidth: 5%

LAN1, HFSC, Bandwidth: 900Mbps
- qLink, CoDel, Bandwidth: 895Mbps
- qInternet, CoDel, Bandwidth: 4Mbps, Upperlimit: 4Mbps
- qHigh, CoDel, Bandwidth: 20%
- qNormal, Default, CoDel, Bandwidth: 10%
- qLow, CoDel, Bandwidth: 5%

LAN2, HFSC, Bandwidth: 900Mbps
- qLink, CoDel, Bandwidth: 895Mbps
- qInternet, CoDel, Bandwidth: 1Mbps, Upperlimit: 1Mbps
- qHigh, CoDel, Bandwidth: 20%
- qNormal, Default, CoDel, Bandwidth: 10%
- qLow, CoDel, Bandwidth: 5%

Refer to LAN1 above, how can traffic from LAN1 to WAN possibly take 6Mbps (my total bandwidth) when WAN is configured as 5Mbps and LAN is configured as 4Mbps?? ???

Derelict

When Windows 10 is updating and saturating LAN (I know it's saturating WAN downloads but that is regulated by shaping LAN out), what queue is the traffic in? qLink?