HFSC/CoDel for 40 devices
-
I understand about have LAN1 and LAN2 Bandwidths set as my desired download speed.
But what does that mean for LAN1-qLink and LAN2-qLink? Since they are my default queues for LAN, it sounds like I should remove the qLink parents completely, and make new qDefaults as children of qInternet, correct? (With the obvious side-effect of limiting inter-vlan comms to whatever download speed I configure for the LANx parent)
WAN, HFSC, Bandwidth: 5Mbps
- qInternet, CoDel, Bandwidth: 5Mbps
- qDefault, Default, CoDel, Bandwidth: 10%
- qOthersHigh, CoDel, Bandwidth: 20%
- qOthersLow, CoDel, Bandwidth: 5%LAN1, HFSC, Bandwidth: 4Mbps
- qLink, Default, CoDel, Bandwidth: ???Mbps
- qInternet, CoDel, Bandwidth: 4Mbps, Upperlimit: 4Mbps
- qOthersHigh, CoDel, Bandwidth: 10%
- qOthersLow, CoDel, Bandwidth: 5%LAN2, HFSC, Bandwidth: 1Mbps
- qLink, Default, CoDel, Bandwidth: ???Mbps
- qInternet, Bandwidth: 1Mbps, Upperlimit: 1Mbps
- qOthersHigh, CoDel, Bandwidth: 10%
- qOthersLow, CoDel, Bandwidth: 5% -
qLink is meant to be used for non-WAN related traffic, like inter-LAN or between PFSense and the LANs. That way all of that traffic can run full LAN speed and not affect the WAN traffic coming in.
-
Yes that's what I understood qLink to be for too :D But I'm not understanding the purpose of setting LAN bandwidth. Does the bandwidth of the parent queues (qLink, qInternet) need to be equal/less than the interface bandwidth?
With the config I posted, is the correct approach (see bold):
LAN bandwidth = 900Mbps
- qLink bandwidth = 896Mbps
- qInternet bandwidth = 4Mpbs
- qOthersHigh, CoDel, Bandwidth: 10%
- qOthersLow, CoDel, Bandwidth: 5%
??It seems like that should be correct, from how I interpret what you're saying.
-
But I'm not understanding the purpose of setting LAN bandwidth.
You can only shape egress traffic. This means if you want to shape your download, you need to shape it as it leaves your LAN interface, not as it comes into your WAN interface. I like to shape my download so downloads don't make my ping jump high and reduces packet-loss.
-
You've inadvertently answered my question in another thread yesterday ::)
For the sake of completion for this thread, I'll link it here:
https://forum.pfsense.org/index.php?topic=112038.msg623926#msg623926
@Harvy66:Your LAN interface is set to 1Gb/s. Your traffic is probably going into the default queue of qLink, which is limited to….. 1Gb/s. If you want your traffic to be under your qInternet, you need to place it in there somewhere
P.S. Don't place any traffic directly in qInternet, you're only supposed to place traffic in a leaf queue with HFSC.
-
Here's what I've currently got:
WAN, HFSC, Bandwidth: 5Mbps
- qInternet, CoDel, Bandwidth: 5Mbps
- qNormal, Default, CoDel, Bandwidth: 10%
- qHigh, CoDel, Bandwidth: 20%
- qLow, CoDel, Bandwidth: 5%LAN1, HFSC, Bandwidth: 900Mbps
- qLink, CoDel, Bandwidth: 895Mbps
- qInternet, CoDel, Bandwidth: 4Mbps, Upperlimit: 4Mbps
- qHigh, CoDel, Bandwidth: 20%
- qNormal, Default, CoDel, Bandwidth: 10%
- qLow, CoDel, Bandwidth: 5%LAN2, HFSC, Bandwidth: 900Mbps
- qLink, CoDel, Bandwidth: 895Mbps
- qInternet, CoDel, Bandwidth: 1Mbps, Upperlimit: 1Mbps
- qHigh, CoDel, Bandwidth: 20%
- qNormal, Default, CoDel, Bandwidth: 10%
- qLow, CoDel, Bandwidth: 5%And my classification rules (attached).
This give me:
- 5Mbps max upload
- 4Mbps max download for LAN1
- 1Mbps max download for LAN2
- qHigh traffic can use 100% if available, always guaranteed 20% of parent
- qNormal traffic can use 100% if available, always guaranteed 10% of parent
- qLow traffic can always use 100% if available, always guaranteed 5% of parent
- All traffic defaults to qNormal
- email ports are low priority (qLow, saves approx 10% of bandwidth)
- DNS, private cloud and OpenVPN is high priority (qHigh)
- Internal-to-Internal traffic is assigned to qLink, approx 900Mbps speed
Still to do:
- push Pandora, Spotify traffic into qLow
- push Skype, Hangouts into qHigh
- consider making the defaults qLow, and prioritize back to qNormal
-
-
-
I've posted many times int he past what my setup is. I'd have to go over my message history to find it.
I use Codel as a sub-discipline and HFSC as the shaper.
-
It would be great if Harvy, Nullty or sideout could draft a quickie guide to pfSense HFSC and provide some basic examples for common use cases. The pfSense Book is weak on HFSC, there hasn't been a Hangout on it, and every day there is someone new trying to wrap his head around it. I'd even cough in a few bucks if it was bounty-worthy.
-
@KOM:
It would be great if Harvy, Nullty or sideout could draft a quickie guide to pfSense HFSC and provide some basic examples for common use cases. The pfSense Book is weak on HFSC, there hasn't been a Hangout on it, and every day there is someone new trying to wrap his head around it. I'd even cough in a few bucks if it was bounty-worthy.
I have tried to encourage myself to do precisely that, but the combination of documentation being so unrewarding mixed with HFSC being beyond my full comprehension makes the task very daunting.
Defining "common use cases" might be a good beginning.
-
Common HFSC Use Cases
-
1 WAN / 1 LAN - [LAN: VoIP phones, ACK, DNS, WWW]
-
1 WAN / 2 LAN - [LAN: VoIP phones, ACK, DNS, WWW] * [DMZ: WWW, MAIL]
-
2 WAN / 1 LAN - [LAN: VoIP phones, ACK, DNS, WWW]
-
2 WAN / 2 LAN - [LAN1: VoIP phones, ACK, DNS, WWW] * [LAN2: VoIP phones, ACK, DNS, WWW]
-
2 WAN / 3 LAN - [LAN1: VoIP phones, ACK, DNS, WWW] * [LAN2: VoIP phones, ACK, DNS, WWW] * [DMZ: WWW, MAIL]
-
Per-client shaping
-
VPN shaping
-
-
I've posted many times int he past what my setup is. I'd have to go over my message history to find it.
I use Codel as a sub-discipline and HFSC as the shaper.
Do you recommend setting the priority (0-7) in the child queues? Also if using Codel as sub-discipline, do you still check the "Explicit congestion notification" option?
Thanks. -
Sooo … my traffic shaper config for this thread is working well. EXCEPT for Windows10 updates. Holy moly, that brings my internet access to a crawl/stop. Yes, I have Windows10 configured to only get updates from Microsoft and local LAN.
But, when Win10 updates, the data ignores my HFSC rules and uses 100% of my bandwidth. Everything else obeys the rules, except these updates.
For those new to this thread, my rules are:
WAN, HFSC, Bandwidth: 5Mbps
- qInternet, CoDel, Bandwidth: 5Mbps
- qNormal, Default, CoDel, Bandwidth: 10%
- qHigh, CoDel, Bandwidth: 20%
- qLow, CoDel, Bandwidth: 5%LAN1, HFSC, Bandwidth: 900Mbps
- qLink, CoDel, Bandwidth: 895Mbps
- qInternet, CoDel, Bandwidth: 4Mbps, Upperlimit: 4Mbps
- qHigh, CoDel, Bandwidth: 20%
- qNormal, Default, CoDel, Bandwidth: 10%
- qLow, CoDel, Bandwidth: 5%LAN2, HFSC, Bandwidth: 900Mbps
- qLink, CoDel, Bandwidth: 895Mbps
- qInternet, CoDel, Bandwidth: 1Mbps, Upperlimit: 1Mbps
- qHigh, CoDel, Bandwidth: 20%
- qNormal, Default, CoDel, Bandwidth: 10%
- qLow, CoDel, Bandwidth: 5%Refer to LAN1 above, how can traffic from LAN1 to WAN possibly take 6Mbps (my total bandwidth) when WAN is configured as 5Mbps and LAN is configured as 4Mbps?? ???
-
When Windows 10 is updating and saturating LAN (I know it's saturating WAN downloads but that is regulated by shaping LAN out), what queue is the traffic in? qLink?
-
I thought it would be on qLink also.. but no, it's on qNormal.
In an interesting turn of events, when I monitor the network usage on that machine itself, it says it's capping at 4Mbps, which is how pfSense is configured. But the network graphs - and confirmed by the amount of complaints I get - show the bandwidth is at 100% (6Mbps) instead of my HFSC at 4Mbps (see attached graph).
Attached are my floating rules as well.
Any ideas? I'm stumped ???
-
Are you using squid? FreeBSD does not shape incoming bandwidth, which means squid cannot shape incoming, only outgoing.
-
@KOM:
Common HFSC Use Cases
-
1 WAN / 1 LAN - [LAN: VoIP phones, ACK, DNS, WWW]
-
1 WAN / 2 LAN - [LAN: VoIP phones, ACK, DNS, WWW] * [DMZ: WWW, MAIL]
-
2 WAN / 1 LAN - [LAN: VoIP phones, ACK, DNS, WWW]
-
2 WAN / 2 LAN - [LAN1: VoIP phones, ACK, DNS, WWW] * [LAN2: VoIP phones, ACK, DNS, WWW]
-
2 WAN / 3 LAN - [LAN1: VoIP phones, ACK, DNS, WWW] * [LAN2: VoIP phones, ACK, DNS, WWW] * [DMZ: WWW, MAIL]
-
Per-client shaping
-
VPN shaping
I would be willing to add to the bounty on these scenarios too. We could then add it to the pfsense handbook.
-
-
Nope, not using Squid.
-
I have been trying to implement HFSC/Codel in a >100 node environment with VOIP. After four months and many weeks on the forum going through years old information and reading HFSC papers from the late 1990s….I have managed to circle back to the beginning. This has been a monumental task. Every time I feel the setup is good the stats show otherwise. After a few weeks there will be notable drops in all the wrong queues. Something on the order of >50% drops of the default queue happening in the top two queues that should have a network bandwidth priority, as noted by the assignments.
Been trying to find the time to come on the forum and share some of my findings, so hopefully my recent experience can help shed some light on trying to implement this. Also a big thanks to Harvy66, Nullity, and the other forum members for their many posts on traffic shaping. One of the problems is that after making changes, it can take two or three weeks to collect enough information to see reliable results. A couple rounds of mediocre changes and a month has passed with nearly no progress in shaping traffic. (or negative progress, or just ending up cutting 20% off the internet bandwidth while trying to take control of the other 80%)
The big bugbear I feel happens when there are multiple LANs. Everything looks good on paper and during testing after business hours but the true running environment shreds the queue right up, every time. Whether it is because of multiple devices pulling down updates or groups of users eating bandwidth from the internet, it seems like any real stress across all LANs/queues together will immediately start throwing drops in higher bandwidth queues instead of cutting bandwidth out of the default/lower bandwidth queues. This is easily noticed when running a VOIP queue, that almost always has multiple bandwidth streams running, and will immediately start logging drops.
Tried setting M2 levels way below actual limits, for example setting qDefault to 64Kb on a parent queue that is good to 20Mbps, and then setting notably higher limits on the qVOIP and qACK in the order of 768Kb. That's 12 times the bandwidth limit of qDefault. To my understanding this -should- result in the qDefault dropping significantly more information than the other queues, but the results show otherwise. For example consider the 20Mbps interface with all child queue M2 combined equaling only 3Mbps, there is considerable bandwidth available to link share here. (and priority should be granted based on the M2 bandwidth allocation)
Tried removing most of the queues and just using two or three, this didn't seem to help. Then tried creating seven or eight queues, and this didn't help either. One benefit of more queues is that I can see exactly which queues are dropping information and know exactly which services are dropping packets. Also can see in pftop how much data passes through those queues, which can help when looking at dropped packets per gigabyte of data.
Just started using the d parameter (delay) so the plan now is to put a delay into the ramp up on bandwidth available for the queue. The examples seem to use this for bursting data but I'd like to try using it to slowly ramp up bandwidth allocation. The other solution is to assign a real time queue, this works until there is more than one "very important" stream to operate the network. My goal was to implement this on a fully link share setup with no real time queues. That is all I have for now, just tossed out a stack of diagnostics>pftop>queue printouts from the last couple of months....where I thought everything was good to go! The best advice I can share right now is don't believe the shaper is okay until a few weeks of data has run through it. ;D
