Pfctl: the sum of child bandwidth higher than parent
-
HFSC traffic shaping has worked very well for me for the most part. I use it on two pfSense boxes and it beautifully keeps the VOIP traffic flowing etc. Now, on one of the pfSense installs I've added a Wifi card (miniPCIe WLE200NX with Atheros chipset), for guest Wifi access. Rules were added to keep the traffic from accessing the regular LAN, works like a charm.
I then tried to limit the bandwidth of the guest Wifi so guests couldn't eat up too much of the available bandwidth, and that's when things went wrong. No matter what I tried, it always results in the message "pfctl: the sum of child bandwidth higher than parent", even though I am certain the sum is NOT larger than the parent bandwidth. This is the traffic shaper setup:
<shaper><queue><interface>wan</interface> <name>wan</name> <scheduler>HFSC</scheduler> <bandwidth>1</bandwidth> <bandwidthtype>Gb</bandwidthtype> <qlimit>400</qlimit> <queue><name>qInternet</name> <interface>wan</interface> <qlimit>200</qlimit> <bandwidth>0.9</bandwidth> <bandwidthtype>Mb</bandwidthtype> <enabled>on</enabled> <queue><name>qACK</name> <interface>wan</interface> <qlimit>200</qlimit> <priority>7</priority> <bandwidth>40</bandwidth> <bandwidthtype>%</bandwidthtype> <enabled>on</enabled> <linkshare3>40%</linkshare3> <realtime3>40%</realtime3> <linkshare>on</linkshare> <realtime>on</realtime></queue> <queue><name>qVOIP</name> <interface>wan</interface> <qlimit>200</qlimit> <priority>6</priority> <bandwidth>18</bandwidth> <bandwidthtype>%</bandwidthtype> <enabled>on</enabled> <linkshare3>10%</linkshare3> <realtime3>100Kb</realtime3> <linkshare>on</linkshare> <realtime>on</realtime></queue> <queue><name>qHigh</name> <interface>wan</interface> <qlimit>200</qlimit> <priority>5</priority> <bandwidth>30</bandwidth> <bandwidthtype>%</bandwidthtype> <enabled>on</enabled> <linkshare3>30%</linkshare3> <linkshare>on</linkshare></queue> <queue><name>qDefault</name> <interface>wan</interface> <qlimit>200</qlimit> <priority>3</priority> <bandwidth>15</bandwidth> <bandwidthtype>%</bandwidthtype> <enabled>on</enabled> <default>default</default> <linkshare3>15%</linkshare3> <linkshare>on</linkshare></queue> <queue><name>qLow</name> <interface>wan</interface> <qlimit>200</qlimit> <priority>2</priority> <bandwidth>5</bandwidth> <bandwidthtype>%</bandwidthtype> <enabled>on</enabled> <linkshare3>5%</linkshare3> <linkshare>on</linkshare> <upperlimit3>85%</upperlimit3> <upperlimit>on</upperlimit></queue></queue> <queue><name>qLink</name> <interface>wan</interface> <qlimit>400</qlimit> <bandwidth>100</bandwidth> <bandwidthtype>Mb</bandwidthtype> <enabled>on</enabled></queue> <enabled>on</enabled></queue> <queue><interface>lan</interface> <name>lan</name> <scheduler>HFSC</scheduler> <bandwidth>1</bandwidth> <bandwidthtype>Gb</bandwidthtype> <qlimit>400</qlimit> <queue><name>qInternet</name> <interface>lan</interface> <qlimit>400</qlimit> <bandwidth>7.6</bandwidth> <bandwidthtype>Mb</bandwidthtype> <enabled>on</enabled> <queue><name>qACK</name> <interface>lan</interface> <qlimit>400</qlimit> <priority>7</priority> <bandwidth>10</bandwidth> <bandwidthtype>%</bandwidthtype> <enabled>on</enabled> <linkshare3>10%</linkshare3> <linkshare>on</linkshare> <realtime3>10%</realtime3> <realtime>on</realtime></queue> <queue><name>qVOIP</name> <interface>lan</interface> <qlimit>400</qlimit> <priority>6</priority> <bandwidth>10</bandwidth> <bandwidthtype>%</bandwidthtype> <enabled>on</enabled> <realtime3>800Kb</realtime3> <linkshare3>10%</linkshare3> <linkshare>on</linkshare> <realtime>on</realtime></queue> <queue><name>qHigh</name> <interface>lan</interface> <priority>5</priority> <bandwidth>40</bandwidth> <bandwidthtype>%</bandwidthtype> <enabled>on</enabled> <linkshare3>40%</linkshare3> <qlimit>400</qlimit> <linkshare>on</linkshare></queue> <queue><name>qDefault</name> <interface>lan</interface> <qlimit>400</qlimit> <priority>3</priority> <bandwidth>35</bandwidth> <bandwidthtype>%</bandwidthtype> <enabled>on</enabled> <default>default</default> <linkshare3>35%</linkshare3> <linkshare>on</linkshare></queue> <queue><name>qLow</name> <interface>lan</interface> <qlimit>400</qlimit> <priority>2</priority> <bandwidth>5</bandwidth> <bandwidthtype>%</bandwidthtype> <enabled>on</enabled> <linkshare3>5%</linkshare3> <upperlimit3>85%</upperlimit3> <linkshare>on</linkshare> <upperlimit>on</upperlimit></queue></queue> <queue><name>qLink</name> <interface>lan</interface> <qlimit>400</qlimit> <bandwidth>100</bandwidth> <bandwidthtype>Mb</bandwidthtype> <enabled>on</enabled></queue> <enabled>on</enabled></queue> <queue><interface>opt1</interface> <name>opt1</name> <scheduler>HFSC</scheduler> <bandwidth>1</bandwidth> <bandwidthtype>Gb</bandwidthtype> <qlimit>400</qlimit> <enabled>on</enabled> <queue><name>qInternet</name> <interface>opt1</interface> <qlimit>200</qlimit> <bandwidth>1</bandwidth> <bandwidthtype>Mb</bandwidthtype> <enabled>on</enabled> <queue><name>qACK</name> <interface>opt1</interface> <bandwidth>10</bandwidth> <bandwidthtype>%</bandwidthtype> <enabled>on</enabled> <qlimit>200</qlimit> <priority>7</priority> <linkshare3>10%</linkshare3> <realtime3>10%</realtime3> <linkshare>on</linkshare> <realtime>on</realtime></queue> <queue><name>qVOIP</name> <interface>opt1</interface> <bandwidth>10</bandwidth> <bandwidthtype>%</bandwidthtype> <enabled>on</enabled> <qlimit>200</qlimit> <priority>6</priority> <linkshare3>10%</linkshare3> <linkshare>on</linkshare></queue> <queue><name>qHigh</name> <interface>opt1</interface> <bandwidth>40</bandwidth> <bandwidthtype>%</bandwidthtype> <enabled>on</enabled> <qlimit>200</qlimit> <priority>5</priority> <linkshare3>40%</linkshare3> <linkshare>on</linkshare></queue> <queue><name>qDefault</name> <interface>opt1</interface> <bandwidth>35</bandwidth> <bandwidthtype>%</bandwidthtype> <enabled>on</enabled> <default>default</default> <qlimit>200</qlimit> <priority>3</priority> <linkshare3>35%</linkshare3> <linkshare>on</linkshare></queue> <queue><name>qLow</name> <interface>opt1</interface> <bandwidth>5</bandwidth> <bandwidthtype>%</bandwidthtype> <enabled>on</enabled> <qlimit>200</qlimit> <priority>2</priority> <linkshare3>5%</linkshare3> <linkshare>on</linkshare> <upperlimit3>85%</upperlimit3> <upperlimit>on</upperlimit></queue></queue> <queue><name>qLink</name> <interface>opt1</interface> <qlimit>400</qlimit> <bandwidth>100</bandwidth> <bandwidthtype>Mb</bandwidthtype> <enabled>on</enabled></queue></queue></shaper>The problem is with the OPT1 queue; disabling that makes it work fine (except that there is no traffic shaping on that OPT1/Wifi interface). The percentages add up to 100%, there is no absolute bandwidth used in the child queues, just percentages (I tried topping out at 90% total for all queues, same problem).
Does anyone have any idea of what may be causing this?
Thank you for your help!-Rob-
-
I would simplify. Do you really need so many queues?
I played with HFSC a lot and found that pfctl was always right and I was confused… though, I know that's not very helpful.
Personally, I think percentages should be avoided since they imply a lack of concern for precision, which most likely means you should just delete that queue and let the traffic do as it pleases. Focus on traffic that needs prioritization and default everything else, unless you have a good reason to do otherwise.
Also remove the 0-7 numeric priority, if you are using it. HFSC has no such parameter.
Ultimately, simplify.
-
Nullity, thank you for getting back!
Agreed! Simplify is always a good strategy!
With that, I set the OPT1 shaper with just a single child queue, and set the child at 80%. The moment that queue is enabled I get the same error message. I tried leaving the bandwidth of the parent OPT1 queue open (so it reads it off the interface), tried some ridiculously low bandwidth of 1kbps, same error message. The short of it is that any attempt to create a queue and enable it on the Wifi interface throws the error the moment it is enabled. Disable that interface or queue and the shaper purrs along just fine on WAN/LAN.Like you I have spent quite a bit of time divining the ins and outs of HFSC, heck, I even read the original scientific publication that proposed it. It's been working very well for me on two pfSense boxes for the last year-and-a-half, and like you I've only seen this error to mean what it says.
In this particular case I think there may be more to it. Possibly a bug? Thing is that a Wifi card does not have a "link speed" when you check it with ifconfig. I wonder if that is at the root of this (though I have tried explicitly setting a speed for the interface in the HFSC settings, and pfSense has not complained in the past when I set an interface to 1 Gbps even if the link speed is 100 Mbps).
Does anyone have traffic shaping working, preferably HFSC, on a Wifi card?
-Rob-
-
<realtime3>10%</realtime3>
Realtime always comes from the root, that means you assigned it 100Mb of realtime since your root is 1Gb, which is 50% of your qInternet. Your totals are too high.
/guessing
P.S. I don't use any realtime because it just makes things more confusing. I can still maintain sub 1ms pings under load.
-
Harvy, good point!
I actually had those as absolute numbers, and changed them to percentages while troubleshooting this problem. I've put them back to absolute numbers since. The realtime values are not causing this problem though:The WAN and LAN queues hum along nicely with them. It's when I add a queue for OPT1/Wifi that things go wrong, and I've tried the most basic setup for that; just a "default" queue. Any queue, any value for the bandwidth of OPT1, and using just a percentage (80%) as the one and only (default) queue on OPT1 will set off the "sum of child bandwidth …" message.
Regarding your point about the need for realtime queue values: You're probably right. But then again this is available in HFSC and it gives both speed and latency guarantees, and the realtime queues get processed before the linkshare queues. Those queues are never large enough to encroach on the overall bandwidth limits, so they actually do get processed completely before anything else. For ACK and VOIP queues it makes sense, and it works beautifully for another pfSense system I run, that is a business site with fairly limited overall bandwidth and an Asterisk server (VOIP) behind the firewall. No matter how filled-up the pipe is, the VOIP phone calls are always perfect.
-Rob-
-
pfctl related code is mostly (completely?) unmodified from FreeBSD, so look to the FreeBSD man-pages/forums for a better answer.