The HFSC wizard gives about 50% of the download speed I expect.
-
Hi,
As I'm trying to learn how traffic shaping works in pfSense, the configuration I get from the wizard has me a bit confused. I'll list my questions first and then explain how I tested.
-
What's the purpose of the qLink queue? I've read that it's intended for multi-LAN setups, but it's limited to the speeds I set for upload download and that doesn't make sense to me. The LAN-to-LAN transfers get limited to those speeds. The only explanation I can think of is that I've misunderstood the upload / download speed settings and I'm supposed to be setting them close to the interface speed instead of my internet speeds.
-
Why does my super simple, wizard base config limit download speeds to half the speed I expect? This is my main question for now and I describe how I set up and tested below.
-
Why does the wizard bump my WAN download speed to 51200Kbit/s when I enter 50000Kbit/s?
-
Why does the wizard include outbound in the floating rules description when the direction is set to all? Based on my testing they match incoming (port forwarded) traffic and outgoing traffic. Did I get it wrong?
-
What are the implications of un-categorized traffic having ACKs going into the default queue? I noted this on one of my listings of queue stats (note qACK isn't used). Is it something I should be trying to avoid?
As for question #2, I've tried to create the simplest 1 WAN, 1 LAN configuration possible. I've done the following:
-
Run the multi-WAN, multi-LAN wizard and select 1 WAN, 1 LAN.
-
Set the WAN to 50000Kbit/s download and 5000Kbit/s upload.
-
Enable Other networking protocols, but don't adjust any of the pre-defined services.
This creates the following queues:
QUEUE BW SCH PRIO root_em0 5000K hfsc 0 qInternet 5000K hfsc qACK 1000K hfsc qDefault 500K hfsc qOthersHigh 500K hfsc qOthersLow 250K hfsc root_em1 51M hfsc 0 qLink 10M hfsc qInternet 40M hfsc qACK 8192K hfsc qOthersHigh 4096K hfsc qOthersLow 2048K hfscNext I create 2 floating firewall rules (in/out aka all directions):
Source Port Destination Port Gateway Queue Description * * * 5201 * qACK/qOthersHigh iperf * * * 5202 * qACK/qOthersLow iperfI also create LAN interface rules to block all traffic except the ports I'm going to use for testing with iperf (5201, 5202, 5203). I run iperf3 in server mode on a machine outside the WAN:
10.10.10.1 (iperf server) -- WAN (pfSense) LAN -- 192.168.70.10 (iperf client)At this point, I run some tests with iperf. The first test transfers data from the client to the server. It should be limited to 5Mbit/s. It looks reasonable to me:
iperf3 --client 10.10.10.1[ ID] Interval Transfer Bandwidth Retr [ 4] 0.00-10.00 sec 5.98 MBytes 5.02 Mbits/sec 141 sender [ 4] 0.00-10.00 sec 5.71 MBytes 4.79 Mbits/sec receiverThe queue stats after the above look like this:
QUEUE BW SCH PRIO PKTS BYTES DROP_P DROP_B QLEN BORROW SUSPEN P/S B/S root_em0 5000K hfsc 0 0 0 0 0 0 0 0 qInternet 5000K hfsc 0 0 0 0 0 0 0 qACK 1000K hfsc 10 660 0 0 0 0 0 qDefault 500K hfsc 1 42 0 0 0 0 0 qOthersHigh 500K hfsc 4155 6276586 141 213474 0 0 0 qOthersLow 250K hfsc 0 0 0 0 0 0 0 root_em1 51M hfsc 0 0 0 0 0 0 0 0 qLink 10M hfsc 21 4050 0 0 0 1.0 149 qInternet 40M hfsc 0 0 0 0 0 0 0 qACK 8192K hfsc 2535 179010 0 0 0 0 0 qOthersHigh 4096K hfsc 6 601 0 0 0 0 0 qOthersLow 2048K hfsc 0 0 0 0 0 0 0Next I run the test in the reverse mode so data is transferred from the server to the client. Based on the Upper Limit of 40Mbit/s that's set on the qInternet queue I expect this test to be limited to 40Mbit/s. However, it's not. It's much slower:
iperf3 --client 10.10.10.1 --reverse[ ID] Interval Transfer Bandwidth [ 4] 0.00-10.00 sec 21.8 MBytes 18.2 Mbits/sec sender [ 4] 0.00-10.00 sec 21.6 MBytes 18.2 Mbits/sec receiverThe queue stats look like this:
QUEUE BW SCH PRIO PKTS BYTES DROP_P DROP_B QLEN BORROW SUSPEN P/S B/S root_em0 5000K hfsc 0 0 0 0 0 0 0 0 qInternet 5000K hfsc 0 0 0 0 0 0 0 qACK 1000K hfsc 12566 900668 0 0 0 0 0 qDefault 500K hfsc 0 0 0 0 0 0 0 qOthersHigh 500K hfsc 10 1079 0 0 0 0 0 qOthersLow 250K hfsc 0 0 0 0 0 0 0 root_em1 51M hfsc 0 0 0 0 0 0 0 0 qLink 10M hfsc 13 3138 0 0 0 1.0 293 qInternet 40M hfsc 0 0 0 0 0 0 0 qACK 8192K hfsc 6 412 0 0 0 0 0 qOthersHigh 4096K hfsc 15766 23277K 536 809547 0 0 0 qOthersLow 2048K hfsc 0 0 0 0 0 0 0When I run the same tests on port 5202 so they get put into the qOthersLow queue the results are almost identical. However, if I run the second test on port 5203 which defaults into the qLink (default) queue, I get results that I expect. There's no other traffic on the link and there's no Upper Limit, so it runs at 50Mbit/s. My SSH connection to pfSense also puts a small amount of traffic into the qLink (default) queue, but it's negligible:
iperf3 --client 10.10.10.1 --port 5203 --reverse[ ID] Interval Transfer Bandwidth [ 4] 0.00-10.00 sec 58.6 MBytes 49.2 Mbits/sec sender [ 4] 0.00-10.00 sec 58.6 MBytes 49.2 Mbits/sec receiverThe queue stats look like this after the above test (note qACK isn't used):
QUEUE BW SCH PRIO PKTS BYTES DROP_P DROP_B QLEN BORROW SUSPEN P/S B/S root_em0 5000K hfsc 0 0 0 0 0 0 0 0 qInternet 5000K hfsc 0 0 0 0 0 0 0 qACK 1000K hfsc 0 0 0 0 0 0 0 qDefault 500K hfsc 20874 1401547 0 0 0 0 0 qOthersHigh 500K hfsc 0 0 0 0 0 0 0 qOthersLow 250K hfsc 0 0 0 0 0 0 0 root_em1 51M hfsc 0 0 0 0 0 0 0 0 qLink 10M hfsc 42524 62774K 40 60560 0 1.0 149 qInternet 40M hfsc 0 0 0 0 0 0 0 qACK 8192K hfsc 0 0 0 0 0 0 0 qOthersHigh 4096K hfsc 0 0 0 0 0 0 0 qOthersLow 2048K hfsc 0 0 0 0 0 0 0If I run my high priority and low priority tests simultaneously, the aggregate bandwidth is still less than I expect.
High priority:
iperf3 --client 10.10.10.1 --reverse[ ID] Interval Transfer Bandwidth [ 4] 0.00-10.00 sec 16.8 MBytes 14.1 Mbits/sec sender [ 4] 0.00-10.00 sec 16.6 MBytes 13.9 Mbits/sec receiverLow priority:
iperf3 --client 10.10.10.1 --port 5202 --reverse[ ID] Interval Transfer Bandwidth [ 4] 0.00-10.00 sec 15.5 MBytes 13.0 Mbits/sec sender [ 4] 0.00-10.00 sec 15.3 MBytes 12.9 Mbits/sec receiverQueue stats:
QUEUE BW SCH PRIO PKTS BYTES DROP_P DROP_B QLEN BORROW SUSPEN P/S B/S root_em0 5000K hfsc 0 0 0 0 0 0 0 0 qInternet 5000K hfsc 0 0 0 0 0 0 0 qACK 1000K hfsc 19659 1419178 0 0 0 0 0 qDefault 500K hfsc 1 42 0 0 0 0 0 qOthersHigh 500K hfsc 11 1133 0 0 0 0 0 qOthersLow 250K hfsc 11 1133 0 0 0 0 0 root_em1 51M hfsc 0 0 0 0 0 0 0 0 qLink 10M hfsc 20 4840 0 0 0 1.0 165 qInternet 40M hfsc 0 0 0 0 0 0 0 qACK 8192K hfsc 14 956 0 0 0 0 0 qOthersHigh 4096K hfsc 12155 17937K 467 705838 0 0 0 qOthersLow 2048K hfsc 11245 16596K 441 667437 0 0 0Are the results I'm seeing are consistent with what I should expect? Should I be seeing 40Mbit/s on the test I think is too slow or am I misunderstanding something about the way the wizard configures things?
-
-
@ryan29:
Hi,
As I'm trying to learn how traffic shaping works in pfSense, the configuration I get from the wizard has me a bit confused. I'll list my questions first and then explain how I tested.
-
What's the purpose of the qLink queue? I've read that it's intended for multi-LAN setups, but it's limited to the speeds I set for upload download and that doesn't make sense to me. The LAN-to-LAN transfers get limited to those speeds. The only explanation I can think of is that I've misunderstood the upload / download speed settings and I'm supposed to be setting them close to the interface speed instead of my internet speeds.
-
Why does my super simple, wizard base config limit download speeds to half the speed I expect? This is my main question for now and I describe how I set up and tested below.
-
Why does the wizard bump my WAN download speed to 51200Kbit/s when I enter 50000Kbit/s?
-
Why does the wizard include outbound in the floating rules description when the direction is set to all? Based on my testing they match incoming (port forwarded) traffic and outgoing traffic. Did I get it wrong?
-
What are the implications of un-categorized traffic having ACKs going into the default queue? I noted this on one of my listings of queue stats (note qACK isn't used). Is it something I should be trying to avoid?
qLink is for multi lan, so you can maintain high speed lan to lan connections while traffic shaping internet. The traffic shaper wizard is insane, and sets things terribly. It conceptually had the right idea and then did it wrong.
Q2: the traffic wizard is insane
Q3: binary math is weird, but it's better that way. Rounds to the nearest binary multiple, in theory.
Q4: the traffic wizard is insane
Q5: putting uncategorized traffic into a prioritized ack queue will increase their total throughput, which may cause situations where non utp bit torrent traffic gets prioritized -
-
That's a lot of dropped packets. Did you size your queues correctly? They default to 50, which is crazy small. I recommend just checking the Codel box under each queue, if you don't know what you're doing.
-
Harvy66, you nailed it. I left the default queue sizes and didn't realize they were too small. I guess pfTop must be averaging QLEN or something because it was rarely hitting double digits. Bumping the queue limit to 500 (since I know qLink works with 500) gives me the results I was expecting. It was the first thing I tried, so you saved me a bunch of time. Thanks.
moscato359, it helps to know the wizard might not be making the best choices. When I'm trying to learn something I usually assume the defaults are well tuned and that if they don't make sense to me it's a misunderstanding on my end. Also, thanks for the explanation regarding my 5th question.