Limiting and shaping are two different things.  With a limiter, it is a hard cap regardless of maximum bandwidth.  With a shaper, low-priority queues will get to use all bandwidth until something more important comes along.  This is a simplified explanation of how it really works.

Thanks Deric. Your "1:79, 81:65535" suggestion is what I was looking for.

It's basically the same problem as in here…
https://forum.pfsense.org/index.php?topic=91299.0

Sam

My next venture is to learn more about debugging ALTQ so I can see more stats. I am assuming I will need to resort to a full FreeBSD to get more control of the system though. Learning more about FreeBSD is probably a good idea.

I bet that the bufferbloat/Codel mailing-list has some great info about how to simulate traffic for testing.

I have only played with netperf casually. The few worthwhile tests I ran I did with either multiple sessions of iperf, or multiple sessions of netcat/nc. Netcat is insanely powerful. Every few months since I learned of netcat I am amazed by how powerful the simple tool can be; "netcat host2 port < /dev/zero" to a listening netcat and bam, single stream TCP/UDP traffic. or send a file with the same method. or create a remote shell. That is just the standard uses.

There is gns3, but it is Cisco emulation primarily. Good userbase and polished interface.

There must be a good, simple simulator, right? Call DARPA maybe? ;)

The wizard attempts to make sure that VoIP traffic goes into qVOIP by directing your SIP server traffic to it, but it doesn't do it by DPI or any magic – just it's address.  If you already have your phones segmented off by themselves then it will be stupid simple.  Create an alias for your phone subnet, then modify the floating rule that directs the VoIP traffic and change the source to be your alias.  Done.  All traffic coming from your phone subnet will be shunted to qVOIP.

Keep sniffing while the queue is active and see which ports are triggering the rule.

1.  Traffic shaping forum is specifically for issues involving the traffic shaper.

2.  You likely had an environmental issue with your line after the storm.  I've seen cases of excess water shorting out a connection that fixed itself with the help of evaporation.

You could also try to make a tunnel specifically for the IPTV traffic –- Then you could shape traffic based upon which tunnel the traffic was received from.

@KOM:

I wonder how that TBR interacts with the PRIQ caveat that high priority queues will starve low priority queues of bandwidth if there is enough high traffic?  I believed PRIQ was absolute: higher priority goes first no matter what.

That is my understanding as well. The only thing that has changed for me is what controls the bitrate.

I had always wondered how PRIQ could have no rate-limiting, since it is practically a necessity for QoS.

Yes SamTzu is the same issue and here we don't run Squid…And i think is a generic issue but i believe still not fixed

LOL  ;) I should have mention it, I upgraded to 2.2.1 after my first post, delete the existing traffic shaper rules, and re created them

@Nullity:

@Harvy66:

Don't forget to set the bandwidth on the interface. You should set it to a value slightly less than your actual upload speed. If you get 5Mb to say speedtest, then set your  upload to 4.9Mb.

It also depends on the stability of your upload. If it fluctuates a lot, you may want to lower if further. In theory, you should set your interface to be less than your lowest speed, but that is not always practical.

The emboldened section of the quote is very important. If pfSense is not the "bottle-neck" of throughput, then traffic-shaping does not work. You might try setting qACK to ~500Kb and set other (unimportant) queues to 1Kb. Leave m1/d/m2 alone, for now.
Perhaps try check-marking the "Codel Active Queue" box.

Regardless of the algorithm you choose (HFSC, CBQ, PRIQ, etc), you need to follow standard QoS practices and understand a good amount of networking basics. I quickly found out that "easy" and "QoS" rarely coincide, sadly.

<- What he said  ;D

There is no easy money in this… but this golden link helped me with basic understanding of QoS:

http://www.linksysinfo.org/index.php?threads/using-qos-tutorial-and-discussion.28349/

If you read all of it... you can go further.
I also thought that using basic pFSense wizard would get me covered but...

State table uses 10% of RAM by default, but you can modify this as per SamTzu.

https://doc.pfsense.org/index.php/How_can_I_increase_the_state_table_size

Thanks for the reply guys.

Tried simple limiters?

Drops are a good thing to signal the sender to back off, but drops are not good if they're too aggressive. Codel just makes it simple. It can be useful on the LAN side, but less useful. Especially since download tends to be much faster than upload for most people.

Shaping downloads as worked decently well for me, but not nearly as well as upload. Upload is a 1Gb link going into a 100Mb link, so going over 100Mb is not an issue because the shaper will buffer the data and not affect the other queues. The problem with download is it's a 100Mb link going into a 1Gb link, so I need to make sure my download never goes above 100Mb because it will buffer upstream instead of in my traffic shaper.

For the most part this just means I can't use a tight 98% of my link speed for download, it needs to be a bit looser, like 95%, which wastes more bandwidth. One issue that I have found out is because PFSense is stateful, when the WAN interface see duplicate TCP packets, it just drops the packet and sends a dup-ack. Since this Dup packet never makes it to the LAN interface, the LAN thinks there is less than 100Mb coming in, so it doesn't cause the other traffic to back-off. This is not an "issue" with PFSense, but an issue with bad-actors.

Well, I just tested my theory and it failed. HFSC needs to be configured as the throughput bottle-neck with the root queue's "Bandwidth" parameter.

I will be testing more scenerios, like perhaps moving the actual link's queue from the root (root queue cannot change it's bandwidth dynamically, apparently) to an interior leaf queue (a leaf's bandwidth is dynamic). I will report back if find anything new. How the hierarchy of the queues is laid out is actually important, so I have some hope it may work. :)

It may be interesting to note that HFSC was not originally implemented with an "upper-limit".
https://github.com/freebsd/freebsd/blob/428b45aa532260e8c6ddf0217ec31db2234d29a8/sys/contrib/altq/altq/altq_hfsc.c#L39

Anyway, here are my results:

WAN Bandwidth set to 640Kb. (Less than real-world throughput of 666Kb)

pfTop: Up Queue 1-6/6, View: queue, Cache: 10000                        16:55:57 QUEUE              BW SCH  PR  PKTS BYTES DROP_P DROP_B QLEN BORR SUSP P/S  B/S root_pppoe0      640K hfsc  0    0    0      0      0    0            0    0 qBulk            1000 hfsc      214 76697      0      0    1            1  695 qACK            500K hfsc    1697 94153      0      0    0            3  211 qNTP            25000 hfsc        0    0      0      0    0            0    0 qTEST1          **1000** hfsc    1285 1500K    19  7521  38            9  **11K** qTEST2          **6000** hfsc    3067 3792K      0      0  35            52  **67K**

You can see that the ratios between the 2 test queues are almost exactly 1:6.
These are the speeds recorded in my ftp client, "lftp", during the above test. The speeds are not pasted exactly as I observed, because I had to freeze the terminal with Ctrl-S.
tmp/linux-3.18.5.tar.xz.1' at 5651256 (11%) **63.2K/s** eta:11m [Sending data] progit.en.pdf' at 3180520 (56%) 11.0K/s eta:2m [Sending data]

WAN Bandwidth set to 250Mb.

pfTop: Up Queue 1-6/6, View: queue, Cache: 10000                        17:01:25 QUEUE              BW SCH  PR  PKTS BYTES DROP_P DROP_B QLEN BORR SUSP P/S  B/S root_pppoe0      250M hfsc  0    0    0      0      0    0            0    0 qBulk            1000 hfsc      14  2667      0      0    0            0    0 qACK            500K hfsc      34  1874      0      0    0            0    0 qNTP            25000 hfsc        0    0      0      0    0            0    0 qTEST1          **1000** hfsc    1592 1921K      0      0    0            27  **34K** qTEST2          **6000** hfsc    1551 1940K      0      0    0            35  **46K**

As you can see, this test did not function as I expected.

Edit: "Bandwidth" and link-share's "m2" are set to the same value.