@ermal:

But even if you need 15 you can use th emask functionality to not need to create that many and resuse the same limiter definition.

Didn't find any info about emask, I'd like to keep within standard config which can be set up via GUI so that backup config can stay safe.
If I use one single limiter queue for all the 30 directions (15 in  + 15 out), will they be limited to 2Mbit/sec all, or each (totalling 30Mbit/sec on WAN side max)? (provided that the same limiter is selected for both in and out and on each subnet interface)

Edit: it doesn't let me use the same limiter for in/out, the message is: " In and Out Queue cannot be the same. "
So would it be enough to just create 2 queues like "2mbit_in" and 2mbit_out" and select these on all 15 interfaces? Will these limit at 2Mbit separately or in total?

@Harvy66:

"I have been saying HFSC schedules both inter-queue and intra-queue. If HFSC does no Fair Queueing intra-queue then any flow could saturate a queue."

HFSC does not do anything with flows, it does not do hashing, it doesn't do anything with IP, nothing. All it does is pull the head packet from a child queue and decide which queue goes next. It's a queue scheduler.

Fair queuing, in the context of a queue, fights buffer bloat by isolating flows from each other within the queue.
Fair queuing, in the context of a scheduler, gives a fair amount of resources between queues.

Both HFSC and fq_CoDel do "fair queuing" at different levels.

No. Fair Queueing is exclusively concerned with flows.

https://en.m.wikipedia.org/wiki/Fair_queuing

Fair queuing is a family of scheduling algorithms used in some process and network schedulers. The concept implies a separate data packet queue (or job queue) for each traffic flow (or for each program process) as opposed to the traditional approach with one FIFO queue for all packet flows (or for all process jobs). The purpose is to achieve fairness when a limited resource is shared, for example to avoid that flows with large packets (or processes that generate small jobs) achieve more throughput (or CPU time) than other flows (or processes).

To claim "Fair Queueing", you must separate all flows (or most of the flows, like with SFQ). Above, it says each flow gets a "separate data packet queue", meaning this is automatic and not dependant on the user manually separating the flows like your "pseudo fair-queueing" setup. HFSC is a Fair Queueing algo therefore it separates all flows, by definition.

HFSC cites many other Fair Queueing algorithms including one paper which all modern Fair Queueing algorithms attempt to approximate as closely as possible, and it is titled "A generalized processor sharing approach to flow control in integrated services networks".

For the sake of clarity, the definition of a "flow" can be found here: https://en.m.wikipedia.org/wiki/Traffic_flow_(computer_networking)

Do me a favor and read the Generalized Processor Sharing paper (or even just the wikipedia entry) along with some papers cited by HFSC and any other academic papers you can find concerning Fair Queueing. Confirm or disprove your suspicions before replying. I have read all HFSC-cited papers and dozens of related papers and I can assure you that your posts in this thread are mostly misinformation.

Edit: Fixed link, trimmed cruft.

https://doc.pfsense.org/index.php/Limiters

Alright, thank you. That avoids me spending hours on this.

You can prioritize traffic leaving an interface, but you cannot make interfaces work together and prioritize among interfaces.

Unless you're running a VoIP call center, rate limiting UDP is not an issue. Except for BitTorrent, then UDP is sensitive to to rate limiting and will function similarly to TCP.

One of the ways the traffic is limited is by slowing it down. When the limiter is "full" then traffic will take longer to get through, so you see that as increased latency.

Or to put it both simply and more confusingly: There isn't a way to slow the traffic down without slowing the traffic down.

@KOM:

But Ill go ahead, read and see if I can figure it out.

That's how the rest of us do it.  Nobody has time to spoon-feed solutions, and you learn more by doing it yourself.

Best response ever!!!

https://doc.pfsense.org/index.php/Traffic_Shaping_Guide

https://doc.pfsense.org/index.php/Limiters

All of my LAN shaping works fine. While the first interface rule gets processed first, floating rules get processed before even those.

With pftop, confirm that everything is going to the proper queues. This is usually my problem.

Queue bitrates on sending interface must be the lowest bitrate of the route. (I think you have success already)

What you have seems like it should work.

I dunno crap about IPSEC/VPN.

First off, the main issue of traffic shaping is you must set the interface to rate limit to just below the minimum amount of bandwidth you expect to have. I have a dedicated 100Mb connection, so I can safely set my bandwidth to 98Mb/s.

If you have a 100Mb connection of lesser quality during peak hours, you need to rate limit to your lowest, so if you dip to 80Mb, you need to limit to 78Mb/s, or some value below 80Mb.

Next problem. You cannot see into a VPN tunnel, PFSense will see a single encrypted flow. If you want to rate limit inside the tunnel, you need to set your tunnel interface to rate limit to the minimum rate you want to give the tunnel as a whole.

This does mean in order to properly rate limit, you need to give it a maximum rate if you want to shape the bandwidth with something like HFSC inside the tunnel.

If you don't want to artificially set a maximum, but instead want the tunnel to be able to use an "free" bandwidth, then you could probably use PRIQ or FAIRQ. I would recommend trying FAIRQ first. CoDel may also work. If we had fq_Codel or Cake, I would recommend those because they do well with fluctuating bandwidth where your interface is doing the buffering.

PRIQ doesn't need to know the bandwidth, but your interface still needs to have the bandwidth rate limited, otherwise your interface will just pump out data as fast as the interface, which is probably 1Gb/s. When data comes in faster than 1Gb/s, PRIQ will start to re-arrange packets.

Thanks Harvey, worked like a charm…

PFSense has two types of shaping, interface shapers like HFSC and limiters. HFSC can shape the egress of an interface. In other words, you can shape the data leaving your WAN and you can shape the data leaving each of your VLAN interfaces, but you cannot have your interfaces share state. Each interface does not know anything about the shaping of another interface.

Some people claim there are some round-about ways to effectively share bandwidth across several interfaces, but at least for easy setups, you'll need to forget sharing bandwidth and instead just carve out dedicated bandwidth.

@stephenw10:

You are probably hitting this: https://redmine.pfsense.org/issues/4326
Set the limiter on the LAN side or try a 2.2.3 snapshot where I believe a patch has now gone in: https://redmine.pfsense.org/issues/4596

Steve

Steve,

Thank you!  A quick scan of that bug looks like it's a good bet as to the source of the problem.  I've been pulling my hair out trying to figure out what's wrong.  Everything's working and then I insert the two limit queues into the firewall rule and everything just stops.

Regards,
  Fred

Ah thank you.

I was trying to make sense of why such a broad rule was created by the wizard.

@mcwtim:

Proper upgrade procedure is to backup your config, uninstall any packages, do your upgrade, reinstall your packages then re-import your config.

RTFM  ;) Thank you, all this time and I did not know that.
What must have happened was that I tried to setup traffic shaping on the old release, it failed, updated to latest, re-ran the traffic wizard and failed.
Could have been that the box did not reboot. Couldn't get to a prompt on the local console .Had to have someone on site hit the power button.

Client is still up.
I'll build a new box and ship it to them.

Thanks again

TL