From what I have been able to discern, and a quick test, you can use your floating rule set to the IPsec interface and select your queue like any other interface. The traffic will fall into the respective WAN interface and queue for the VPN connection.

I tested with the following lab setup.

PC1–pfGreenBay-WAN---pfInternert---WAN-pfMilwaukee--PC2
              |                                                          |
              |                                                          |
        IPsecTunnel---------------------------------IPsecTunnel

I placed all ICMP in my "VoIP" queue and watched the PPS count on the queues as I ping from PC1 to PC2 and saw the packets show in the VoIP queue.

As for how to "match" the traffic, you can use the DiffServ flags (don't trust them to be there) or by IP/port numbers..... I just re-read you post. I see you are familiar with the DiffServ flags. as for the magic rules? I have no idea I don't work with the wizards much. I don't think I'm telling you anything new at this point but this may help clarify things for other noobs.

I've got it… In the Interfaces->LAN configuration page, I have configured the static IPv6 address. Once that address is removed from the configuration, traffic shaper interface works again. I don't know if this is a bug, or a feature. If traffic shaper does not work with IPv6, it would be good idea to leave the message in the GUI instead of the silent failure.

Regards,

Alex

That's my point, your limiter will bufferbloat you. Consider trying to limit bandwidth with download queues instead of the limiter.

@spcolyvas:

Thanks Nullity,

ICMP packets do go through the same limiter.  Eventually I'll be pumping video through these limiters to see how the video client and server adapts to the bandwidth constraints, latency, packet loss etc.  the client/server should do stuff like adjust the framerate, resolution etc.  The problem is that the behavior that pfsense is showing starting with 40ms latency and counting down to 6ms latency will really mix it up.  it may be a good test but I'd like to run other tests as well.

For testing, use limiters, sure. Limiters, AFAIK, make no worst-case latency guarantees.

but for actual deployment of video/audio services use HFSC, optionally with "CoDel Active Queue" enabled. I'd at least test your scenerio with HFSC to see your latency fluctuation is being caused by limiters or something else.

I dunno. Without more details it's hard to even know where to begin. Maybe iperf is queueing packets in bursts… maybe... ? More tests are in order. :)

@KOM:

On your floating rules, for each one change the interface to WAN, and set the Source from LAN net to any.

okay, I have cheated a bit and only changed my qDNS entires, qDNS now is populated and it seems to work now as I try to browse the net and try to observe it (refer to attached image).

and for what I notice, qDNS on the LAN side does not have any activity, is this okay?

pfTop.png
pfTop.png_thumb

KOM's solution is good, but here's another solution that uses the low queue instead of the default one.

Floating match rules are executed in order.
If you don't want all your IPs to be in the default queue, you might create two floating match rules.
The first, sets everyone in the low queue, and the second one raises the queue for the IP you want.

I've wrote a quick tutorial from my multi WAN traffic shaper experience here: https://forum.pfsense.org/index.php?topic=120380
Any improvements are welcome !

And hey, thank you Harvy66 for your solution !

@Nullity: There's still some serious packet loss going on. You thought of maybe too much packets. Is there a rule of thumb for the packet number / bandwidth ?

@KOM:

You could try getting the ASN for Googlevideo.com/YouTube.com perhaps and then shape all traffic from those IPs.

Yes that crossed my mind but I thought it might not be reliable to rely on IP addresses as they might change. But it looks like it is the only option

It depends on the limiter. If the limiter is not masked, then it's one bucket for all traffic reaching the limiter, no matter what rule sends traffic there.

If you put a mask on the limiter, for example a /24 mask, then it would work as a "per-subnet" limit so each separate subnet would have a different bucket of the declared size.

So unmasked 30 Mbit/s limiter = 30 Mbit/s total
/24 masked 30 Mbit/s limiter with two different subnets = 60 Mbit/s grand total, 30 Mbit/s per subnet.

It was never an issue except when I turned on Codel on the queues.  If I left it off , it ran fine.  It was mainly the torrenting that caused me to change tactics.

Since you can't really block them the easiest and fastest fix is to limit them.

https://redmine.pfsense.org/issues/6836

@shapoval:

Working (for me on 2.3.2) by simply adding a LAN rule at the top, Destination, Any, From (other) 3128 to (other) 3128 Custom.

Credit to: Adrea Guglielmini http://guglio.xyz/pfsense-2-3-limiters-and-squid-bugfix/

It really works. Thank you for your message.

I little study and did this.
1. setup shaper with wizard
2. edit Traffic Shaper/By Interface. Click on LAn/DMZ/Wifi and edit Bandwidth (LAN,DMZ to 1Gbps, Wifi to 150Mbps)
3. edit LAN/DMZ/Wifi qLink. Click on LAN/DMZ/Wifi qLink and edit Bandwidth (LAN,DMZ to 1Gbps - 15Mbps = 985Mbps, Wifi to 150Mbps - 15Mbps = 135Mbps). 15Mbps is my internet download speed
4. apply settings,  reload firewall rules.

I dont know if it is ok, because i dont know what set to "Queue limit in packets" in qLink.

lan_dmz_qlink_edited.PNG
lan_dmz_qlink_edited.PNG_thumb
wifi_qlink_edited.PNG
wifi_qlink_edited.PNG_thumb
lan_dmz_edited.PNG
lan_dmz_edited.PNG_thumb
wifi_edited.PNG
wifi_edited.PNG_thumb
wifi_qinternet_not_need_edit.PNG
wifi_qinternet_not_need_edit.PNG_thumb

@Harvy66:

HFSC gives you strong control over bandwidth distribution while allowing other classes of flows to use spare capacity. I have a pretty over-powered system of an i5 3ghz quad and Intel i350-T2, and I'm only seeing about 10% cpu usage when running at 2Gb/s(1Gb full-duplex). Even when I used iperf to forcefully push 960kpps 64byte UDP packets, I was only seeing about 7% cpu usage. Seems UDP is much easier to process than TCP, probably because of the state validation.

The network card is the single most important part. The second is the CPU. You really don't need a high frequency CPU, just one with a decent amount of cache and not something like an Atom that has been aggressively optimized for low power. My next system, whenever that may be, will target 2.5ghz and 8 cores with decent cache.

Thanks for the info, HFSC sounds like what I need.  I'll have to read up on it, whether traffic is prioritized by DSCP tag (fine for outgoing as I control the tags) or port number and/or IP address (incoming, can't rely on DSCP tags).

All the sub-kilobuck appliances sold at the pfsense store use flavors of Atom like the SG-2220 or SG-4860.  I'm not sure I need any more ports than WAN and LAN, as I have a Netgear GS716Tv3, which I think can do VLAN for traffic segregation.  If I could figure out how to use it.

What do you think are reasonable CPUs for QoS-ing the entirety of 250Mb or greater cable connection, if not the Atom appliances?  I do use VPN occasionally, although highest performance here, while nice, is not a huge deal.  So I would want a processor with AES-NI also?  Intel NICs are a given, from what I've read.

Thanks for the help,

@Nullity:

The restart likely worked because it reset the states, which you can do without restarting by going to Diagnostics -> States in the pfSense GUI.

That was incorrect of me. I've just restarted the modem and everything was fine. During the configuration I've resetted the states several times on the pfsense machine.

VOIP traffic likely needs to be prioritized with traffic-shaping.

DMZ or port-forwarding is likely a non-issue since these things would only help if the VOIP was non-functioning. Since VOIP is functioning, but not functioning optimally, it likely needs to have the proper bandwidth allocated with traffic-shaping.

@Nullity:

@AbdulCebbar:

@vesikk:

https://forum.pfsense.org/index.php?topic=63531.0

Follow foxale08's guide on that page for what you want. That's what I used to achieve what you are trying to achieve.

That config was working, after one of the pfsense updates there was an alert saying layer 7 limiter won't work anymore. And it didn't work. Now there is no guide to do it in new version.

Limiters have nothing to do with layer 7. foxale08's tutorial should still wor,,k.

Ok it's working but now my nat reflection is broken somehow, is this related?

I'm going off of memory, but I noticed qACK and other realtime queues only have realtime set. You may also need to set the non-realtime bandwidth.