@Harvy66: My understanding is that if you want Diffserv to be honored, it must be set before reaching the firewall. Traffic shaping is set at the time the connection is made. Because you have Squid running inside the firewall, the diffsrv is being set after reaching the firewall. Ok thanks. So there is no way a rule on LAN can tell which traffic passing thru is from cache.

They're physically "sub" interfaces, but logically, they are all completely separate.

Or will it cancel the load balancing as the ports are already assigned to which wans?

Got most other issues resolved.  Anyone else seeing this, or just me?

But if you use esxi it works without problem :/ Its seems to be so unfair. I cant get a good xen firewall

Many thanks!

No.  You didn't mention that.  That makes it a lot easier.  You should see those states whenever the VPN is connected no matter what you're doing.  Uploading, downloading, etc.  That is a state for the tunnel, not anything inside the tunnel.  pfSense can't see that traffic.  it's just a router in the middle just like all the other hops between you and the server. Just create the queues on LAN and WAN and pass the OpenVPN connection with a rule on the LAN interface and put it into the right queue. Looks like: Pass IPv4 UDP source 192.168.2.20 port any dest 209.xxx.xxx.xxx port 443 Set the queue to qVPN (or whatever you named them.)

Thank you I will read about this

@Derelict: Not really.  It still needs a shaper to avoid over-saturating your links. Yes, but because the limiter has that nifty feature where it can limit evenly per IP address, it leads me to believe that it doesn't limit the queue. It should be easy enough to test. limit the interface and test two clients trying to saturate upload at the same time and monitor ping remove limit from the interface and instead use the limiter and do the same upload test My guess is the limiter happens before the queue, which means it limits how quickly the queue fills up. If you want Codel to work, you need to limit how quickly it's drained. *Entirely a guess based on what features the limiter has

What is the proper way of using floating rules? From what I understand, floating rules are simply a way of having one rule that acts on multiple interfaces in multiple directions.  That's it.

Aside this, i also realized from Status >> RRD Graphs >> Queues & QueueDrops for WAN are missing too, another Codel specific cosmetic bug? Related system log: ""php-fpm[245]: /status_rrd_graph_settings.php: RRD create failed exited with 1, the error is: ERROR: you must define at least one Data Source""

Ok got it.

No idea what you need to do to make it work with a proxy.  sorry. I do see one more error.  You have both limiters masked by source address. On LAN: your out queue will be your clients' download and should be masked by dest address your in queue will be your clients' upload and should be masked by source address. These should be applied to your lan rules with in as in and out as out.

@Harvy66: I don't know much about enterprise solutions offered by ISPs, but don't ISPs primarily offer bandwidth and redundancy but it's up the the client to shape their traffic or the client is given a device that will do the shaping? When you shape per customer, you're talking about N number of rules, but when you start shaping per combination of customers, now it's N^2. Anyway, you seem to have a business case for this, but I still wonder if there is a "proper" solution short of telling the customer to do it or helping them do it on their end. Good Luck. I can't wait to see a more seasoned person's response. Hi Harvy, We don't apply traffic prioritization per protocol  , or something similar , you're right  our customers apply their own Qos rules at their end according to their convenience. We only apply a limit (upper) for each customer, simple as that.  As said before , in addition to that  I want to have a way of prioritize the whole traffic from some special clients over the others. Any ideas? bests wishes

I changed the units bat to gbits like you suggested, and my bandwidth values did not change from what they were a few minutes ago.  It sure seems like just changing that value on the form is not taking effect when you save / apply it.

https://forum.pfsense.org/index.php?topic=87931.0 It is on their mind, but no plans as of yet.