@lihuizhan:

hello, i see a list of instant messengers to choose from in the Traffic Shaping setup wizard but Skype is not in the list… this needs to be added. Is there any work around in the mean time?

Also, I'd like to see in the wizard a way to leverage DSCP values in the priority queues. Such as prioritize all packets with DSCP value EF, AF31, CS3. if there is also another way to do this without the wizard? id love to know. thank you guys...

If you create a catchall rule (put above the default), you can scroll down to advanced settings and choose the DSCP tags to match traffic.

Skype doesn't use DSCP or any TOS/ COS markings. You can however, use the integrated QoS policy in Windows to mark Skype traffic with DSCP tags.
See the technet article on how to achieve this:
https://technet.microsoft.com/en-us/library/dd759093.aspx

More Graphs

NexusLANPacketsOverall.JPG
NexusLANPacketsOverall.JPG_thumb
NexusStatesOverall.JPG
NexusStatesOverall.JPG_thumb
NexusLANOVerall.JPG
NexusLANOVerall.JPG_thumb
NexusWAN2QueuesOverall.JPG
NexusWAN2QueuesOverall.JPG_thumb
NexusWAN1QueuesOverall.JPG
NexusWAN1QueuesOverall.JPG_thumb
NexusWAN3QueuesOverall.JPG
NexusWAN3QueuesOverall.JPG_thumb
NexusLANQueueOverall.JPG
NexusLANQueueOverall.JPG_thumb
NexusLANOVerall.JPG
NexusLANOVerall.JPG_thumb
NexusWAN3Overall.JPG
NexusWAN3Overall.JPG_thumb
NexusWAN2Overall.JPG
NexusWAN2Overall.JPG_thumb
NexusWAN1Overall.JPG
NexusWAN1Overall.JPG_thumb

Thanks gents.

They seem to be some sliding window and approach the correct value, but they are not realtime

I installed a pfSense instance from scratch by installing it through CD.iso. (is is virtual and runs on a VMware ESXi, so this is hassle free)

No configuration except the WAN/ LAN interfaces.

Same! Download max 2Mb/s- nearly the same as the uplink!

So I doubt it is a pfSense issue. Could confirm it is not by attaching a Win7 directly to the cable modem- speedtest showed same results.

So definetly not a pfSense issue.

Still on searching for the root cause together with the provider's support team.

Anyways, thanks for the ideas!

/KNEBB

There was some other traffic in qDNS for a bit that inflated that queue.  I will see about making a single post after this weekends lan party that we will be having and then just keep updating that thread.

It's an ALIX Board.

Ah OK.  Nevermind, it's not your NIC, or at least not for that reason.

I have read the article pointed to by Nullity: http://www.linksysinfo.org/index.php?threads/qos-tutorial.68795/

according to the above link and what you guys are saying…, it all goes to controlling/shaping up the "upload" queue which will also directly influences the download stuff.

I have researched a bit and the thing I see ATM is squid's "delay pools"..., but I will still have to try it out.

anyone can point me on how to limit/shape all kinds of streaming (and download as 2nd)? as this is the only thing gives problems on games [when my poor 5mbps link is saturated]

Hi,

I am using traffic shaping and squid3 and squidguard, and there are problems indeed. The traffic shaping does not work anymore 100%. In version 2.1.5 I used to change the port from 3128 to 81 in transparent mode and everything was OK.

I hope this is a bug what will be fixed soon.

???

Probably means you did something wrong. The only way to tell if you did something wrong is to see what you did. Please post your limiter setup.

Ok, so i think my entire hold-up was probably how i have a multi-LAN setup.  IF i set a traffic shaper on one of the LAN interfaces, with the goal being to throttle downstream internet traffic, it would also have the side effect of shaping any LAN to LAN traffic that passes through that interface (like from LAN wired to LAN wireless).    Traffic going from one host to another on the same LAN (two hosts on wired LAN, for example), since those hosts are connected off a switch,  and are in the same subnet, they aren't routing to a different subnet and their traffic isnt being throttled.

this was probably my hold-up the entire time as I was testing from my wireless laptop to a wired server.

If i put every thing on my LAN on the same subnet and turn on the shaping on the LAN and WAN interface, i'll get my expected throttling of internet traffic (Because i'm just dealing with a single WAN and single LAN interface).

Anyways, i think my multi-LAN setup had me tripped up and i was missing the obvious.

Thanks everyone for your responses and tips/tricks.

I think i'll just set up a basic CODELQ shaper (unless there is a better scheduler to use) for WAN and one LAN and have all my hosts on the same LAN – then i'll get full gigabit between hosts on the LAN and throttled back internet from WAN <> LAN

50 is the default. I recommend just enabling CoDel on each queue. Large buffers are bad because they cause bufferbloat, but they're great for high throughput(except in extreme cases, like more than 1,000ms of bloat).

If you are interested in persuing a bug report I would see how other successful pfSense bug reports were conducted.
https://redmine.pfsense.org

What I do, which may not be what you do, and remains (to my testing) incompatible with transparent squid on the same box…

Avoid wizard. Backup configuration before starting. Traffic shaper screw-ups can be epic and being able to back out and do over is a good plan. I've personally never had a good outcome from the wizard, YMMV.

Traffic shaper, first tab "by interface" Wan (codelq, set nothing, it's codelq, nothing should need to be set) Lan (same.) Enable.

Third tab, Limiter, create LanIn (this is what you think of as "out" to the world) and LanOut (this is what you think of as "in" from the world) set values for the traffic limits you want on the directions. You may tune these later on. These should be (or possibly become at the next step) yellow folder icons.

Leave "mask" set to none here.

With those created and enabled, select LanIn and add queue, which should be a white page icon. Under the the lanin queue I named it LanInQ) , select source addresses. Same with LanOut, create LanOutQ, Destination addresses.

Change firewall rules, LAN, "advanced" "In/Out" to run traffic in LanInQ/LanOutQ.

Lanin (traffic into LAN, out to world is pretty closely controlled (you actually have direct control here) LanOut is a bit less under your direct control, but the setting does have an influence.

This specific setup is to divide the bandwidth among hosts "evenly" (only even if they all want more than they can have)  - you can also use other variations to provide pipes of a specific limited BW; I came down on the side of BW is wasted if not used, so if one hog gets it all when nobody else is using it, fine, but I needed to make sure that if 9 or 90 other folks showed up they would get a "fair" share as near as possible, and this mostly does that (far better than just capping everyone's BW, which means the hogs are on there longer hogging and nobody's speed is EVER good.)

The limiter numbers do need to be less than the actual BW, but not by quite as much as you are proposing (90-95% is generally fine) - I look at what my "quality" figures (ping times) are running to adjust my tuning - if the limiter size is too large, the ping times go to heck in a handbasket.

I played around with HFSC for quite a while before arriving here, and here does what I want much better, IME.

The manual (could be automated) scheme I use (with the limiter) is to review use in bandwidthd and put the winners into a lower-priority queue - this is by writing IP addresses to an alias, and the LAN rules run the alias through the appropriate queue.

For your scheme you could put them into a limiter queue that was speed-limited per pipe (rather than my scheme of a queue that has a lower priority, but no actual numerical limit, if nobody else is using BW.)

I think the portal has a built in setup to simply cut them off after X amount (I have not used the portal myself)

And if you want to test this - open a game on your PC - do a packet capture on it when your playing it on a Monday then wait till the weekend or the even the next day and do the same thing and compare the captures.

See if they differ at all.

Shaping gaming traffic is kind of like hunting land mines with a field knife - it is a slow and methodical process that requires patience because if you rush it - boom!!!  :)

Yes, I started using that queueing system (please help me God)… and priorities are not relevant anymore...

Thanks for your explanations...

I went with this setup lastnight.

20% for ACKs, but then qClassified is the golden ratio more than qUnclassified. qUnclassified is where my unclassified and P2P traffic goes, because P2P is so hard to classify. I have broken qUnclassified into to groups, UDP and the normal default queue, which will primarily be unclassified TCP. I have a floating rule at the very top to match all UDP traffic and place it in qUDP.

The bandwidth is split 50/50 between qDefault and qUDP, but qUDP has a service curve that gives a 25% boost over 5ms. My connection is quite fast, so 5ms is a long time. Based on my limited understanding of service curves, m1 is the pseudo-bandwidth, for lack of a better term, and d is the target latency(must be a realistic value for your connection), and of course m2 is your actual real bandwidth.

I don't want to get into the exactness of how service curve work, but the one example that I saw was a 64kb rate where they wanted to cut the jitter in 1/4, so they gave the queue an m1 of 128kb and m2 of 64kb, because they wanted a 64kb average but wanted the link to act like 128kb when it came to scheduling the packet. I saw "packet" and not "packets", because 64kb is such a slow rate and the size of the packets, that it worked out to take 10ms to transfer one packet. So they setup the curve to have a d(duration) of 20ms. The final result was 128kb 10 64kb m1/d/m2.

The way I interpreted that one example, is if you have latency sensitive traffic there the intervals of traffic result in a burst relative to the provisioned sustained bandwidth, then you can set m1 and d such that if the queue is still within its average bandwidth, it can get a "burst"(term used very loosely), in order to allow the packets to get scheduled sooner than they would have otherwise for their sustained bandwidth. Of course in order to reduce the delay of one queue, you need to increase the delay of the other queues, not that I care for my "normal" traffic. The over all average bandwidth is still maintained, and the two queues will still have an average split of 50/50. This also implies that the "burst" is a debt to be repaid by consuming less bandwidth after the burst.

ShaperHierarchy.png
ShaperHierarchy.png_thumb