FLAWLESS VoIP w/ Saturated Download - No Wasted Bandwidth!!

hidden772

We all know the problem. Saturated download wreaks havoc on a lot of things, especially VoIP. You can aggressively limit TCP to the point where voice sounds great, even in the face of saturated downlink - but the cost is a lot of permanently wasted bandwidth.

Using pfSense and a few other tools (tcpdump, Snort, Simple Event Correlator, rate-limiting router), I've figured out a way around that wasted bandwidth. By introducing aggressive rate limiting only while on a VoIP call, I can guarantee excellent voice quality. The only wasted bandwidth, per-se, is strictly for the duration of the call. Once the call is complete, full bandwidth access is automatically restored. I call it Application Aware Triggered Quality of Service (AATQoS).

I've written a complete how-to with instructions, config files, Snort rules, telnet scripts, etc. here: http://www.xmission.com/~hidden/aatqos

I've been using this on my pfSense 1.2-RC3 (and now RC4) setup for the last week and it works perfectly. Comments are welcome and feedback is appreciated.

I'm sure there's a way to do this 100% within pfSense (and not needing a rate-limiting router) - but I'll need help in figuring out which file(s) to modify and how to restart/reload the shaper via the CLI.

Any idea if the "new" shaper will have capabilities similar to this?

eri--

This is over complicated! (my opinion anyway)

On the shaper even now if you do setup the qVoIP with HFSC scheduler and set realtime to something like this(values depend on your choice, i am setting them with percentage regarding they parent):

m1 = 25%
d = 30
m2 = 20%

should keep your quality to acceptable levels.

Dynamic allocation is being discussed http://forum.pfsense.org/index.php/topic,7406.0.html
and here http://forum.pfsense.org/index.php/topic,7502.0.html

hidden772

@eri--:

This is over complicated! (my opinion anyway)

On the shaper even now if you do setup the qVoIP with HFSC scheduler and set realtime to something like this(values depend on your choice, i am setting them with percentage regarding they parent):

m1 = 25%
d = 30
m2 = 20%

should keep your quality to acceptable levels.

I turned off all rate limiting on my WAN router and used m1 = 25%, d = 30, m2=20% and even higher values (40%) and am still unable to get acceptable levels of quality. ;( That's with my qlanroot set for 1200k, which is about 20% less than what the downlink is. I've even tried qlanroot=1000k (33%less than downlink) and still quality suffers.

I've checked the queues and see that bulk file transfer is placed in a low queue and that VoIP is being placed into the qVOIPUp/Down queues as well.

Any other ideas? I'd love to get this working inside pfSense without having to resort to external scripts, etc.

Thanks again!

eri--

If possible can you try the values without qACK queue at all or if you cannot get rid of it, please try setting its d parameter two something twice thar of qVoIP.

I am very interested in your findings.

Thanks

sullrich

Create an alias with all of your VOIP phones and servers in it. During the traffic shaping wizard, feed it to the red alias box on the VOIPStep. This will hopefully get the bulk data in the VOIP queues.

hidden772

@eri--:

If possible can you try the values without qACK queue at all or if you cannot get rid of it, please try setting its d parameter two something twice thar of qVoIP.

I am very interested in your findings.

I removed the ACK queues and re-tested. Here's the current status report:

1.) speedtest w/o Traffic Shaping enabled = 1292 Down / 820 Up
2.) speedtest w/ qlanroot=1200, qwanroot=700: 1160 Down / 644 Up
3.) speedtest w/ qlanroot=1100, qwanroot=700: 1080 Down / 643 Up

With a high-speed download, glitching occurs even with the settings for 2 and 3, and with the ACK queues deleted. My assumption is that VoIP still isn't getting enough "headroom". I tested this theory by doing something extreme:

4.) speedtest w/ qlanroot = 500, qwanroot = 400: 489 Down / 376 Up

For setting #4, VoIP is perfect with saturated download - but my bandwidth is hampered for as long as shaping is enabled. This is the reason I wrote the Snort scripts, so that I can have perfect VoIP without having to sacrafice 50%+ bandwidth all the time.

Here's a different way of looking at it. Is it possible to place some "multiplier" on high priority traffic? Let's assume the VoIP takes 100k of bandwidth. Could the shaper multiply that bandwidth by a factor of 5, and use the 500k value for it's traffic shaping purposes? Or, if the VoIP takes 32k, then be able to give that a 15x multiplier for computational purposes?

I'm really looking to starve/penalize the bulk queues (actually all queues, for that matter) for the duration of the VoIP call. It seems to be the only way to get perfect VoIP in times of download saturation.

Let me know if you'd like me to try any other suggestions. Thanks!

hidden772

@sullrich:

Create an alias with all of your VOIP phones and servers in it. During the traffic shaping wizard, feed it to the red alias box on the VOIPStep. This will hopefully get the bulk data in the VOIP queues.

Thank you for the suggestion. I assume you want me to do this because you're not sure if the VoIP traffic is actually showing up in the VoIP queue. I've checked my previous configurations and VoIP falls in the right place. I've watched the queues page while making calls and see the traffic falling into the correct queue - so that's not the problem. That being said, I still tried this out:

The Vonage adapter is the only device going across the pfSense box and out to the Internet. I checked diagnostics/states to determine the IP address of the Vonage adapter (it's the only one using UDP 10000). According to your suggestion, I created a firewall alias for this address and re-ran the Traffic Shaper Wizard. I provided the alias to the red box in the VoIP section.

This provided no noticeable change from the previous posting. I see traffic in the VoIP queue when I make a call. Unfortunately, VoIP traffic still glitches.

I should probably define "glitches"… incoming audio (download) skips due to packet delay/loss. Without shaping at all, things are quite bad. With shaping enabled (and a sensible qlanroot), things are mostly intelligible, but there is still an unacceptable level of audio artifacting. When qlanroot is set for something extreme, the artifacting goes away and call quality is perfect... unfortunately at the permanent cost of bandwidth.

Any other suggestions? Thanks!

eri--

Well i forgot to tell you that you need to tweek the qlimit also.
The scheduler that is on pfSense 1.2 creates configuration differently form the one that i wrote so please try the suggestion below:

I am assuming only one phone so from the discussion on ACK queue on the same category on this thread we have:
N = 1
D = 10ms
S = 150 bytes
m1 = 8 * S / D
than m1 = 8 * 150 / 10 = 120 Kbit/s

d = (N+2) * D
d = (1 + 2) * 10 = 30ms
So your config should look like.
No ACK queue. For qVoIP{up,down} queue set m1 = 120Kb d=30 m2 = 75Kb

Can you please tweak the qlimit also of q{Wan,Lan}root and qVoIP{up,down} queue to something like 50 +-20% instead of 500.
Play with that since IT SURELY is why you're phone works fine when you lower bandwidth. So you will not need to do hard scripting.

eri--

Please set even m1 = m2 on the q{Wan, Lan}root queue.

Seems that all this should do.

hidden772

@eri--:

No ACK queue. For qVoIP{up,down} queue set m1 = 120Kb d=30 m2 = 75Kb

Easy enough. I assume that m1/d/m2 are located in the "realtime" Service Curve, not one of the others (upperlimit, link share)?

@eri--:

Can you please tweak the qlimit also of q{Wan,Lan}root and qVoIP{up,down} queue to something like 50 +-20% instead of 500.

Where can I make the change for qlimit? I did a quick search of the forums for "qlimit" and didn't see where I might be able to change that. It is not located in the GUI (that I can tell). I've hand edited /cf/conf/config.xml, but that file seems to be rebuilt by some other process. I looked at all of the files in /tmp, but none of them have a "qlimit" value either.

Which parent file can I edit to adjust the qlimit? When I know this, I can attempt these suggestions. Thanks again for your help.

hidden772

@eri--:

Please set even m1 = m2 on the q{Wan, Lan}root queue.

After re-running the traffic shaper, there are no values for m1 or m2. All that are in this screen are:

Scheduler Type HFSC
Bandwidth = 800Kb (or 1200Kb for qwanRoot)
Priority = 0
name qlanRoot (or qwanRoot)
Scheduler Options: "This is a parent queue"

(Also see attached .jpg for qwanRoot)

All other values are blank. Should I be populating m1 and m2? What value should I use?

qwanRoot.jpg_thumb

eri--

Get a backup of your config after creating the configuration i told you.
You can get a backup from System->Backup or something similar.

Edit that file and restore it back on the section where you got the backup.

This should allow you to make the changes made by hand to the config.xml be respected by pfSense.

And yes, i am talking about realtime specifications.

Try this and let me know.

Edit: Yeah you should fill them yourself. But for q{Wan, Lan}root this modifications should be done to the linkshare curve.

eri--

To explain even better:

1- calculations of realtime service curve should only be applied to qVoIP{up,down}
2- m1 = m2 should only be applied to q{Wan, Lan}root linkshare service curve.

hidden772

@eri--:

m1 = m2 should only be applied to q{Wan, Lan}root linkshare service curve.

What values should I place in here? Am I using the same values for the q{Wan, Lan}root "Bandwidth" section, ie: m1=m2=1200Kb for qwanRoot, and m1=m2=800Kb for qlanRoot - or is there some other value to be placed here?

eri--

To explain even better:

1- calculations of realtime service curve should only be applied to qVoIP{up,down}
2- m1 = m2 should only be applied to q{Wan, Lan}root linkshare service curve. They should be equal to the bandwidth specification on the queue and make d = 200.
3- qlimit to all of the queues or at least those mentioned here

eri--

hidden any luck with this?!

dusan

According to this source http://www.dslreports.com/forum/remark,10277184
the Vonage adapter may need more bandwidth: at least m2 = 87.2 kb/s and m1 = 174.4 kb/s (for D=10 ms).

hidden772

Thanks for all of the suggestions. That seemed to help quite a bit. If I set my root values to 1000 or 1100Kb, then the VoIP has acceptable quality. Setting it at 1200Kb is still a little choppy.

Unfortunately, one of these settings has caused some adverse side effects:

1.) some VoIP call-setup doesn't work and I get fast busy

What's the ETA for a build with the "new" shaper? Any other suggestions?

(Edit: webpage issue removed - unreleated to traffic shaping).

eri--

1.) some embedded images in webpages don't show up (server timeout, etc.)
2.) some VoIP call-setup doesn't work and I get fast busy

Well can you tell me the settings you used?!

For 1200Kb that's normal if you're on DSL since you don't get it all.
So it seems that qlimit does it for you. I would suggest to increase the qlimit for the q{Wan, Lan}root as i said by 20 or even 30% and of the queue where http traffic goes.
To fix the webpages get back the qACK on both WAN and LAN and set its realtime to 100ms with a m2 of 20% for each of them. This should get back your web pages.

One downside of the shaper in 1.2 is that you cannot control the qlimit and tbr of the tocken bucket which would give you the perfect setup for your environment.
Try to increase the m1 and m2 with what dusan suggested or if you don't mind some more to get propper quality and bandwidth. You can even try to increase the latency to some more since it seems you have a higher latency on your line. You can be safe till 50ms that is VoIP upperlimit on average.

This is to give you a better setup on current pfSense. I am working on finishing it and even waiting for the bounty people to complete thier commitement so i cannot give a ETA for that, sorry.

hidden772

@dusan:

According to this source http://www.dslreports.com/forum/remark,10277184
the Vonage adapter may need more bandwidth: at least m2 = 87.2 kb/s and m1 = 174.4 kb/s (for D=10 ms).

I'm currently using Vonage 90 codec. When you say "at least", would that mean that using values higher than M1=87.2Kb and M2=174.4Kb should reserve more bandwidth for VoIP? Like I've mentioned before, I would love it if I could slow <everything>down while I'm on a VoIP call… wasted bandwidth during the call is completely acceptable and is exactly what I want to do. If that is the case, could I just use M1 & M2 numbers of 512Kb? </everything>