CoDel - How to use
-
I do not see much on CoDel anywhere really. I did do some research and I wanted to use it to manage a single link.
How do I do that with pfSense and does it work?
Do I just enable it on the WAN interface (the interface with the link) and go?
Everything I read says that it does not have any parameters. I know that changing the scheduler does not change the form for QoS on an interface. I do not need to enter my BW speed in right?
-
With my understanding, you will need to enter in your bandwidth, but then just enable codel, nothing more. No floating rules or anything, should just work.
-
Do you know the command to run to see if it is working?
-
pftop -s1 -v queue
I suppose SCH and in the column reading code means Scheduler codel….
I see it active in the queues display.
-
I would like confirmation that a BW value is NOT needed when enabling this on an interface.
-
If you don't enter in your bandwidth your Interface will run at full speed unless a Pause frame comes a long to stop it, at that point, you already at the whims of the buffer upstream of you, which may be bloated.
The point of CoDel is for your to manage buffers, not other's to handle buffers for you, because they do a horrible job of it.
-
If you don't enter in your bandwidth your Interface will run at full speed unless a Pause frame comes a long to stop it, at that point, you already at the whims of the buffer upstream of you, which may be bloated.
The point of CoDel is for your to manage buffers, not other's to handle buffers for you, because they do a horrible job of it.
Okay, I get that then, do any of the other parameters work then? I can find no documentation on it anywhere. Last question, should I put the full speed or do something like 98%
-
I just enabled it on both the WAN and LAN port. From what I read in other posts and observed on my system you do not need to enter any data rates. It works by watching the queue and discarding packets in a controlled manner by how long it takes a packet to traverse the queue. This way as the throughput of your link drops, CoDel will track it.
It seems to work fine on my system.
I really which they would implement FQ-CoDel. It maps the various flows into their on queue and then applies CoDel to each queue. This is even more fair to the flows. CoDel only has one queue per port if enabled.
-
Who do I need to contact to get really how CoDel works in pfsense?
-
The developers.
Easiest way to check is set your Up/Down to 1/10 your actual speed and enable CoDel. Start a large file download and watch your speeds. Your actual download speed should be great than what you set as limits.
I don't think I ever tried it. I read in one of the posts I believe that as the form used is for multiple traffic shaping setups an that in CoDel case, it was not required.
-
As I understand it, there are no settings for codel. No maximum bandwidth, no widgets or knobs. It simply looks at the packets it has and discards them if they have been there for longer than 20ms or something.
I think you can safely assume codel in FreeBSD works as described here: http://en.wikipedia.org/wiki/CoDel
-
It has no bandwidth setting because it assume your connection is as fast as your interface, aka, self limiting. If your interface is not self limiting, then you need to artificially limit it. Cable modems are self limiting because they can only send when they're allowed to by the head unit, a 1Gb port is self limiting because it can not send faster than 1Gb, but a 1Gb port plugged into a 30Mb internet connection is not self limiting. It is true that there are pause frames, but those do not occur until the modem's buffer is nearly full, at which point you have about 1-5 seconds of buffer bloat.
-
Argument and counter argument again. Still looking to figure this out.
-
“CoDel Active Queue Management
The CoDel Active Queue Management (AQM) discipline was recently added to pfSense 2.1. The name is short for Controlled Delay and is pronounced "coddle". It was designed to combat some of the problems associated with bufferbloat in networking infrastructure. Bufferbloat is described in detail at http://www.bufferbloat.net/projects/bloat/wiki/Introduction. Basically, due to the size of buffers in network equipment, traffic can pile up and go in chunks rather than a smooth stream. By controlling the delay of the traffic this effect can be lessened.
CoDel has no specific configuration controls or options. When activated for a queue, it will automatically attempt to manage traffic as described in the CoDel wiki at http://www.bufferbloat.net/projects/codel/wiki. It attempts to keep traffic delays low but does permit bursting, it controls delays but it does not pay attention to round-trip delay, load, or link speed, and it can automatically adjust if the link speed changes.
The target for CoDel is mid-range networking. It does not work well at very low bandwidth (512Kbps or less) and it does not gracefully handle large numbers of simultaneous flows or datacenter-grade traffic loads.”
Excerpt From: Christopher M. Buechler. “pfSense-2.1-book.epub.” iBooks.
They haven't yelled at me for excerpting the book yet. It really is worth the cost of a Gold membership. If you want more than that, I suggest opening a support ticket.
-
I submitted a ticket but it asks me to login to view it now. I do not have an account and I see no where to register.
https://portal.pfsense.org/support/index.php?/Tickets/Ticket/View/OMS-55668
-
You have to pay for support.
I see you were unsatisfied with the answers here and have now gone to the mailing list too.
Do you really want someone to spend their time rewriting the bufferbloat.net site just for you.
You just enable codel. That's it. What's so hard to understand?
-
The only reason that I am perusing the issue is because there are conflicting views of how the webui implements the settings put in along with it. I see when I do implement it that it only enables one scheduler so I will have to assume that putting anything in along with the codel setting for the traffic shaper for just an interface does nothing.
Just because bufferbloat says that it is parameterless does not mean that the parameters do nothing in the pfsense webui.
This is the only question I have. If it follows what bufferbloat says then nothing on a Traffic Shaper interface page should matter if I select CoDel.
I suppose instead of asking in this forum, irc, and a mailing list I will just look at the web form code and dig into it myself.
Thanks for the help, but even this forum post shows people are confused on how it works in pfsense.
The point is, I have to assume. I have spent many hours asking around if anyone knows how that single page works because I did not assume that it works someway.
Maybe I should get a gold subscription but you know what: The guide does not say much about it either.
-
OMG.
1/ Go to Firewall - Traffic Shaper - By Interface
2/ Click your WAN
3/ Check Enable/disable discipline and its children, select CODELQ from Scheduler Type dropdown.
4/ Click Save.
5/ Click Apply.What guide you need for this?
-
He's talking about it is of my opinion that CoDel won't be as effective if you don't set your interface bandwidth. It is logically impossible for CoDel or other forms of traffic shaping or queue management to work without having some means of knowing how quickly the queue should be drained. This is easy for a synchronous interface like plugging a 1Gb WAN into a 1Gb internet connection, but it is not so simple when you plug a 1Gb wan into a 30Mb internet connection. If your upstream does something like sending back pause frames, the WAN port can know to back off, allowing packets to buffer and CoDel to do its magic. Pause frames still mean that buffering is happening on the receiving interface, which is not desirable because you cannot control buffers in other systems.
CoDel doesn't need to know the bandwidth because it's the interface's job to know how fast it's allowed to dequeue. CoDel just monitors the delays on the packets. Without something to limit CoDel, it will dequeue at full interface rate.
-
This thing is "no knobs" by design. https://tools.ietf.org/html/draft-ietf-aqm-codel-00#section-4.2
-
But many think "no knobs" or needing to tell CoDel about your bandwidth means you don't need to rate limit your interface so your interface doesn't attempt to dequeue packets at line rate. CoDel tells your interface which packet to dequeue next, not how fast to dequeue them.
-
But many think "no knobs" or needing to tell CoDel about your bandwidth means you don't need to rate limit your interface so your interface doesn't attempt to dequeue packets at line rate. CoDel tells your interface which packet to dequeue next, not how fast to dequeue them.
I'm using CoDel and see no difference setting my line rate or leaving it blank. On a 30/5 Mbps connection. Love it so far, even with VoIP.
-
What happens when you completely max out your upload? If CoDel is fully working, your latency should barely budge, maybe an increase of 10-15 ms, but minor packet loss.
-
What happens when you completely max out your upload? If CoDel is fully working, your latency should barely budge, maybe an increase of 10-15 ms, but minor packet loss.
That's exactly what is happening.
My main issue was maxing out my download connection (newsgroups) causing latency on my home network. Now it doesn't matter how bad I saturate upload/download, VoIP works and web browsing (latency) is snappy. Very impressed.
-
Strange since codel works on the sending interface. Hard to believe your bottleneck was LAN, but glad it's working for you.
-
You apply codel to both lan and wan. That way it shapes traffic both directions. Even though logic says that if the packet has already made it to you, you should keep it. But if you drop it, that causes a resend which in turn causes the remote end to slow down the sending which then allows packets from other flows to traverse the queue faster.
Admittedly, as the lan is typically faster than the wan, there should not be any slow down or drops. But I applied codel to both lan and wan.
-
Strange since codel works on the sending interface. Hard to believe your bottleneck was LAN, but glad it's working for you.
The heavy downloading (30 simultaneous newsgroup connections) lagged my home network web browsing likely because of the uplink ack packets it needed to send to sustain the speed.
In any event, I've enabled codel on both my wan/lan and it has totally changed the experience.
-
@ tuffcalc,
You might want to drop the number of newsgroup connections in half. Depending on you newsgroup provider, they may be able to fill a smaller number of streams at a higher rate.
-
You apply codel to both lan and wan. That way it shapes traffic both directions. Even though logic says that if the packet has already made it to you, you should keep it. But if you drop it, that causes a resend which in turn causes the remote end to slow down the sending which then allows packets from other flows to traverse the queue faster.
Admittedly, as the lan is typically faster than the wan, there should not be any slow down or drops. But I applied codel to both lan and wan.
I turned codel off on the LAN side and notice no difference.
I'm running an SG300-50P switch with 3 Engenius EAP1750 AP's for wireless clients, so admittedly a pretty fast LAN. I'm just going to leave it off for the LAN side - intuitively it makes more sense to me.
-
I removed all my shapers and applied only codel. My downloads were as fast as I've ever seen them and simultaneous pings to my ISP's first hop were unaffected. Uploads, however, resulted in ping latency going from about 12ms to about 175ms. HFSC completely cures that at the expense of a little top-end speed. I did leave the codel checkboxes checked on all my queues though.
-
Strange since codel works on the sending interface. Hard to believe your bottleneck was LAN, but glad it's working for you.
The heavy downloading (30 simultaneous newsgroup connections) lagged my home network web browsing likely because of the uplink ack packets it needed to send to sustain the speed.
In any event, I've enabled codel on both my wan/lan and it has totally changed the experience.
I don't doubt that it has helped, but I wonder by how much in actual numbers. If you could find an IP that returns table pings, get maybe 30 seconds of samples, then start downloading and get another 30 seconds of samples.
Unfortunately, I cannot do any sort of tests on my network because my ISP has designed their network to have no buffer bloat and stable bandwidth. If I had my old ISP, I could have done such tests. While they were pretty good, they had classical issues of buffer bloat and bandwidth could briefly drop upwards of 30% during peak hours. Nothing horrible, but not "perfect".
I recently watched an interview from one of the CoDel people showing a reduced number of pause frames when using CoDel, but unfortunately did not show his exact network setup or where CoDel was applied, so I made some assumptions that sound as if they are a bit incorrect. The difference between theory and practice, implementation details. Perhaps pause frames are sent some time prior to full buffer.
Thanks for everyone helping to fix some of my false assumptions. There is obviously something more at play that I am missing. I love learning and I apologize for spreading somewhat false information.
-
I use some stuff out of a youtube video.
You run one process that pings 5 per second and outputs to a file.
You run another that plots it with gnuplot.
The video (which includes the very short scripts) is here: https://www.youtube.com/watch?v=EfXImr5q-sw
 -
I wonder if my ISP uses fq_CoDel. This was taken during my recent issue with BitTorrent flooding my connection with up to 103Mb/s, yet my average was 98.6Mb/s as reported by RRD.
Even then, my pings remained low. I cannot think of a way that my pings could remain so low while still maintaining large enough buffers to be practical. Packet loss indicates a full buffer, yet the pings do not reflect such a thing. fq_CoDel is the only algorithm that comes to mind. It was not this way prior to their recent upgrades. Packet loss was typically accompanied by latency, albeit 10-20ms.
edit: Seems Cisco used the idea of CoDel and made PIE. Both are similar. Cisco even has fq_PIE. I assume this is why I see stable latency from my ISP.
-
I removed all my shapers and applied only codel. My downloads were as fast as I've ever seen them and simultaneous pings to my ISP's first hop were unaffected. Uploads, however, resulted in ping latency going from about 12ms to about 175ms. HFSC completely cures that at the expense of a little top-end speed. I did leave the codel checkboxes checked on all my queues though.
While not perfect, it seems CoDel and make things a lot better without any configuration. I would love to see this same test with fq_codel, if we remember by then. ;D
-
He's talking about it is of my opinion that CoDel won't be as effective if you don't set your interface bandwidth. It is logically impossible for CoDel or other forms of traffic shaping or queue management to work without having some means of knowing how quickly the queue should be drained. This is easy for a synchronous interface like plugging a 1Gb WAN into a 1Gb internet connection, but it is not so simple when you plug a 1Gb wan into a 30Mb internet connection. If your upstream does something like sending back pause frames, the WAN port can know to back off, allowing packets to buffer and CoDel to do its magic. Pause frames still mean that buffering is happening on the receiving interface, which is not desirable because you cannot control buffers in other systems.
CoDel doesn't need to know the bandwidth because it's the interface's job to know how fast it's allowed to dequeue. CoDel just monitors the delays on the packets. Without something to limit CoDel, it will dequeue at full interface rate.
I just wondered if inputing a number in the Bandwidth box would actually do anything because CoDel is knobless. It does do something. It does limit the connection. (I tested it by limiting my connection severely) The only reason I wondered if this knob/setting would work is because everything describes CoDel as knobless. But I also understood that it was an evolution of RED. RED having all the settings it did I could see that knobless in relation to CoDel ment, no settings but still takes a Bandwidth setting.
This has, at least, turned into an interesting post with all these result studies. I may mirror some of the methods and post my results just to compare.
-
I use the bandwidth settings to make sure that my upstream never ever receives data from me faster than it can process it. Even the CoDel people have stated in at least one of their many videos that it is desirable to make sure that your upload speed is slightly less than your actual rated speed. I have 100Mb, so I set it to 96Mb. If your immediate upstream buffer already uses CoDel or similar AQM, then you gain little or nothing, but most ISPs do not implement modern AQMs or any AQM.
You want CoDel on the egress interface at any junction of a fast and slow interface pairing. Example, if you have a 10Gb interface that feeds into a 1Gb interface, you want the egress side of the 1Gb interface to have CoDel, as that is where the choke point is located. Another more common example would be a 1Gb uplink with 2 or more 1Gb feeds, then you want CoDel on the egress side of the 1Gb uplink.
-
I just found this on their page. They have a whole section in tuning your rate limiter to work best with CoDel. They even warn against using hardware offloading because stuff like TSO can ignore your rate limiter, causing your interface to send data too quickly.
http://www.bufferbloat.net/projects/codel/wiki/Best_Practices_for_Benchmarking_CoDel_and_FQ_CoDel
Broadband Links
Fq_codel runs extremely well on asymmetric links such as your commonly available 24.5/5.5 service from a cable modem provider like Comcast. (in conjunction with setting a shaper to your providers's rates and htb rate limiting)Lastly, just running fq_codel by itself, does not help you very much when the next device in line is overbuffered (as in a home router next to today's cable modems). (it DOES help break up microbursts, however, and generally "does no harm") In that case, using HTB to rate limit your router to below the next gateway device and then applying fq_codel will work. See the note above about limitations to HTB.
-
Just another cent that keeps me wondering…..
Basically both WAN and LAN are using Scheduler Type: HFSC . ( see attached pf1.jpg)
But all its children (qACK, qDefault,qP2P…etc) are using Scheduler Options: Codel Active Queues (attached pf2.jpg)=========================================
pfctl -sa
…
ALTQ:
queue root_vmx2 on vmx2 bandwidth 1Gb priority 0 {qLink, qInternet}
queue qLink on vmx2 bandwidth 976.50Mb qlimit 500
queue qInternet on vmx2 bandwidth 23.50Mb hfsc( codel upperlimit 23.50Mb ) {qACK, qP2P, qVoIP, qOthersHigh, qOthersLow, qDefault}
queue qACK on vmx2 bandwidth 5.88Mb hfsc( codel )
queue qP2P on vmx2 bandwidth 2.35Mb hfsc( codel )
queue qVoIP on vmx2 bandwidth 96Kb hfsc( codel realtime 96Kb )
queue qOthersHigh on vmx2 bandwidth 8.22Mb hfsc( codel )
queue qOthersLow on vmx2 bandwidth 1.18Mb hfsc( codel )
queue qDefault on vmx2 bandwidth 3.52Mb hfsc( codel default )
queue root_vmx0 on vmx0 bandwidth 1.14Mb priority 0 {qACK, qDefault, qP2P, qVoIP, qOthersHigh, qOthersLow}
queue qACK on vmx0 bandwidth 285Kb hfsc( codel )
queue qDefault on vmx0 bandwidth 171Kb hfsc( codel default )
queue qP2P on vmx0 bandwidth 114Kb hfsc( codel )
queue qVoIP on vmx0 bandwidth 96Kb hfsc( codel realtime 96Kb )
queue qOthersHigh on vmx0 bandwidth 399Kb hfsc( codel )
queue qOthersLow on vmx0 bandwidth 57Kb hfsc( codel )
=========================================Does this best-of-both-worlds "hybrid" setup actually works for more granular control?
-
Yes. CoDel manages queue latency and HFSC manages queue bandwidth. The other thing to remember is that any queue that is not using all of its bandwidth should effectively have "0'" latency. So if I give my qGames 5Mb of bandwidth, but it is only using 4Mb, then the latency should be 0, except for microbursts.
CoDel only really helps when you do not have enough bandwidth. Technically, enabling CoDel on every queue is probably a good thing, but some traffic types tend to be low or fixed bandwidth and I see no reason to use CoDel if I've given them enough real time.
HFSC and CoDel used together really shines when you are using all of your bandwidth and you want to keep all queues low latency, even bulk data queues, like web traffic. You might have web traffic set with lower bandwidth than VPN traffic, but web traffic can be bursty or consume large amounts of bandwidth for long periods of time. CoDel is great for this because it keeps web traffic low latency and responsive for users attempting to browse while allowing good bandwidth utilization for file transfers.
HFSC is not as good as a simple priority queue if your link bandwidth is not stable because HFSC does not dynamically adjust to changing bandwidth conditions.
fq_CoDel will be that much better once available.
-
I am going to put this in place for my configs I use at LAN parties. Going to test at home then apply to the routers used for the LAN's.
I use the following queues for high traffic - qGames and qHTTPSteam so using CODEL on them should have the desired effect as they are commonly the most used queues.
Next LAN isnt' until March but I might be able to squeeze some testing in at a smaller event in Feb.
