Dropped packets and low queue usage
-
I was testing some of my shaping rules a couple days ago, and noticed that when I was uploading a file (350MB) my queue length would go to around 10 out of 50, but my dropped packet count shot up to about 200 in a couple minutes while this transfer was happening. My connection is 15Mbit down /1 MBit up, and I was uploading at around 100KB/s which is what I expect. My question is why were there so many dropped packets when the queue length never went above 10? I thought that should 'only' happen when the queue is full. Can someone please explain this behavior? BTW I am using CBQ if it matters.
-
You probably just never saw the queue at 50. It shouldn't be at 50 for very long. TCP should figure out what's going on and adjust its window. Dropped packets are perfectly normal. It's what shaping is - selective packet drops.
-
It just surprised me there were that many dropped packets so fast. Maybe it's just that I never knew they were happening that often before is why it surprised me.
-
Does this queue handle TCP, UDP, or both?
(I have no idea if this matters, but I figured I would ask.)
-
just TCP traffic. I was transferring a file through SCP.
-
I recommend just using Codel and set your queue size to something large, like 512, or greater if you feel. Codel will make sure you never get more than 5ms buffered up.
-
Codel doesn't give me what I want as I have other queues and hierarchies I am dealing with. I will try increasing the length to see if that makes any difference though.
Is there any suggestion on queue length for things other than the Ack Queue? I read https://doc.pfsense.org/index.php/Traffic_Shaping_Guide#ACK_Queue_Size, but it did not give any guidance on sizing for other queues.
-
Codel doesn't give me what I want as I have other queues and hierarchies I am dealing with. I will try increasing the length to see if that makes any difference though.
Is there any suggestion on queue length for things other than the Ack Queue? I read https://doc.pfsense.org/index.php/Traffic_Shaping_Guide#ACK_Queue_Size, but it did not give any guidance on sizing for other queues.
I think that article is referring to the bandwidth allocation, not the queue length, when it says "queue sizing". The queue length should not matter, because ACKs should be the highest priority.
On my 6.33Mbit/666Kbit connection, during a single-stream download that fully saturates my connection, my ACK bitrate is ~128Kbit. My MRU is 1492.
-
If queue length doesn't matter, I could set them all to 1 ???, but I think length does matter to some extent. I just don't know how to determine what that is.
-
If queue length doesn't matter, I could set them all to 1 ???, but I think length does matter to some extent. I just don't know how to determine what that is.
ACK queue length, assuming that it is the highest priority queue, is unimportant (just leave it default). The length of all other queues is important, as it may determine the delay of the traffic passing through those queues.
-
An interesting tool to use is ISCI's Netalyzr. It at least attempts to tell you what your buffer delay is. It might be useful. It tests a bunch of different things. Java required.
http://netalyzr.icsi.berkeley.edu/
-
Codel doesn't give me what I want as I have other queues and hierarchies I am dealing with. I will try increasing the length to see if that makes any difference though.
Is there any suggestion on queue length for things other than the Ack Queue? I read https://doc.pfsense.org/index.php/Traffic_Shaping_Guide#ACK_Queue_Size, but it did not give any guidance on sizing for other queues.
Codel is just a queue. All a queue does is allow you to buffer packets, but drop packets when it get's "full". That is all Codel does. The only difference is Codel defines "full" as 5ms instead of a fixed number of packets. Codel also has better properties when the queue gets full, it tends to drop packets from heavy streams more than light streams. fq_Codel would be even better, but we don't have that one yet.
-
So what does Codel do when I have traffic that I want high priority and is saturated? Does that mean it will start dropping packets from this traffic to let lower priority traffic through?
-
Codel does not do traffic shaping, it is a FIFO queue. All it does is drop packets when the queue gets "full". All of the current queues work this way, the only difference is how they determine if the queue is full and how they drop packets. The benefits that it gives over a dumb FIFO queue, which is the default:
- Head drop which affects heavy flows more than smaller flows.
- Does not do strict drops when full, but instead does an ever increasing rate of packet drops
- Does not drop based on the number of packets, but instead based on how long the packets have been queued
Queue managers are completely different that traffic shaping and are used in conjunction with them. The issue with buffer bloat is typical internet packets range in size from 64bytes to 1500bytes. Sizing a queue based on the number of packets completely ignores the size of the packets. All traffic in a queue is the same "priority". Use use a traffic shaper like Priq or HFSC to determine which queue is high priority and which one is low priority, but the queue itself can be Codel.
fq_codel is not a FIFO queue and will re-arrange packets between flows based on the rate difference between enqueue and dequeue.
-
Thank you for the explanation of that. That helps ;D. I will play around with that and see what I get.
-
Changing to Codel gave me no drops. I did see some suspends while the transfer was happening, but had 0 drops, so I am happy.
I read somewhere that I only need to implement Codel on the WAN side, is that correct? It's not necessary on the LAN side of the queues right?
-
Drops are a good thing to signal the sender to back off, but drops are not good if they're too aggressive. Codel just makes it simple. It can be useful on the LAN side, but less useful. Especially since download tends to be much faster than upload for most people.