Simple, effective traffic shaping (£50)
-
Dusan: I'm not sure why you get different results to me. I am going by what pfSense is telling me… maybe the queue graphs are wrong?
Also, what data is going through qOthersUpH? You don't make it clear if you are uploading anything. Unless you are reading many very small articles, the amount of data being uploaded, excluding ACKs, should be very small, probably less than 1k/sec for NNTP. That is what I see, almost no upload except ACKs.
Yes, I downloaded a large amount of articles. The packet size in qOthersDownH was about 800-900 bytes.
Sorry for not making clear that I didn't upload anything through qOthersUpH/qOthersDownH (except NNTP and a very little amount of ICMP and DNS). For NNTP, I tried qOthersUpL/qOthersDownL before, but there're no differences.
At the moment I am running Bittorrent, uploading at 11kB/sec and downloading at around 550kB/sec (note bytes). pfSense shows 150kb/sec for qwanacks (expected as BT uses many more connections than NNTP) and 117kb/sec for qP2Pup (88kb is data upload, presumably rest is BT protocol overhead). qlanacks is 6-8kb/sec and qP2Pdown is 5-6mb/sec. All other queues empty.
For that, the total upload is around 270kb/sec, of a total 512kb theoretical, lets say 480kb after overheads. pfSense traffic graph confirms this. There should be bandwidth left over.
Based on my P2P download-only and upload-only patterns, I've calculated that your uplink is saturated at qP2Pdown=6 mb/s and qP2Pup=169 kb/s. (There are many other solutions, for example 497 kb/s upload-only or 9.1 mb/s download-only.) Thus for bittorrent, your upload-and-download pattern seems similar to mine.
I have found that the basic traffic shaping in m0n0wall seems to work just as well as pfSense, in terms of response times when browsing and ability to download while uploading.
m0n0wall can classify packets based on packet size. pfSense can't. So with pfSense one must find it hard to shape, say, bittorrent upload and download.
Simply prioritising ACKs and non-P2P traffic actually works much better than the default pfSense config from the traffic shaping wizard. By tweaking qwanacks up to 65% bandwidth and 20% m2, download speed is maintained at 11kB upload speed.
That's new to me. I've been believing that linkshare' m2 overrides bandwidth setting…
It seems you are satisfied with your config. Congratulation. :-)
-
Based on my P2P download-only and upload-only patterns, I've calculated that your uplink is saturated at qP2Pdown=6 mb/s and qP2Pup=169 kb/s. (There are many other solutions, for example 497 kb/s upload-only or 9.1 mb/s download-only.) Thus for bittorrent, your upload-and-download pattern seems similar to mine.
Sorry, I still don't follow you… While I was writing my previous post, BT jumped up to a full 10Mb for a while (couple of minutes). During that time, I was still uploading at 11k/sec, and the total upload bandwidth in use was about 300kb/sec. So, it is possible in real world conditions.
Are you trying to say that my link should be saturated at 6mb/169kb total (including acks etc)? If that were so, I don't think it would be possible to download at 10mb/sec. I am currently downloading from NNTP with 24 connections at 9.4mb/sec, pfSense showing 280kb/sec for qwanacks and 1.8kb/sec for qwandef. Strange thing is, if I now try to upload at 120kb/sec download speed will drop. The total outgoing would only be 400kb/sec, and traffic shaping is supposed to ensure that ACKs and NNTP control traffic (in qwandef) get priority and so downloads remain unaffected.
Simply prioritising ACKs and non-P2P traffic actually works much better than the default pfSense config from the traffic shaping wizard. By tweaking qwanacks up to 65% bandwidth and 20% m2, download speed is maintained at 11kB upload speed.
That's new to me. I've been believing that linkshare' m2 overrides bandwidth setting…
Maybe it does, frankly I don't know… It was the only parameter not 0 by default, so I tweaked it. Having said that, it didn't seem to make much difference, only bandwidth did.
I'm still not happy with traffic shaping, and am still offering a bounty to anyone who can get it to work properly. To that end, let me restate a primary goal:
Allow unlimited uploading in, say, FTP or BT while still being able to download at over 8.5Mb/sec and without dropped connections for HTTP/NNTP/SMTP/POP3 etc (a few BT drops are OK).
-
Based on my P2P download-only and upload-only patterns, I've calculated that your uplink is saturated at qP2Pdown=6 mb/s and qP2Pup=169 kb/s. (There are many other solutions, for example 497 kb/s upload-only or 9.1 mb/s download-only.) Thus for bittorrent, your upload-and-download pattern seems similar to mine.
Sorry, I still don't follow you… While I was writing my previous post, BT jumped up to a full 10Mb for a while (couple of minutes). During that time, I was still uploading at 11k/sec, and the total upload bandwidth in use was about 300kb/sec. So, it is possible in real world conditions.
Are you trying to say that my link should be saturated at 6mb/169kb total (including acks etc)? If that were so, I don't think it would be possible to download at 10mb/sec.
Only uplink is saturated, and 169kb/s was only qP2Pup, not including qWANack. This was only one among infinitely many possible combinations based on a download-only and a BT upload-only pattern. This wasn't accurate, as you didn't show your own patterns (upload-and-download data don't help very much). If you are interested in calculation, try to disable BT download to construct your own upload-only pattern, then disable BT upload for download-only one.
I am currently downloading from NNTP with 24 connections at 9.4mb/sec, pfSense showing 280kb/sec for qwanacks…
9.4 mb/s and 280 kb/s makes sense to me now. In a previous post, your figures said 9 mb/s and 100 kb/s that's why I made a quick test which didn't confirm them.
Now, let's back to the main problem:
…and 1.8kb/sec for qwandef. Strange thing is, if I now try to upload at 120kb/sec download speed will drop. The total outgoing would only be 400kb/sec, and traffic shaping is supposed to ensure that ACKs and NNTP control traffic (in qwandef) get priority and so downloads remain unaffected.
As you said, and I agree, that 1.8 kb/s are NNTP control packets, which are very important to maintain your download speed. If for some reason, it drops to say, 0.1 kb/s then your 9.4 mb/s download proportionally drops, ie. to 522 kb/s. Control packets have to be priotized, you're absolutely right.
Unfortunately, they do not get any priority over other packets sent through qWANdef, as packets in the same queue are sent in first-in-first-out order.
Maybe for NNTP outbound, instead of using qWANdef/qLANdef, you should try qWANack/qLANdef and see if it looks any better. I've never tried this before.
-
snip
Thanks for that, we are on the same page now I think…
I'm afraid I can't make download only stats because BT clients limit download speed when you try to upload at less than a certain amount (usually 5kB/sec). I will try seeding (upload only) though, see what I can get.
Maybe for NNTP outbound, instead of using qWANdef/qLANdef, you should try qWANack/qLANdef and see if it looks any better. I've never tried this before.
Ah, so you think the speed problems could be down to the control info being squashed by the massive upload? That sounds like a good theory. All uploads are either ACKs or in qP2Pup, so should be lower priority than the control commands which are in qwandef. Maybe qwandef needs more bandwidth allocated to it? The default from the wizard seems to be totally useless.
Many thanks for your assistance so far. It looks like it might actually be possible to make traffic shaping work with your ideas on configuring it. I wish someone like you had been around earlier to help develop the wizard or write the shaper documentation.
-
Many thanks for your assistance so far. It looks like it might actually be possible to make traffic shaping work with your ideas on configuring it. I wish someone like you had been around earlier to help develop the wizard or write the shaper documentation.
Bleh! Do you know how much time the authors of the Traffic shaper has spent? It took 3 iterations to even come up with what we have now….
-
Well, I'm not trying to be harsh… I am a computer programmer (C/assembler/Java mostly) so I know how hard it can be. Actually, it reminds me of writing drivers back in the Amiga days. It becomes very hard to write something good because there is so much variation, so many different configurations to consider. I think that is probably what has happened here.
There are some really weird things in the TF config that the wizard puts out. For example, making everything default to 1% bandwidth. Maybe it works okay for VOIP or whatever it was tested with, but if you look at the forums a lot of people want it for controlling P2P (especially on shared connections, which mine is).
Even people like Netgear seem to have trouble implementing TF in a way that works well for everyone. At least with pfSense, the ability to configure it is there. At first I was sceptical about the new shaper code, since it was performing worse than the old ALTQ stuff in m0n0wall. However, it looks like a solution is possible...
If I get time, I'll try writing up some notes for the wiki. Dusan, do you have a PayPal address?
-
…since it was performing worse than the old ALTQ stuff in m0n0wall. However, it looks like a solution is possible...
m0n0wall never used ALTQ. It uses dummynet which is completely different.
-
Mojo-chan, an (unverified) idea is not worth to pay for. Rather, I'm curious about how you solve the problem. For me, pfSense is an excellent work. It does it job so well that I couldn't reproduce your problem conditions. (I've only experienced bittorrent download drop, but that's an other problem.)
Perhaps your bounty would be possibly good for the current "Traffic Shaper Changes" project that seems to meet your original requirements. But that's up to you. :-)
Hope to see your notes on wiki soon.
-
Dusan, I'd be happy to pay you something for your hints. Although you did not provide a full solution like I asked, you at least gave me enough hints to make it work properly. Today, after much fiddling based on your ideas and methods, I managed to have an unlimited BT upload and download at over 8Mb/sec at the same time. Upload speed hovered around 300-330Kb/sec, with ACKs taking the total to around 480Kb/sec which is frankly amazing.
It's not perfect - you at at least a few high speed connections and not too many uploads (so, for example, eMule and a single HTTP download still isn't right) but generally it is excellent. Web browsing is also very responsive.
-
PS. I added a page to the wiki, last link on the main page. Please take a look.