One problem I see is all your devices are wireless, even if pfsense puts traffic in lower priority its already went over your shared wireless network.

Not sure how well that would work.

Wouldn't it be easier to just set your p2p (why anyone would do that over wireless in the first place?) to throttle down or just pause at night.  Pretty much any p2p client I have ever looked at has a scheduler built into it, so say after 5pm pause, then resume after bedtime.

edit:  So here is part of the problem of running p2p over wireless.  Wireless is SHARED, only really 1 device talking at a time.  So with p2p there is traffic even when your not downloading or uploading anything.  Once you have joined a swarm or two, your going to be seeing traffic to your ip and port be it your actively running your p2p client even.  Now have you forwarded your ports on your firewall for p2p? So that unsolicited traffic gets sent to your p2p box right.  Well that is all traffic eating away at your shared wireless bandwidth.  Now it might not be a huge amount, but it is still traffic taking up "shared" bandwidth

so I turned on logging for just a couple of seconds on my p2p forward on 43212

pass Jul 7 08:07:53 WAN 77.31.49.71:30700 192.168.1.8:42312 UDP
pass Jul 7 08:07:43 WAN 87.16.223.199:63782 192.168.1.8:42312 UDP
pass Jul 7 08:07:41 WAN 109.254.1.15:64355 192.168.1.8:42312 UDP
pass Jul 7 08:07:41 WAN 201.76.108.87:33911 192.168.1.8:42312 UDP
pass Jul 7 08:07:40 WAN 176.32.4.140:36355 192.168.1.8:42312 UDP
pass Jul 7 08:07:37 WAN 193.151.106.142:1027 192.168.1.8:42312 UDP
pass Jul 7 08:07:33 WAN 78.34.146.138:55016 192.168.1.8:42312 UDP
pass Jul 7 08:07:33 WAN 95.96.26.78:27581 192.168.1.8:42312 UDP
pass Jul 7 08:07:29 WAN 85.243.118.210:57270 192.168.1.8:42312 UDP
pass Jul 7 08:07:29 WAN 77.85.164.13:23640 192.168.1.8:42312 UDP
pass Jul 7 08:07:21 WAN 128.71.69.106:63151 192.168.1.8:42312 TCP:S
pass Jul 7 08:07:19 WAN 41.99.20.19:13383 192.168.1.8:42312 UDP

Why not run your p2p box on a wire, so that traffic does not eat up your shared bandwidth..  And then sure put it in a penalty box so it does not eat up your inet connection.  You have 10 that you mention devices all sharing "shared" bandwidth.  Are your devices all N, the Cells for example?  If not - they are sure not helping either - its shared bandwidth, putting slower speed devices ie B on G, B/G on N only slow it down.

You have some box moving packets at G speeds - since its shared, you can not at same time have data moving at full N speeds, N is going to see something slower than if it was only N devices.

So I wonder is it your isp connection that is saturated, or is more just wireless bandwidth issue?

any help would be much appreciated.

not to push you away from pfsense, i only mention this because its floating around in my head as well.  you might look at smoothwall.  it looks to be a more newb friendly interface.  i'm struggling with wrapping my mind around some of the technical stuff in pfsense and might demo smoothie myself.  i'm gonna give pfsense a shot first though and hopefully learn a little on the way.  just a though.

I have a somewhat similar problem.. although not at that high speeds. My isp gives me 50/50 atm, and disabling the limiter i get 49/46 (atleast on the isp bandwidth meter). I have set the limiter to 48/48, but when i do that, i get 44-46/36-38.

Ofc the download limit should be around that i guess.. but why almost 10 mbps less on the upload with the limiter running? Would the purpose of the limiter be kinda waste if i put the speeds more than i actually have? (i would think so.. but just asking)

Also tested this with 2.0.1 and 2.1 beta.

Oh, and im running PriQ setup, as i find this the easiest to manage.

C

I read the same articles, dont really get it fully.  To many ifs and buts on the tutorials I have seen I want a lets say kind of how-to.. know what I mean?

@dhatz:

Does the remote host support ECN ?

Read more http://en.wikipedia.org/wiki/Explicit_Congestion_Notification

I contacted my ISP (Cox Business) and they indicated that ECN is only available on their fiber lines (not cable).  However I've not taken any measures to enabled ECN on my Windows 2008 R2 server which is the one doing the downloading.  Would there be any benefit to enabling on the WS2008R2 box via this command (from Wiki you linked me to):

netsh interface tcp set global ecncapability=enabled

Thanks again.

EDIT 6/16/12 - appears once I reduced the total # of NNTP connections to my provider from 20 to 7, I am still able to achieve full download speed without queue drops.

Some questions:

1.) is the qLink (default) queue necessary for the LAN interface?  It's auto setup by the traffic shaping wizard.
2.) are "drops" in a queue something that should be expected?  should they be ignored?  or have you found there rarely to be "drops" listed beneath your status > queues?  On large file downloads at high speeds I see 5000, 7000+ although the resulting file is fine.

I would love to see something like this.

Turns out, I've managed to get this working if I put all on the FLOATING tab.  Unique rules for WAN vs. LAN interface.  No need to place any rules on the LAN tab.

I may have answered my own question!
The flow of data sent through the default queue is minimal, in my case about 1packet/sec or 520 bytes/sec.
Given that I am using pfSense to handle the PPPOE connection for my ADSL, I am wondering if this could be the ICMP packets required to maintain the PPPOE link.

If this is the case the ICMP data must be injected into the network flow after the firewall packet inspection but before being queued to leave the wan adapter.

Can anyone confirm that this is the case and/or know of a network flow diagram for pfSense that may be able to confirm this?
Also, is there a way to log the packets through a specific queue to show what exactly is being sent?

In the case of this scenario, how would you ensure ssh_bulk gets priority over ssh_login?

WAN
  SSH
      ssh_login - interactive ssh shell access
      ssh_bulk - SFTP transfer

Thanks dhatz.  Here are my HFSC rules as a starting point.  I have only one WAN (em3) and one LAN (em2) interface.  My down/upstream are 28/4 Mbit from my ISP.  I backed each down to ~97% to start.  Now I wasn't quite sure how to setup my SSH rules so that SFTP traffic goes into the ssh_bulk queue and ssh interactive shell goes into the ssh_login queue.  Appreciate all your guidance.

Lastly, I still notice drops. but my ack is currently set to 30% on both interfaces.  I've read some places that say to set it as high as 60% but I wasn't sure whether that was accurate?

altq on  em3 hfsc bandwidth 3.88Mb queue {  ack,  dns,  ssh,  bulk,  usenet,  backup,  bittor  }
queue ack on em3 bandwidth 30% qlimit 500 hfsc (  realtime 20% ) 
queue dns on em3 bandwidth 5% qlimit 500 hfsc (  realtime 5% ) 
queue ssh on em3 bandwidth 20% qlimit 500 hfsc (  realtime 20% )  {  ssh_login,  ssh_bulk  }
queue ssh_login on em3 bandwidth 50% qlimit 500
queue ssh_bulk on em3 bandwidth 50% qlimit 500

queue bulk on em3 bandwidth 20% qlimit 500 hfsc (  ecn  , default  ,  realtime 20% ) 
queue usenet on em3 bandwidth 5% qlimit 500 hfsc (  realtime 5% ) 
queue backup on em3 bandwidth 5% qlimit 500 hfsc (  upperlimit 95%  ) 
queue bittor on em3 bandwidth 1% qlimit 500 hfsc (  upperlimit 95%  )

altq on  em2 hfsc bandwidth 28Mb queue {  ack,  dns,  ssh,  bulk,  usenet,  backup,  bittor  }
queue ack on em2 bandwidth 30% qlimit 500 hfsc (  realtime 20% ) 
queue dns on em2 bandwidth 5% qlimit 500 hfsc (  realtime 5% ) 
queue ssh on em2 bandwidth 20% qlimit 500 hfsc (  realtime 20% )  {  ssh_login,  ssh_bulk  }
queue ssh_login on em2 bandwidth 50% qlimit 500
queue ssh_bulk on em2 bandwidth 50% qlimit 500

queue bulk on em2 bandwidth 20% qlimit 500 hfsc (  ecn  , default  ,  realtime 20% ) 
queue usenet on em2 bandwidth 5% qlimit 500 hfsc (  realtime 5% ) 
queue backup on em2 bandwidth 5% qlimit 500 hfsc (  upperlimit 95%  ) 
queue bittor on em2 bandwidth 1% qlimit 500 hfsc (  upperlimit 95%  )

I have yet to get PRIQ shaping to work even after following the Hammerweb guide  I really wish there was a solid how-to available.

PRES,

I would reconsider userbased Up-down q's!
Departements will do fine.

And then again, it's the traffic type you gonna shape, not the user q!
Departements then again should be or VLAN'd and/or Subnetted (higher security) so you can wel…. if you have a network that large most of these things are in place!

@podilarius:

go to Firewall -> Rules -> Floating. In there create a rule that passes port 22 either as a source or destination ( you might have to create 2 rules if you want it bidirectional).

Ah.  This is what I was looking for.  I found the queues, but had no idea where the matching of traffic to queues was happening.  I duplicated another high priority queue rule and just set it to port 22.

One thing I don't know how to do is to differentiate interactive vs. bulk ssh traffic.  For example, I want my terminal sessions to take priority over an scp or sftp bulk transfer.  The ssh client deals with this (see more here: http://kerneltrap.org/node/505) by setting the ToS field differently for interactive and bulk ssh traffic.

It would be kind of nice to have ssh in the wizard, there's a ton of fairly obscure stuff in there already, I was quite surprised to not see ssh in the list of protocols.