This was working and I think I hit a bug, but I am not sure exactly what it is or why it happened. I had the initial 5 remote sites and their respective queues. I had floating rules to direct all the traffic to the queues for each site. I added in a 6th site, a 6th set of queues and a 6th set of floating rules, and now ALL open vpn traffic destined for HQ's lan is ignoring the queue assignments in the floating rules. All traffic is going to qlink or qack on the lan interface and I haven't found out why just yet. Floating rules that apply to traffic going out the wan, or going out the lan with traffic from the lan, are still categorized to the correct queues. I am absolutely stumped right now and this is a network in use 24/7 so I can't constantly try things to fix it. I am going to have to setup a lab on VSXi and try to figure out what the heck is going on. That is, unless someone else out there knows? I still haven't found a way to prioritize OSPF packets yet either since they never touch the wan. I don't think there is a way. The way I have delt with ospf packet loss was raising the dead timers to 5 minutes, far from optimal, but it works for this setup.

There's actually 2 parts to traffic shaping - the shaper queues that determine what should happen with the traffic that's put into them, and floating firewall rules that assign traffic to the queues. What you need to do is split the firewall rule that assigns those ports into 2 (or more) separate ranges. You'll find the rules on the Floating tab in the Firewall Rules menu. What you want to end up with is a rule that does (suppose you want to exclude port 4000) port 3000-3999 and a second rule that does 4001-32000.

@cmb: it gets a bit complicated because part of your shaping has to accommodate the fact it's ESP traffic on WAN. Ah, I see … I wonder if it would be possible to copy the ToS byte from the original IP header to the new ESP header (or perhaps it's being done already?) In Cisco it's done by default, it's called the “ToS Byte Preservation” feature. Edit: Based on a quick Google search, there seems to be a system tunable net.inet.ipsec.ah_cleartos that is set by default, but I don't see a corresponding ESP tunable.

ALIX 2d13< – same as 2D3 with an added port header i believe, same hardware performance specs I can Pull 37mbit on my comcast connection while using HFSC (m1, d, m2) to mirror 'powerboost' and there is CPU headroom to do more. From the below: total throughput UP + DOWN = 4256445 bytes/sec or 34051560 bits / sec (34Mbit).  I have hit 37+, but didn't manage that testing just now. PFTOP on my setup -- note pftop is BYTES not BITS, so multiply x 8. pfTop: Up Queue 1-20/20, View: queue, Cache: 10000 PAUSED                                                          09:37:01 QUEUE                             BW SCH  PRIO     PKTS    BYTES   DROP_P   DROP_B QLEN BORROW SUSPEN     P/S     B/S root_vr1                       6500K hfsc    0        0        0        0        0    0                     0       0 q_Internet                    6500K hfsc             0        0        0        0    0                     0       0  q_ACK_6                      1300K hfsc         99847  9311650        0        0    0                  2406  196073  q_Default_3                   650K hfsc         50534  6710816        0        0    0                     0       0  q_VoIP_7                      260K hfsc         13948  1600184        0        0    0                     0       0  q_High_4                     1300K hfsc         99767 14564592        0        0    0                   1.0      77  q_Low_1                       325K hfsc             0        0        0        0    0                     0       0  q_WORK_5                     2600K hfsc        144461 26449303        0        0    0                    18    3193 root_vr0                         36M hfsc    0        0        0        0        0    0                     0       0 q_Internet                      36M hfsc             0        0        0        0    0                     0       0  q_ACK_6                      7200K hfsc         11044   635208        0        0    0                     0       0  q_VoIP_7                      360K hfsc         10181  1591153        0        0    0                     0       0  q_High_4                     7200K hfsc        116494  146517K      334   503654    5                  2679 4053841  q_Low_1                      1800K hfsc             0        0        0        0    0                     0       0  q_Default_3                  3600K hfsc         48156 36041316        0        0    0                   1.0     390  q_WORK_5                       14M hfsc             0        0        0        0    0                     0       0 root_vr2                         36M hfsc    0        0        0        0        0    0                     0       0 q_Internet                      36M hfsc             0        0        0        0    0                     0       0  q_ACK_6                      7200K hfsc            27     1838        0        0    0                     0       0  q_WORK_5                       14M hfsc        224248  112203K        0        0    0                    13    2871 sidenote:  Can you change pftop to bits vs bytes.  90% of the time i'm thinking in bits

Ok Josh, thx for the answers.  :)  Cya

i put again modem in bridge mode and add mikrotik. so far it works. upload is on maximum. i guess that pf has problem with mtu and packets (change tcp mss packets)?

correct

@dusan: NTP's destination port is 123 UDP and (rarely) TCP. Does NTP ever use TCP/123 ? It doesn't according to the NTP folks: "Note that NTP does not use TCP in any form. Also note that NTP requires port 123 for both source and destination ports." – http://www.eecis.udel.edu/~mills/ntp/html/debug.html

You can install squid3 High performance web proxy cache. It combines squid as a proxy server with it's capabilities of acting as a HTTP / HTTPS reverse proxy. It includes an Exchange-Web-Access (OWA) Assistant.

It should just create a default queue. The default queues don't need a rule assigned to it. All traffic we be applied to that queue. Mine just happens to be qP2P, but if you don't enable that, it should create one called Default.

@Bonline: HI thank you again for replying me :) so did I unterstood it too. would you know where I can check this into freeradius? Best regards That depends on where you store the users. If you store them in the "users" file in ../raddb/ folder then it would look like this: "Testuser" Cleartext-Password := "testpassword" WISPr-Bandwidth-Max-Up := 524288, WISPr-Bandwidth-Max-Down := 2097152 If you store it in a MySQL database - I do not use any database - this would be probably stored in any "radreply" table. If you are running freeradius in debug mode "radiusd -X" then you can check this if a user connects then the radius server must answer with the two attributes from above. If the server does not send these attributes you must configure your freeradius server.

One other suggestion, you should test your network with something like jperf (iperf gui for windows).  File transfers are not always a good judge of bandwidth limits because of all the dependencies involved.  The problem could be with the sending/receiving  hard drive, filesystem, OS, NIC, or some combination of those.  If Jperf shows a good amount of bandwidth the bottleneck is not the network. http://openmaniak.com/iperf.php#jperf - Jperf tutorial Josh

@awesomo: In your situation, your best bet is to use "realtime" ignore m1 and d and just specify the amount of bandwidth you need available to whatever traffic and set the priority to 7. It will send data out of that queue as fast as it can, always. I just wanted to point out that HSFC has no sense of Priority, the fact that there is a spot to specify priority is just a mistake in the GUI. HSFC just tries to share bandwidth fairly between all the queues.  Other than the realtime queue which gets it's bandwidth first, which you mentioned. It is my mission to make sure everyone knows that HSFC doesn't use priority. :) http://forum.pfsense.org/index.php/topic,42798.msg222507.html#msg222507 Josh

The default rules just means that it is the rule that matches everything.  Create a rule that matches all traffic and have that be the last rule in the list.  That rule should assign traffic to the limiters that manage the not reserved bandwidth.

In most cases rsync would look like ssh traffic, so if you shape/filter ssh it should catch rsync. Rsync also supports a –bwlimit=KBPS parameter so you can have the NAS box limit its own backup if you set, say --bwlimit=80 then it would go no higher than 80KByte/s (not that's Bytes, not bits.)