Update and more info…

I'm running: 2.0.1-RELEASE (i386) built on Mon Dec 12 19:00:03 EST 2011 FreeBSD 8.1-RELEASE-p6...

I found that a different rule was stepping on the one above and placing it in the default queue.   (I feel a little more sane now).   Here's where I'm getting tripped up.   if I remove all floating rules and ensure that no other rules have a queue action and add a default rule for to prioritize ACK traffic things start to fall apart.

Here's a test I performed trying to understand how 'quick' performs on non-final rules (Queue only, not pass, block, reject, etc.)

Test 1: Default rules before specific 'work' rules. In this test all work 'outbound' traffic is placed in the default rule.

    pfctl -sr | grep queue     match quick on vr0 all label "USER_RULE: Default Queue - Prioritize ACK" queue(q_Default_3, q_ACK_6)     match quick on vr1 all label "USER_RULE: Default Queue - Prioritize ACK" queue(q_Default_3, q_ACK_6)     match quick on vr2 all label "USER_RULE: Default Queue - Prioritize ACK" queue(q_Default_3, q_ACK_6)     match in quick on vr1 inet from any to 192.168.1.0/24 label "USER_RULE: QoS Work (inbound)" queue q_Work_5     match in quick on vr2 inet from 192.168.1.0/24 to any label "USER_RULE: QoS Work(outbound)" queue q_Work_5     pfctl -k 192.168.0.0/16     killed 49 states from 1 sources and 0 destinations     re-establish tunnels on appliance and watch pftop

Test 2: Default rules after specific 'work' rules. In this test all work 'outbound' traffic is placed in the default rule.

    pfctl -sr | grep queue     match in quick on vr1 inet from any to 192.168.1.0/24 label "USER_RULE: QoS Work (inbound)" queue q_Work_5     match in quick on vr2 inet from 192.168.1.0/24 to any label "USER_RULE: QoS Work(outbound)" queue q_Work_5     match quick on vr0 all label "USER_RULE: Default Queue - Prioritize ACK" queue(q_Default_3, q_ACK_6)     match quick on vr1 all label "USER_RULE: Default Queue - Prioritize ACK" queue(q_Default_3, q_ACK_6)     match quick on vr2 all label "USER_RULE: Default Queue - Prioritize ACK" queue(q_Default_3, q_ACK_6)     pfctl -k 192.168.0.0/16     killed 49 states from 1 sources and 0 destinations     re-establish tunnels on appliance and watch pftop

Test 3: No Default Rules. In this test all work traffic is placed in the correct q_Work_5 queue.

    pfctl -sr | grep queue     match in quick on vr1 inet from any to 192.168.1.0/24 label "USER_RULE: QoS Work (inbound)" queue q_Work_5     match in quick on vr2 inet from 192.168.1.0/24 to any label "USER_RULE: QoS Work(outbound)" queue q_Work_5     pfctl -k 192.168.0.0/16     killed 49 states from 1 sources and 0 destinations     re-establish tunnels on appliance and watch pftop

I guess I'm confused at how 'queue' type rules work when there are multiple matches in the ruleset.  Can someone provide any clarity.

Thanks!

Do you have a specific rule that puts the various IMAP ports into a different queue?  p2pcatchall will match everything that isn't specifically matched.
Josh

Thankyou ermal, dreamslacker & Metu69salemi.
Yes i'm on v 2.1 . I saw the path now.
I will walk as per your direction. I'm sure i will reach the destination.
many thanks
Kalu

This was working and I think I hit a bug, but I am not sure exactly what it is or why it happened.

I had the initial 5 remote sites and their respective queues. I had floating rules to direct all the traffic to the queues for each site. I added in a 6th site, a 6th set of queues and a 6th set of floating rules, and now ALL open vpn traffic destined for HQ's lan is ignoring the queue assignments in the floating rules. All traffic is going to qlink or qack on the lan interface and I haven't found out why just yet.

Floating rules that apply to traffic going out the wan, or going out the lan with traffic from the lan, are still categorized to the correct queues.

I am absolutely stumped right now and this is a network in use 24/7 so I can't constantly try things to fix it. I am going to have to setup a lab on VSXi and try to figure out what the heck is going on.

That is, unless someone else out there knows?

I still haven't found a way to prioritize OSPF packets yet either since they never touch the wan. I don't think there is a way. The way I have delt with ospf packet loss was raising the dead timers to 5 minutes, far from optimal, but it works for this setup.

There's actually 2 parts to traffic shaping - the shaper queues that determine what should happen with the traffic that's put into them, and floating firewall rules that assign traffic to the queues. What you need to do is split the firewall rule that assigns those ports into 2 (or more) separate ranges. You'll find the rules on the Floating tab in the Firewall Rules menu.

What you want to end up with is a rule that does (suppose you want to exclude port 4000) port 3000-3999 and a second rule that does 4001-32000.

@cmb:

it gets a bit complicated because part of your shaping has to accommodate the fact it's ESP traffic on WAN.

Ah, I see …

I wonder if it would be possible to copy the ToS byte from the original IP header to the new ESP header (or perhaps it's being done already?) In Cisco it's done by default, it's called the “ToS Byte Preservation” feature.

Edit: Based on a quick Google search, there seems to be a system tunable net.inet.ipsec.ah_cleartos that is set by default, but I don't see a corresponding ESP tunable.

ALIX 2d13< – same as 2D3 with an added port header i believe, same hardware performance specs

I can Pull 37mbit on my comcast connection while using HFSC (m1, d, m2) to mirror 'powerboost' and there is CPU headroom to do more.

From the below: total throughput UP + DOWN = 4256445 bytes/sec or 34051560 bits / sec (34Mbit).  I have hit 37+, but didn't manage that testing just now.

PFTOP on my setup -- note pftop is BYTES not BITS, so multiply x 8.

pfTop: Up Queue 1-20/20, View: queue, Cache: 10000 PAUSED                                                          09:37:01 QUEUE                             BW SCH  PRIO     PKTS    BYTES   DROP_P   DROP_B QLEN BORROW SUSPEN     P/S     B/S root_vr1                       6500K hfsc    0        0        0        0        0    0                     0       0 q_Internet                    6500K hfsc             0        0        0        0    0                     0       0  q_ACK_6                      1300K hfsc         99847  9311650        0        0    0                  2406  196073  q_Default_3                   650K hfsc         50534  6710816        0        0    0                     0       0  q_VoIP_7                      260K hfsc         13948  1600184        0        0    0                     0       0  q_High_4                     1300K hfsc         99767 14564592        0        0    0                   1.0      77  q_Low_1                       325K hfsc             0        0        0        0    0                     0       0  q_WORK_5                     2600K hfsc        144461 26449303        0        0    0                    18    3193 root_vr0                         36M hfsc    0        0        0        0        0    0                     0       0 q_Internet                      36M hfsc             0        0        0        0    0                     0       0  q_ACK_6                      7200K hfsc         11044   635208        0        0    0                     0       0  q_VoIP_7                      360K hfsc         10181  1591153        0        0    0                     0       0  q_High_4                     7200K hfsc        116494  146517K      334   503654    5                  2679 4053841  q_Low_1                      1800K hfsc             0        0        0        0    0                     0       0  q_Default_3                  3600K hfsc         48156 36041316        0        0    0                   1.0     390  q_WORK_5                       14M hfsc             0        0        0        0    0                     0       0 root_vr2                         36M hfsc    0        0        0        0        0    0                     0       0 q_Internet                      36M hfsc             0        0        0        0    0                     0       0  q_ACK_6                      7200K hfsc            27     1838        0        0    0                     0       0  q_WORK_5                       14M hfsc        224248  112203K        0        0    0                    13    2871

sidenote:  Can you change pftop to bits vs bytes.  90% of the time i'm thinking in bits

Ok Josh, thx for the answers.  :)  Cya

i put again modem in bridge mode and add mikrotik.
so far it works. upload is on maximum.
i guess that pf has problem with mtu and packets (change tcp mss packets)?

correct

@dusan:

NTP's destination port is 123 UDP and (rarely) TCP.

Does NTP ever use TCP/123 ?

It doesn't according to the NTP folks:

"Note that NTP does not use TCP in any form. Also note that NTP requires port 123 for both source and destination ports." – http://www.eecis.udel.edu/~mills/ntp/html/debug.html

You can install squid3

High performance web proxy cache. It combines squid as a proxy server with it's capabilities of acting as a HTTP / HTTPS reverse proxy. It includes an Exchange-Web-Access (OWA) Assistant.

It should just create a default queue. The default queues don't need a rule assigned to it. All traffic we be applied to that queue. Mine just happens to be qP2P, but if you don't enable that, it should create one called Default.

@Bonline:

HI thank you again for replying me :)

so did I unterstood it too.

would you know where I can check this into freeradius?

Best regards

That depends on where you store the users. If you store them in the "users" file in ../raddb/ folder then it would look like this:

"Testuser" Cleartext-Password := "testpassword" WISPr-Bandwidth-Max-Up := 524288, WISPr-Bandwidth-Max-Down := 2097152

If you store it in a MySQL database - I do not use any database - this would be probably stored in any "radreply" table.

If you are running freeradius in debug mode "radiusd -X" then you can check this if a user connects then the radius server must answer with the two attributes from above. If the server does not send these attributes you must configure your freeradius server.

One other suggestion, you should test your network with something like jperf (iperf gui for windows).  File transfers are not always a good judge of bandwidth limits because of all the dependencies involved.  The problem could be with the sending/receiving  hard drive, filesystem, OS, NIC, or some combination of those.  If Jperf shows a good amount of bandwidth the bottleneck is not the network.

http://openmaniak.com/iperf.php#jperf - Jperf tutorial

Josh

@awesomo:

In your situation, your best bet is to use "realtime" ignore m1 and d and just specify the amount of bandwidth you need available to whatever traffic and set the priority to 7. It will send data out of that queue as fast as it can, always.

I just wanted to point out that HSFC has no sense of Priority, the fact that there is a spot to specify priority is just a mistake in the GUI.

HSFC just tries to share bandwidth fairly between all the queues.  Other than the realtime queue which gets it's bandwidth first, which you mentioned.

It is my mission to make sure everyone knows that HSFC doesn't use priority. :)

http://forum.pfsense.org/index.php/topic,42798.msg222507.html#msg222507

Josh

The default rules just means that it is the rule that matches everything.  Create a rule that matches all traffic and have that be the last rule in the list.  That rule should assign traffic to the limiters that manage the not reserved bandwidth.