Working CBQ-based traffic shaping, but …

[netsysadmin]

Hello,

After having searched and read a number of posts in this forum and other articles/tutorials on the web, I "sort of" got traffic shaping to work using CBQ.

I currently have the following setup:
1. pfSense 2.0.2-RELEASE (amd64)
2. Physical network interfaces re1 and re2 (both 1 Gbps)
3. Two VLANs on the LAN side, but only using one (interface LAN_VLAN6, which is VLAN 6 on re1)
4. "Two VLANs" on the WAN side: it's actually one VLAN (interface WAN_VLAN12, which is VLAN 12 on re2) and interface WAN_VLAN1 is actually re2
5. There are 2 Internet connections: WAN_VLAN1 is connected to a symmetric 2 Mbps line and WLAN_VLAN12 to a 4M/1M ADSL line

In order to simplify things, I decided to implement traffic shaping only on the WAN_VLAN1 (wan) and LAN_VLAN6 (lan) interfaces.

I manually created the following traffic shaper:

For uploading:
1. On interface WAN_VLAN1, I chose "CBQ" as the "Scheduler Type" and set the bandwidth to 1 Gbps.
2. I created the queue "qWAN-Internet" and set its bandwidth to 2 Mbps
3. I created the queue "qWAN-Link" on the same level and set its bandwidth to 998 Mbps, checked "Default queue" and "Borrow from other queues when available"
4. I added the queue "qWAN-TEST1" under "qWAN-Internet" with the following parameters: RED, ECN, set its bandwidth to 1200 Kbps and checked "Borrow from other queues when available"
5. I added the queue "qWAN-TEST2" under "qWAN-Internet" with the following parameters: RED, ECN, set its bandwidth to 800 Kbps

For downloading:
1. On interface LAN_VLAN6, I chose "CBQ" as the "Scheduler Type" and set the bandwidth to 1 Gbps.
2. I created the queue "qLAN-Internet" and set its bandwidth to 2 Mbps
3. I created the queue "qLAN-Link" on the same level and set its bandwidth to 998 Mbps, checked "Default queue" and "Borrow from other queues when available"
4. I added the queue "qLAN-TEST1" under "qLAN-Internet" with the following parameters: RED, ECN, set its bandwidth to 1200 Kbps and checked "Borrow from other queues when available"
5. I added the queue "qLAN-TEST2" under "qLAN-Internet" with the following parameters: RED, ECN, set its bandwidth to 800 Kbps

To shape uploads, go to "Firewall", "Rules", then click on "Floating".
Add a new rule by clicking the "+" symbol on the right. The rule I added is as follows:
1. For "Action", select "Queue"
2. For "Interface", I selected "WAN_VLAN1"
3. Direction: "Out"
4. Protocol: "TCP"
5. Source: Here I put the IP Address of my PC
6. Destination: Here I checked "Not", then Network "10.0.0.0/8" (we are on 10.0.0.0 network)
7. For Ackqueue/Queue, I chose "none/qWAN-TEST1"

Now, the strange thing is that, in order to shape downloads, I had to specify the queue on the LAN_VLAN6 firewall rule.
It did not work on the "Floating" tab!

On the "LAN_VLAN6" tab, I have the rules for allowing different LAN computers to access the Internet.
For example, I already have a rule that permits my PC to access the Internet as follows:
1. Action: "Pass"
2. Interface: "LAN_VLAN6"
3. Protocol: "TCP"
4. Source: IP Address of my PC
5. Destination: checked "Not", Network "10.0.0.0/8"

To specify the queue to use for downloads, I edited the above rule and for Ackqueue/Queue, I chose "none/qLAN-TEST1"

Tests show that both uploads and downloads are being shaped.

However, I don't understand why a rule to shape downloads did not work on the "Floating" tab.
I tried the following floating rule:
1. Action: "Queue"
2. Interface: "LAN_VLAN6"
3. Direction: "Out"
4. Protocol: "TCP"
5. Source: checked "Not", Network "10.0.0.0/8"
6. Destination: IP Address of my PC
7. Ackqueue/Queue: "none/qLAN-TEST1"

1. Can anyone shed some light on this?
2. Can anyone confirm whether the traffic shaping as I've described above is correct?

Thanks for any feedback.

[jimp]

A PC on your LAN downloading a file from the Internet would create states in two places:

IN on your LAN
OUT on your WAN

You have to match/shape where the state is created.

[netsysadmin]

Hello jimp,

Thanks a lot for your input.
I do understand that traffic shaping shoud be done at the egress interface, ie, on the LAN interface for downloading and on the WAN interface for uploading.

However, correct me if I'm wrong, on the LAN tab in the firewall rules, we specify traffic coming into the interface, not outgoing traffic.
So, I supposed that to shape downloads, I had to create a floating rule, in which I specify the interface to be the LAN interface and the direction to be "out". This did not work.
This is what I did to shape uploads, ie, I created a floating rule, with the interface set to be the WAN interface and the direction set to "out". This did work.

Maybe I haven't understood floating rules well!

Can you please explain where I'm wrong?

Thanks again.

[jimp]

You have the directionality where shaping happens correct, but what you are missing is where the states are created.

Connections from the LAN to the Internet create states only on IN on LAN and OUT on WAN, so that's where the rules go to shape that traffic. Traffic in both directions is shaped when the connection is matched.

A rule OUT on LAN will only be matched by a connection that came IN on WAN, e.g. a connection initiated by a host on the Internet coming in to a server/port forward on LAN.

The queuing will happen properly based on the queues assigned in the rules that match. The queues matter more than the directionality.

[netsysadmin]

jimp,

Thanks for your time to explain all these fundamentals.

So, it seems to me that there are 4 cases here:
1. LAN user initiates an upload to an external server
2. LAN user initiates a download from an external server
3. WAN user initiates an upload to an internal server (download from the perspective of the firewall)
4. WAN user initiates a download from an internal server (upload from the perspective of the firewall)

For cases 1 & 2, the states created are IN on LAN & OUT on WAN.
For cases 3 & 4, the states created are IN on WAN & OUT on LAN.

So far, I think this is OK.

Now, the objective is to shape all uploads and all downloads, whatever the origin of the connection.

Let's take an example for each of the 4 cases:
1. local user at IP address 10.0.0.100 on LAN initiates an HTTP upload to external server 100.101.102.103
2. local user at IP address 10.0.0.100 on LAN initiates an HTTP download from external server 100.101.102.103
3. external user at IP address 200.210.220.230 on Internet initiates an HTTP upload to internal server 10.0.0.200 (download from the perspective of the firewall)
4. external user at IP address 200.210.220.230 on Internet initiates an HTTP download from internal server 10.0.0.200 (upload from the perspective of the firewall)

Firewall rules on pfSense (Cisco-style):
On LAN tab:
permit ip host 10.0.0.100 host 100.101.102.103 (this rules caters for cases 1 & 2)
On WAN tab:
permit ip host 200.210.220.230 host 10.0.0.200 (this rules caters for cases 3 & 4)

Return traffic is dealt with by stateful pf.

Do I need to create the queues as follows for each of the 4 cases above?
1. qLAN-Upload (applied on LAN interface)
2. qLAN-Download (applied on LAN interface)
3. qWAN-Download (applied on WAN interface)
4. qWAN-Upload (applied on WAN interface)

How should I apply these queues to the rules (LAN, WAN, floating)?

LAN tab:
A. permit ip host 10.0.0.100 host 100.101.102.103 => Queue: qLAN-Upload/qLAN-Download ???

WAN tab:
B. permit ip host 200.210.220.230 host 10.0.0.200 => Queue: qWAN-Upload/qWAN-Download ???

Floating tab:
src 10.0.0.100 => dst 100.101.102.103: Action: Queue, Direction: Out, Interface: WAN, Queue: qLAN-Upload

Please help clear the confusion…

Thanks