A definitive, example-driven, HFSC Reference Thread
-
Thanks sideout, georgeman.
A couple things:
georgeman:
I know you specify this in the text, but in the floating match rules we still do not apply "quick" right? This means that the qBulk rule has to come before the more specific qDNS rule. One might be misled by the order in the text.
sideout/all:
The realtime queue you're discussing only applies when there's contention/congestion right? If I specify 30% realtime for qDNS and there is no DNS traffic being placed in the queue, the other queues are not absentmindedly robbed of the 30% bandwidth, if I understand things.
This has all been a really big help. I appreciate it. I am currently implementing the solution provided by georgeman on my bench. More later.
-
From my understanding - RT means that X% is taken automatically for that queue even if there is no traffic in the queue. At least that is the way it reads to me. For what I use it for , qGaming that is a non issue as my use of HFSC is with traffic shaping at LAN parties so qGaming is my highest priority queue. I use the following queues:
LAN
qInternet - bandwidth - 50MBit - I have 4 cable modems but putting in 200Mbit here would not be the correct thing to do as I cannot bond them to get 200Mbit
qGaming - gaming traffic - RT 30% / bandwidth - 40% / LS - 40%
qHTTPSTeam - bandwidth - 20% / LS 20%
qWEBTraffic - bandwidth - 20% / LS 20%
qACK - bandwidth - 20% / LS 20%qLink - bandwidth - 20% / LS - 20% - default queue
WAN - 5MBIT - I have 4 WAN's so each are 5Mbit upload
qLink - bandwidth - 10% / LS - 10% - Default queue
qGaming - bandwidth 40% / RT - 20% / LS 40%
qHTTPSteam - bandwidth - 20% / LS 20%
qWEBTraffic - bandwidth - 20% / LS 20%
qACK - bandwidth - 10% / LS 10%I use the floating rules to put DNS in the qGaming so it gets good response. General web traffic goes into qWEBTraffic and then I use rules to put Steam traffic into qHTTPSteam.
I use interface rules to direct gaming traffic out different wans via gateway groups. I allocate one modem / wan to like LoL gaming traffic , another to BF4 . I dedicate one modem to strictly web traffic and another modem is reserved for staff use and downloads as I have a limiter on for DHCP addresses to restrict all TCP connections to typically 25Mbit for everyone.This way I can get the best ping times and still give everyone bandwidth to do what they need without being too restrictive.
-
I'd like to see your ruleset if it isn't too much trouble.
-
Okay. This is all coming along nicely.
I have created the following queues:
WAN
qLink default bw 20% ls 20%
qInternet bw 15Mb ul 15Mb ls 15Mb
qDNS bw 5% rt 5% ls 5%
qACK bw 10% ls 10%
qVPN bw 10% rt 5% ls 10%
qBulk bw 50% ls 50%
qOpenWireless bw 2Mb ul 2Mb ls 2MbLAN
qLink default bw 20% ls 20%
qInternet bw 50Mb ul 50Mb ls 50Mb
qDNS bw 5% rt 5% ls 5%
qACK bw 10% ls 10%
qVPN bw 10% rt 5% ls 10%
qBulk bw 50% ls 50%OPENWIRELESS
qLink default bw 20% ls 20%
qInternet bw 10% ul 10Mb ls 10Mb
qOpenWireless bw 50% ls 50%The screen shot details the floating rules. All are !quick on WAN direction OUT.
The only thing that's not going to the right queue are connections from OPENWIRELESS into the qOpenWireless on WAN.
I have reset states.
I have told my pass any any rule on OPENWIRELESS to queue to qOpenWireless. That gets downloads into the right queue on the OPENWIRELESS interface but uploads are still going to qBulk.
I am assuming this is because the floating rule on WAN OUT is happening post-NAT so the match on the source address is failing. I just fixed this by, instead of my pass any any rule on OPENWIRELESS setting the queue to qOpenWireless, I instead mark the packet with "OW" and use that in a floating rule on WAN OUT to match on "OW" and place the traffic into qOpenWireless. Seems to work.
![Screen Shot 2014-07-27 at 11.15.48 AM.png](/public/imported_attachments/1/Screen Shot 2014-07-27 at 11.15.48 AM.png)
![Screen Shot 2014-07-27 at 11.15.48 AM.png_thumb](/public/imported_attachments/1/Screen Shot 2014-07-27 at 11.15.48 AM.png_thumb) -
@KOM:
I'd like to see your ruleset if it isn't too much trouble.
I will post up my latest rule set on Monday when I pull it off my lab firewall.
-
Here are my FW rules and alias's. You need the Alias's as well to use my rule set.
https://www.dropbox.com/s/y7dtifw12y6ghmx/fwrulesalias.zip
-
@KOM:
In your example, what is the relationship between qLink and qInternet as far as bandwidth is concerned? They are both at the same level, but qLink has 20% and qInternet has 95% for a total of 115%. I don't know how that is.
No, again, do not set "95%" for the value, but the real downlaod/upload speed multiplied by 0.95 aprox
Consider that the interface speed is usually either 100 Mbps or 1000 Mbps, while the real download speed is, let's say, 5 Mbps. In this case, you would set the m2 and bandwidth values of the qInternet queue at 4.75 Mbps.
Then, if you set qLink at 20% that would be 100 Mbps x 0.2 = 20 Mbps.
So the sum of the bandwidth assigned to both root queues is actually 24.75 Mbps (< 100 Mbps)We say that the 20% value for the qLink queue doesn't matter because in this examples, it is destined just for local traffic, there is not upperlimit on it and usually the qInternet values are way lower than the interface speed. If this is not the case, or if you want to be strictly accurate, you would need to set the qLink value to the difference between the interface speed and qInternet values, so the sum of them adds up to 100% (or the interface speed)
Anyway, linkshare values are not absolute values, but relative to each other. It doesn't matter if they don't add up to 100%, what matter is the proportions between them. If you set ls to 20 on one queue and to 1 on another, that means that ls will try to give the first queue 20 times more bandwidth than the second one (when the link is saturated)
georgeman:
I know you specify this in the text, but in the floating match rules we still do not apply "quick" right? This means that the qBulk rule has to come before the more specific qDNS rule. One might be misled by the order in the text.
Exactly. Traffic will first match the qBulk, then the qDNS too, and will stick to the last one that matched
sideout/all:
The realtime queue you're discussing only applies when there's contention/congestion right? If I specify 30% realtime for qDNS and there is no DNS traffic being placed in the queue, the other queues are not absentmindedly robbed of the 30% bandwidth, if I understand things.
Yep, as you say. By specifying realtime on one queue, if that bandwidth isn't being used it is still available to other queues.
–------
There are tons of factors and caveats to consider for an accurate implementation. For example, RT considers the service curve since traffic started, so it could be "penalized" on high-usage times if it was exceeded during non-peak times and we are not using linkshare (not what we want!)
This is why some people (I tend to agree with this), suggest to use RT only to fulfill latency requirements and not bandwidth requirements (which can be handled by linkshare). And when you use RT, set the service curve on LS to the same values as RT (to account for the above scenario)
-
Ok. Moving on to the OpenVPN prioritization.
My Site-to-Site OpenVPN to the office is on server aliased to work_vpn UDP 1195.
I have qVPN on WAN and LAN set at bw 10% rt 5% ls 10%
Floating rule: WAN out dest work_vpn UDP 1195 none/qVPN
That places traffic sent to the VPN in qVPN but none of the return traffic is going into qVPN on LAN.
I haven't been able to get traffic received through the VPN into qVPN on LAN.
I have tried
Floating: LAN out source remote_vpn_lan any none/qVPN
Floating: WAN in source work_vpn UDP 1195 none/qVPNI know that I can't apply queues to virtual interfaces (OpenVPN) only physical. Not sure what I need to do here.
Edited:
I think I solved this with the following rules:
Floating Match LAN in any source any dest remote_vpn_lan none/qVPN
Floating Match WAN out UDP source any dest work_vpn 1195 none/qVPNIt looks like one of the necessary concepts to grasp is your rules have to be implemented so they catch the traffic at the point of state creation.
It looks like this also works:
Floating Match OpenVPN any any source any dest any none/qVPN
Floating Match WAN out UDP source any dest work_vpn 1195 none/qVPNI would think that the former could be used to queue a specific VPN out the LAN interface and the latter would be an easy way to do the same with all OpenVPN traffic.
-
We say that the 20% value for the qLink queue doesn't matter because in this examples, it is destined just for local traffic, there is not upperlimit on it and usually the qInternet values are way lower than the interface speed. If this is not the case, or if you want to be strictly accurate, you would need to set the qLink value to the difference between the interface speed and qInternet values, so the sum of them adds up to 100% (or the interface speed)
Hello Mr. Georgeman,
How the local traffic is directed through qLINK ? is there any floating/lan/wan rule I need to apply?
Regards,
CP
-
Hello phoenixsampras,
I suppose that, given we've made qLink the default queue, anything that does not match the other queues, go into the qLink queue "by default".
-
Hello all,
I have a few queries:
1. I need some clarifications regarding the relationship between incoming/outgoing connections and downloading/uploading.
I understand that, whether a connection is incoming or outgoing depends on the location/interface where the associated state is first created.
So, if this is correct, then we can say that incoming or outgoing connections are not related to downloading or uploading.
As an example, a local FTP client makes an outgoing connection to a remote public FTP server, but afterwards, a download or an upload can be made.
Similarly, a remote FTP client makes an incoming connection to a local FTP server, but afterwards, a download or an upload can be made.
Is my understanding correct?2. From what I've understood (as Derelict also mentioned), in order for traffic shaping to work and for the packets to be placed in the correct queue, the rule should be on the interface where the state is first created. Can anyone confirm this?
3. When we create and define the queues in "Firewall->Traffic Shaper" in pfSense, queues created on the LAN interface shape downloads and queues created on the WAN interface shape uploads. Is this correct?
Thanks for any help.
-
Hello all,
I have a few queries:
1. I need some clarifications regarding the relationship between incoming/outgoing connections and downloading/uploading.
I understand that, whether a connection is incoming or outgoing depends on the location/interface where the associated state is first created.Yes, but this can be either the ingress or the egress interface for the state. Here's an example using the diagram at the top of the thread and FTP. Say you have an FTP client on LAN connecting to an internet FTP site. You can either set the queue with a firewall rule on LAN, or a floating rule on WAN out. Like georgeman and sideout have indicated, it's a lot easier to shape on WAN out. This is because you probably do not want to shape FTP from, say, LAN to DMZ so you can either have a lot of rules for LAN governing what is shaped and what is not in or just put it on floating WAN out with qLink the default queue on LAN.
So, if this is correct, then we can say that incoming or outgoing connections are not related to downloading or uploading.
As an example, a local FTP client makes an outgoing connection to a remote public FTP server, but afterwards, a download or an upload can be made.Yes. When either interface matches and sets up a state, the queue on the other interface of the same name is also set. In this example qFTP on WAN will shape traffic out of WAN and qFTP on LAN will shape out of LAN. So your LAN queue will regulate "downloads" and your WAN queue will regulate "uploads", regardless of how the state was created.
Similarly, a remote FTP client makes an incoming connection to a local FTP server, but afterwards, a download or an upload can be made.
Is my understanding correct?Yes. But for inbound sessions to a local FTP server, you probably want to set the queues on the firewall rule on WAN that allows such connections in in the first place. As georgeman mentioned, you have to have the rule anyway and it will only apply to WAN traffic.
2. From what I've understood (as Derelict also mentioned), in order for traffic shaping to work and for the packets to be placed in the correct queue, the rule should be on the interface where the state is first created. Can anyone confirm this?
Not exactly. They have to be on an interface that is included in the initial state creation. Like with the outbound FTP session example above, it will work with a floating rule on WAN out even though the session is actually initiated from LAN. This way it will not impact ftp sessions from, say, LAN to DMZ which you probably want to shape differently if at all.
3. When we create and define the queues in "Firewall->Traffic Shaper" in pfSense, queues created on the LAN interface shape downloads and queues created on the WAN interface shape uploads. Is this correct?
You might be better off thinking in terms of flow direction. The shaper shapes traffic going OUT the interface on which the queue is defined. Someone outside could be "uploading" to your local FTP server on LAN, but that "upload" would be shaped by the queue on the LAN interface.
Also remember that rules created on interface tabs only apply to states created coming IN that interface. The only way to create rules to catch states being created going OUT an interface is with a floating rule.
Thanks for any help.
Hope I haven't misled you here.
Someone please correct me if I made any mistakes. I'm learning too.
-
…
I agree on pretty much everything mentioned here ;)
As regards the OpenVPN prioritization mentioned before, in fact I have never tried to do it on an OpenVPN site-to-site tunnel. I can tell that I don't think you can shape within the tunnel in case of (at least) roadwarrior connections since the packets are seen encrypted out of the WAN interface and the queue selections do not seem to be kept (on IPsec they are, but because it is hooked up to the kernel I guess).
As soon as I can we can continue to elaborate on HFSC (since all this was more about the general shaper config)
Cheers!
-
To keep this simple, always try that the sum of the linkshare values from the children queues sum up to the value of the parent queue. This is because HSFC uses a "sustractive" method for the percentages (I can elaborate of this later).
Could you elaborate on this? Your other information has helped me learn more about this.
-
I've been quite busy lately, but I would like to keep helping here :)
To keep this simple, always try that the sum of the linkshare values from the children queues sum up to the value of the parent queue. This is because HSFC uses a "sustractive" method for the percentages (I can elaborate of this later).
Could you elaborate on this? Your other information has helped me learn more about this.
A CBQ parent queue assigns a generic "100%" of its bandwidth to be shared by its children. With HFSC, the percentage is an absolute value, not a fraction of the parent.
Best way to understand this is to analyze this example, taken from the book "Building Firewalls with OpenBSD and PF - 2nd Edition", by Jacek Artymiak
While CBQ uses a 'proportional' method, HFSC uses a 'subtractive' method. To see how it works in practice, compare the following rules, which divide bandwidth in the same way, yet the percentage notation is different:
CBQ
altq on $ext_if cbq bandwidth 20Mb
queue{dmznet, prvnet, others}prvnet gets 8Mb
queue prvnet bandwidth 40% queue{host1, host2}
host1 gets 4Mb
queue host1 bandwidth 50%
host2 gets 4Mb
queue host2 bandwidth 50%
–--
HFSC
altq on $ext_if hfsc bandwidth 20Mb
queue{dmznet, prvnet, others}prvnet gets 8Mb
queue prvnet hfsc(linkshare 40%) queue{host1, host2}
host1 gets 4Mb
queue host1 hfsc(linkshare 20%)
host2 gets 4Mb
queue host2 hfsc(linkshare 20%)
Basically, the 40% assigned to the parent is divided (50%-50%) on CBQ (relative percentage), while is (20%-20%) on HFSC (absolute percentage)
Best regards!
-
I was wondering if anyone here can lend a hand.
I've read this thread line by line - there's a lot of great info here, but I still can't get my QoS to work.
Everything is still on the default queues for each interface.I'm using CBQ, but it's more or less the same setup.
I have 3 WANS, and 1 LAN.
The WANs are:
WAN
PIAUS <– OpenVPN to a Canadian server
PIACA <-- OpenVPN to a US server
(I've attached a screen shot of the how the queues are setup).
I made sure to divvy up the BW properly, set the right queues to default, etc...Easy enough so far...
The I go to make the floating rules (see attachments).
I want to prioritize entire host machines, not ports. I have dedicated VMs on my network that I give certain tasks to (torrent downloading, usenet downloading, etc... ) so I want to be able to route all traffic from certain hosts to certain queues.
I made aliases: queHigh, queMed, queLow to put the hosts in.Setup the floating rules as instructed, except for the small difference that I'm using more than 1 WAN, so I chose all 3 WANs as the Interface (with direction: OUT) in each of these rules. Then I picked the alias as the Source address. Not sure if this is correct, but I tried it in Source and Destination and neither worked.
So, on to my LAN interface rules (see screen shot).
Here's where I send hosts on my LAN to the specific gateway I want them to leave on, either one of the VPNs or unencrypted on WAN.Thing is, at one point I had this working... then just the other day I did a full reinstall of pfSense in order to install the 64-bit version.
It was so long ago now that I got it up and running that I can't remember what the heck I did!!I really hope someone can lend some advice here, I've been researching all day and still haven't found a solution.
Thanks in advance!!!
-
This thread is about HFSC.
-
Yeah, I get that… but as far as the firewall rules are concerned, are they not very similiar, if not, exactly the same?
-
Yes, HFSC vs CBQ vs PRIQ is just the underlying algorithm. The rules and queue defs are mostly the same with some exceptions like bandwidth definitions and allocation.
Impossible to tell whats going on without seeing your rule definitions. Your floating rules look funky. You have aliases that look like queue names as the Source. Here is my Floating Rules page as a simple example.
-
Hey KOM, thanks for the reply and screen shot.
I've done a bit more reading and found this thread: https://forum.pfsense.org/index.php?topic=61106.0In it, it's stated (regarding floating rules), "Note that NAT has happened before the rules apply so you can't match on a private IP source that has gone through NAT, you have to match on the destination or the translated source."
This is something I've always wondered but could never find an answer to. What order do things happen in pfSense?
LAN traffic -> into pfSense firewall -> LAN interface rules -> NAT -> floating rules -> desired gateway -> out of pfSense firewall -> modem / internet ?
If I had a better understanding of the "signal path", so to speak, I'm sure I could figure this out.In my LAN interface rules I have rules that put certain hosts on my LAN into certain gateways - either one of the two VPNs or the WAN.
Does this happen before the floating rules are applied? If so, the "source" in my floating rules wouldn't be my private LAN IP addresses like I've done there, because the address would already have been translated to the WAN or VPN interface.Am I getting this right?
So, if I want to send certain LAN hosts to: a) a specific gateway and b) a certain traffic shaping queue… I have to... ?
Make LAN interface rules for every combination of priority queue and gateway, then place the hosts into the alias that would correspond to that rule?
Oh, and regarding my aliases that look like traffic queues - I name them that way (high, low, med) and then place the IPs of the hosts in them that I want ALL their traffic to be. I don't do anythng at a port level.