New Queue/Match action on floating tab?
-
Hello,
I noticed in the Git repository that there were a number of commits on Feb 11 for a new Queue/Match action on the floating tab of the firewall rules. I've searched the forums but I can't find any documentation about this new feature. Could one of the developers briefly explain it?
The reason I ask is that existing traffic shaping rules that I had created for the Feb 6 build seem to behave strangely after upgrading to a firmware that includes the new queue/match action. For example, traffic is now going into the correct queue on only one of the interfaces whereas it was working on both interfaces before.
-
I don't have an answer for you but I have noticed some breakage in firewall queues lately and reported it here: http://forum.pfsense.org/index.php/topic,33345.msg173653.html.
Maybe the devs are working on a fix?
-
I just realized my traffic shaper is failing to classify a lot of the traffic it used to. So bulk traffic is squeezing out interactive traffic, as they all land in the default queue for reasons unknown to me.
Exactly!
Previously (e.g. in Feb 6 build), for outgoing traffic to a certain destination port, I just had to add a WAN OUT rule to shape that traffic on both interfaces.
Now with later builds (e.g. Feb 18), I also need to add a LAN IN rule to that destination port to get the traffic shaped on both interfaces, otherwise it is only shaped on one of them and on the other interface it goes into default queue.
In looking at the commits for the traffic shaper, the only changes that seem relevant are the ones made on Feb 11 for the new queue/match action on the floating tab.
Perhaps we have to change existing traffic shaper rules to use action Queue instead of action Pass?
-
The match/Queue action is simply there because people cannot comprehend that the same QoS and filtering be done on same rule.
So a new action was done to only do QoS and the filter rules can than be done separately.
Now you can create Queue/match rules at will and they will not impact you filtering policy just help on queueing traffic. -
Okay, I can show with the following screenshots that something has changed in the traffic shaper behaviour between the Feb 6 build and the Feb 15 build, which was the first build after the queue/match feature was added on Feb 11.
First, using the Feb 6 build, I ran the Single Wan multi Lan traffic shaping wizard and created a qOthersHigh queue for MS RDP traffic (TCP port 3389). The wizard created a pass-out-allInterfaces rule on the floating tab for port 3389, as shown in the screenshot Feb6-OutRule.png. The queue setting in the rule was qACK/qOthersHigh.
To simulate RDP traffic, I setup a netcat server on the internet on port 3389. I then connected to it from a netcat client running behind pfSense. When I typed on the client side, the traffic got shaped into qOthersHigh on WAN and qACK on LAN, as expected (see FromClientToServerFeb6-OutRuleOnly.png). When I typed on the server side, the traffic got shaped into qOthersHigh on LAN and qACK on WAN, as expected (see FromServerToClientFeb6-OutRuleOnly.png).
Next, I upgraded pfSense to the Feb 15 build. Using the exact same pass-out-allInterfaces rule on the floating tab, my test results were different. When I typed on the client side, the traffic got shaped into qOthersHigh on WAN and qDefault on LAN (see FromClientToServerFeb15-OutRuleOnly.png). When I typed on the server side, the traffic got shaped into qDefault on LAN and qACK on WAN (see FromServerToClientFeb15-OutRuleOnly.png). So, on the LAN interface, qDefault is being used instead of qACK/qOthersHigh.
Next, I changed the action of the pass-out-allInterfaces rule to "queue", thinking that would make the traffic shaper work like in the Feb 6 build. After resetting states, I ran my test again. Unfortunately, even with the queue action the traffic still went into qDefault on the LAN interface.
Finally, I added a second rule to the floating tab. It was a queue-in-LAN rule, as shown in Feb15-LanInRule.png. After resetting states, I ran my test again. This time it worked as expected, with qACK/qOthersHigh being used on both WAN and LAN. See the queue status in screenshots FromClientToServerFeb15-InAndOutRules.png and FromServerToClientFeb15-InAndOutRules.png.
Is this a bug or are we now supposed to use two rules (i.e. queue-WAN-out and queue-LAN-in) for each port we want to shape? If this is a new feature and not a bug, perhaps the wizard should create both rules automatically.
-
@ermal:
The match/Queue action is simply there because people cannot comprehend that the same QoS and filtering be done on same rule.
So a new action was done to only do QoS and the filter rules can than be done separately.
Now you can create Queue/match rules at will and they will not impact you filtering policy just help on queueing traffic.Should we change all of our floating rules that are used for traffic shaping from Action:Pass to Action:Queue then? I did a test for HTTP traffic and I see data passing in the queue via pfTop.. Noticed the pfTop the action name for Queue is 11:
RULE ACTION DIR LOG Q IF PR K PKTS BYTES STATES MAX INFO 111 Pass In Q ah K 0 0 0 inet all queue qOthersHigh 112 Pass In Q esp K 12 1632 0 inet all queue qOthersHigh 113 Pass Out tcp K 0 0 0 from any to any port = rtsp flags S/SA queue(qOthersHigh, qACK) 114 11 Out tcp 59 3056 0 inet from any to any port = http queue(qDefault, qACK) 115 11 Out tcp 8 432 0 inet from any to any port = https queue(qDefault, qACK) 116 Pass Out tcp K 0 0 0 from any to any port = smtp flags S/SA queue(qOthersLow, qACK) 117 Pass Out tcp K 0 0 0 from any to any port = pop3 flags S/SA queue(qOthersLow, qACK) -
I did a test for HTTP traffic and I see data passing in the queue via pfTop.
Cino, is the correct queue being used on both the WAN interface and the LAN interface or only on the WAN interface? See my post above with all the screenshots.
Also, which build are you using?
-
FYI, I'm seeing the same behavior (i.e. qDefault on LAN) in the Feb 18 build. I haven't tried Feb 21 or Feb 22 yet.
-
You can remove the direction on the rule afaik and you will be safe.
I will see to make it on pfSense wizard output as well.
Thanks for the analysis. Furthermore i do not think it worked differently before with pass out rules since nothing has changed unless you had selected the queues on the lan rule as well.EDIT: Done https://rcs.pfsense.org/projects/pfsense/repos/mainline/commits/c54c9d15d3c1658e959df44125fa8a4aaee2f4d7
-
I did a test for HTTP traffic and I see data passing in the queue via pfTop.
Cino, is the correct queue being used on both the WAN interface and the LAN interface or only on the WAN interface? See my post above with all the screenshots.
Also, which build are you using?
Using the snapshot that was built on Mon Feb 21 19:15:26 EST 2011. I used the Single Lan multi Wan wizard when i first setup traffic shaping. My LAN side is using PRIQ instead of HFSC. I'll have to test more but it does look like its using the correct queue for the WAN and LAN interfaces.
I really have to sit down and do some testing… I used the wizard and add some rules to it... Usually I set all my rules to Any direction and Quick match which seems to be working for me... Is it correct, idk... Still learning traffic shaping on 2.0... Once I got it tho, I'll be trying to traffic shape my WANs: Cable(50/5), Verizon3G(backup); and 2-5 LAN interfaces... Oh and try to shape the whole OpenVPN tunnel(its not matching the que i want it to goto) and maybe the traffic inside the OpenVPN tunnel
-
@ermal:
Furthermore i do not think it worked differently before with pass out rules
It absolutely did work differently before.
With only a pass out rule (created by the wizard), the Feb 6 build put traffic into the correct queue on both WAN and LAN. On the other hand, with only a pass out rule, the Feb 15 build put traffic into the correct queue only on the WAN interface. On the LAN interface, the traffic went into qDefault. I did not change anything other than upgrading the firmware.
My workaround with the Feb 15 and Feb 18 builds was to add the additional LAN in rule. With both rules, traffic is now put in correct queues on both interfaces on Feb 15 and Feb 18 builds.
@ermal:
You can remove the direction on the rule afaik and you will be safe.
I think that will work but in a way it will expand to 4 rules (LAN in and WAN out but also WAN in and LAN out), which might not be what you want. For example, if my goal is to shape traffic leaving my network that is destined for port 3389 somewhere on the internet, I do not necessarily want to match traffic coming in the WAN destined for port 3389. What I really want to match is only LAN in and WAN out. Even if I did want to match traffic coming in the WAN destined for port 3389, I might want to use a different queue.
-
I do not think it worked differently but i am not going to convince you either.
Everything els eyou say is part of how you want to express policy and match/Queue action rules work with the last matching being the effective one.
There is no quick option for those etc….So 4 rules for one queue are a lot in first thought but that depends on configuration and needs. In no way match/Queue rules make the QoS need more rules.
-
@ermal:
You can remove the direction on the rule afaik and you will be safe.
I just tried it and I can confirm that changing the direction from out to any on the wizard-generated rule (and deleting the additional LAN in rule) does indeed work and traffic gets put into correct queues on both WAN and LAN interfaces. Thanks.
@ermal:
In no way match/Queue rules make the QoS need more rules.
Yes, perhaps the addition of match/queue rules in the Feb 15 build is not what caused the change in queuing behavior shown in the screenshots above. However, there must have been some change somewhere during the period between Feb 6 and Feb 15 that did cause the traffic to go into qDefault on LAN. I'm not crazy. Please go back and look at my screenshots if anyone doubts me.
Anyway, the point is, Ermal's change in the wizard to set the direction from out to any will work. Thanks again.
-
Even if I did want to match traffic coming in the WAN destined for port 3389, I might want to use a different queue.
I think the fundamental issue here is that the assumption of the wizard has changed. It used to be that the assumption was the wizard shaped primarily for outgoing connections. Switching the wizard to "any" really changes that assumption as it will now put incoming connections destined for the same port in the same queue.
While this is not necessarily a bad thing (though really it would help if the user had a choice in the wizard), this does fundamentally change things such that sites like mine are needing some change.
In our set-up, we have remote users from multiple sites who access a terminal server behind one router. This terminal server is at the HQ site. We want this traffic to be queued high with lots of real time so that people perceive the connection to be fast when it is busy with bulk traffic. But we also have a set of sysadmins at HQ who may remotely access desktops at the remote sites for system administration purposes. We want this traffic to be in a low priority queue. This is also on port 3389. The default rule that would be generated by the wizard as proposed here would result all traffic destined for 3389 ending up in the same queue.
This is not insurmountable - certainly one could then change the rules manually, but it is a change that seems to have side effects.
I'm not crazy. Please go back and look at my screenshots if anyone doubts me.
You're right - you are not crazy. I took a close look at the screenshots and you are correct. The behaviour did change between Feb 6 and Feb 15 builds.