PfSense RC3 - Traffic Shaper Issues in resent builds

Rhongomiant

I am using the AMD64 builds of pfSense 2.0 RC3. I have the same build running on an dedicated hardware and in a VM with CARP configured on both. The dedicated hardware was installed with a build of AMD64 RC2. When I setup CARP, I updated the dedicated hardware to the following build and installed the VM using pfSense-2.0-RC3-amd64-20110708-1843.iso. I have updated three times since the install. The current build on both is 2.0-RC3 Built On: Sun Jul 24 04:39:44 EDT 2011.

I noticed that my transfers were slow on the build I was on which was a build between pfSense-2.0-RC3-amd64-20110708-1843 and pfSense-Full-Update-2.0-RC3-amd64-20110724-0242. Connections seemed to be limited to 5Mbps (512KBps). This limitation was the same for uploads and downloads between all interfaces and the WAN as well as to other interfaces. My WAN connection is a physical link to to my gateway device and I have a 50/5 internet connection. My other interfaces are VLANs provided via 2 teamed GigE connections to a Cisco switch teamed with LACP. I do have one other interface that is a direct GigE connection to the Cisco switch. This is my management interface. This transfer limitation was the same no matter the interface I was on and was the same for the primary and backup pfSense devices.

My first step was to upgrade to pfSense-Full-Update-2.0-RC3-amd64-20110724-0242, but the issue remained. I could find no obvious reason for this limitation, but it only occurred when traffic has to pass through pfSense to get to the destination. Therefore, I removed the traffic shaper entries and my performance was at expected levels. I could pass 1.2Gbps (150Mbps) or more through the teamed connection with my VLANs while downloading at 50Mbps from my WAN connection.

I figured I may just need to reset my traffic shaper entries, so I ran the traffic shaper wizard, Single Wan multi Lan, and I get errors.

After it creates the entries, I find that all the interfaces other than WAN interface have a qLink queue rather than the qDefault queue. The qLink queue is set as the default queue, but is set to priority 1 which conflicts with the qP2P queue. I would receive an alert telling me about this conflict.
I changed the priority to 3 for all the qLink queues and applied the changes. After a bit I would receive an alert that the rules need to be explicit.

I am not having this issue with build 2.0-RC3 (amd64) built on Fri Jul 8 02:49:42 EDT 2011 which I have running on hardware I have at home. I used the same options in the traffic shaper wizard on both devices.

Rhongomiant

The following post shows the type of error I am getting when I correct the priority of the qLink queue.

http://forum.pfsense.org/index.php/topic,32833.msg202726.html#msg202726

There were error(s) loading the rules: /tmp/rules.debug:144: direction must be explicit with rules that specify routing/tmp/rules.debug:145: direction must be explicit with rules that specify routing /tmp/rules.debug:146: direction must be explicit with rules that specify routing /tmp/rules.debug:147: direction must be explicit with rules that specify routing /tmp/rules.debug:148: direction must be explicit with rules that specify routing /tmp/rules.debug:149: direction must be explicit with rules that specify routing /tmp/rules.debug:150: direction must be explicit with rules that specify routing /tmp/rules.debug:151: direction must be explicit with rules that specify routing /tmp/rules.debug:152: direction must be explicit with rules that specify routing /tmp/rules.debug:153: direction must be explicit with rules that specify routing /tmp/rules.debug:154: direction must be explicit with rules that specify routing /tmp/rules.debug:155: direction must be explicit with rules that specify routing /tmp/rules.debug:156: direction must be explicit with rules that specify routing /tmp/rules.debug:157: direction must be explicit with rules that specify routing /tmp/rules.debug:158: direction must be explicit with rules that specify routing /tmp/rules.debug:159: direction must be explicit with rules that specify routing /tmp/rules.debug:160: direction must be explicit with rules that specify routing /tmp/rules.debug:161: direction must be explicit with rules that specify routing /tmp/rules.debug:162: direction must be explicit with rules that specify routing /tmp/rules.debug:163: direction must be explicit with rules that specify routing /tmp/rules.debug:164: direction must be explicit with rules that specify routing /tmp/rules.debug:165: direction must be explicit with rules that specify routing /tmp/rules.debug:166: direction must be explicit with rules that specify routing /tmp/rules.debug:167: direction must be explicit with rules that specify routing /tmp/rules.debug:168: direction must be explicit with rules that specify routing /tmp/rules.debug:169: direction must be explicit with rules that specify routing /tmp/rules.debug:170: direction must be explicit with rules that specify routing /tmp/rules.debug:171: direction must be explicit with rules that specify routing /tmp/rules.debug:172: direction must be explicit with rules that specify routing /tmp/rules.debug:173: direction must be explicit with rules that specify routing /tmp/rules.debug:174: direction must be explicit with rules that specify routing /tmp/rules.debug:175: direction must be explicit with rules that specify routing /tmp/rules.debug:176: direction must be explicit with rules that specify routing /tmp/rules.debug:177: direction must be explicit with rules that specify routing /tmp/rules.debug:178: direction must be explicit with rules that specify routing /tmp/rules.debug:179: direction must be explicit with rules that specify routing /tmp/rules.debug:180: direction must be explicit with rules that specify routing /tmp/rules.debug:181: direction must be explicit with rules that specify routing /tmp/rules.debug:182: direction must be explicit with rules that specify routing /tmp/rules.debug:183: direction must be explicit with rules that specify routing /tmp/rules.debug:184: direction must be explicit with rules that specify routing /tmp/rules.debug:185: direction must be explicit with rules that specify routing /tmp/rules.debug:186: direction must be explicit with rules that specify routing /tmp/rules.debug:187: direction must be explicit with rules that specify routing /tmp/rules.debug:188: direction must be explicit with rules that specify routing /tmp/rules.debug:189: direction must be explicit with rules that specify routing /tmp/rules.debug:190: direction must be explicit with rules that specify routing /tmp/rules.debug:191: direction must be explicit with rules that specify routing /tmp/rules.debug:192: direction must be explicit with rules that specify routing /tmp/rules.debug:193: direction must be explicit with rules that specify routing pfctl: Syntax error in config file: pf rules not loaded - The line in question reads [144]: match on { em0 } reply-to ( em0 xxx.yyy.1.31 ) proto tcp from any to any port 3389 queue (qOthersDefault,qACK) label "USER_RULE: m_Other MSRDP outbound" …
This page will automatically refresh every 3 seconds until the filter is done reloading.

eri--

Try tomorrow snapshots should be ok.

grazman

Today's build has the same error. Will this be fixed before the release is declared stable? I've not seen shaping working this year in 2.x.

eri--

Please wait for a new snapshot to come and do not hijack people's threads.
If you are not happy with shaping put some resources on it.

grazman

Hijack. It was said this issue (this thread) would be fixed in today's build. I'm saying today's build did not fix the issue.

Point me to a tracker link instead of assuming it was a hijack (it was not).

As for the resources, it has been suggested to me (and others) many times to let the key person responsible for shaping to come back from vacation, etc., to fix it. If resources are needed it would be good to have the tracker (redmine, whatever the heck you guys use) to follow, comment and help on it.

Link please.

Attitude not warranted I think.

neewbie

hi …..

traffic shaper could run normally.

updater pfSense-Full-Update-2.0-RC3-i386-20110728-2121.tgz

thks.

grazman

@neewbie:

hi …..

traffic shaper could run normally.

updater pfSense-Full-Update-2.0-RC3-i386-20110728-2121.tgz

thks.

Tried that build, and the one from today. Remove shaper and re-ran wizard, no workie.

–--------------------------------------------------
Current Version : 2.0-RC3 Latest Version : Fri Jul 29 06:00:10 EDT 2011

Jul 29 15:34:30 php: : The command '/sbin/pfctl -o basic -f /tmp/rules.debug' returned exit code '1', the output was 'bandwidth for qInternet higher than interface /tmp/rules.debug:43: errors in queue definition parent qInternet not found for qACK /tmp/rules.debug:44: errors in queue definition parent qInternet not found for qVoIP /tmp/rules.debug:45: errors in queue definition pfctl: Syntax error in config file: pf rules not loaded'
Jul 29 15:34:30 php: : New alert found: There were error(s) loading the rules: bandwidth for qInternet higher than interface /tmp/rules.debug:43: errors in queue definition parent qInternet not found for qACK /tmp/rules.debug:44: errors in queue definition parent qInternet not found for qVoIP /tmp/rules.debug:45: errors in queue definition pfctl: Syntax error in config file: pf rules not loaded The line in question reads [43]: queue qInternet on le1 bandwidth 16Mb hfsc ( ecn , linkshare 16Mb , upperlimit 16Mb ) { qACK, qVoIP }
Jul 29 15:34:30 php: : There were error(s) loading the rules: bandwidth for qInternet higher than interface /tmp/rules.debug:43: errors in queue definition parent qInternet not found for qACK /tmp/rules.debug:44: errors in queue definition parent qInternet not found for qVoIP /tmp/rules.debug:45: errors in queue definition pfctl: Syntax error in config file: pf rules not loaded - The line in question reads [43]: queue qInternet on le1 bandwidth 16Mb hfsc ( ecn , linkshare 16Mb , upperlimit 16Mb ) { qACK, qVoIP }

jlepthien

I've also never seen the shaper actually work in 2.x…
Also there is still no real documentation on how to configure that beast. Even on Cisco Routers QoS is easier to configure and it will work as expected afterwards...

eri--

grazman what is the link speed of you rinterface?

grazman

@ermal:

grazman what is the link speed of you rinterface?

16down/2up. At least thats what I am telling it for this configuration.

grazman

@grazman:

@ermal:

grazman what is the link speed of you rinterface?

16down/2up. At least thats what I am telling it for this configuration.

Why does the wizard create the queue's like this?

WAN
qInternet
qACK
qDefault
qVoIP

LAN
qLink
qInternet
qACK
qVoIP

This looks lopsided.

zcache

Hi all,

Many time before I never success for traffic sharp, today with built on Fri Jul 29 14:40:48 EDT 2011, I can finish make a traffic sharp wizard without any error notice,

Thank you to team dev.

Rhongomiant

@ermal:

Please wait for a new snapshot to come and do not hijack people's threads.
If you are not happy with shaping put some resources on it.

ermal,

I apologize, but I was not able to test until today. I updated to 2.0-RC3 (amd64)
built on Fri Jul 29 22:14:50 EDT 2011. I did not have the issues where two queues have the same priority or the "direction must be explicit with rules that specify routing " error. However, I am still having issues with performance. Previously all vlan to vlan traffic was being limited to 5Mbps. Now the limit is 10Mbps (a little more than 1MB/s). I set the WAN download to 50Mbps and WAN upload to 5Mbps, but changing these values do not seem to have an effect.

Without Traffic Shaper settings, I have been able to pass 1.6Gbps from vlan to valn through this firewall. Computers with GigE connections can transfer data at up to 940Mbps (112MB/s).

Again I am not seeing the same on my personal pfSense firewall, but I am running 2.0-RC3 (amd64) built on Fri Jul 8 02:49:42 EDT 2011 which still uses the qDefault as the default queue which is set to a priority of 3 for all interfaces. The only other obvious difference is that I am not using Outbound NAT or CARP Virtual IPs.

On another note, I am unclear on the benefit of the change from qDefault to qLink on all interfaces except the WAN interface in that the qLink queue is given a lower priority than qOthersLow. I would think that the purpose of qOthersLow is to set something lower than the default.

I hope this update helps.

eri--

Please do not cross post, http://redmine.pfsense.org/issues/1734.

It makes it difficult to follow-up.
For me its a configuration tuning and not a bug.
As far as i can see you are using PRIQ discipline.

Please follow the instrunctions on the ticket

Can you please check that putting the bandwidth of the physical interface on the root queues(with interface names) on all the LAN interfaces helps you with this issue?

Rhongomiant

ermal,

You are correct, I am using the PRIQ discipline.

Initial testing indicates that adding a value to the Bandwidth field for an interface does allow increased performance. Again this is initial testing, so I am not 100% sure that there are no performance issues. I will do more through testing ASAP.

As far as the bug or config argument, I understand your point of view. However, from my perspective, something has changed as I do not have to set the bandwidth field on my install of 2.0-RC3 (amd64) built on Fri Jul 8 02:49:42 EDT 2011 to have good performance with Traffic Shaper enabled. So the questions are as follows.

What has changes that requires the bandwidth field be set in resent builds, but not in 2.0-RC3 (amd64) built on Fri Jul 8 02:49:42 EDT 2011.
Is this change intended?
If filling in this field is going to be required to allow more than 10Mbps of traffic going forward, shouldn't that be added to the traffic shaper wizards?

Perhaps not everyone is experiencing this issue, so maybe this issue is limited to users with my type of setup. All the interfaces that I have tested are VLANs provided by a trunk that is provided by an LACP team made up of 2x GigE connetions between the pfSense hardware and a Cisco switch.

I have a connection that is direct, not a trunk, and it seems like traffic to that link was not limited, but I will need to verify that.

eri--

If you want to have the same behaviour as previous snapshots just go to the traffic shaper and select the lan interfaces and remove the shaper for it.
Leave only the WAN ones.

Your issue is that PRIQ can specify bandwidth in only the root queue. So that is the reaon i tell you to remove put the interface bandwidth there.
Possibly at your speeds even increse the queue limit.

PfSense RC3 - Traffic Shaper Issues in resent builds

–-------------------------------------------------- Current Version : 2.0-RC3 Latest Version : Fri Jul 29 06:00:10 EDT 2011

–--------------------------------------------------
Current Version : 2.0-RC3 Latest Version : Fri Jul 29 06:00:10 EDT 2011