Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    1 WAN, 2 "LAN"

    Scheduled Pinned Locked Moved Traffic Shaping
    28 Posts 4 Posters 15.0k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • N
      Numbski
      last edited by

      Just looking over the part that gets malformed in rules.debug:

      altq on em2 hfsc bandwidth queue { qlanRoot qCLRoot }

      The problem here appears to stem from values in the multi-dimensional array that is created in xmlparse.inc, and gets written back out using altq_class.inc.

      The structure appears to be $data[array1][array2].  So in this case, what I suspect is happening is after the last queue name is read in by xmlparse.inc, it is not initializing the value of $data['queue'][].  When the next one gets written in, it is appending the value of the next read.  Within my config.xml, you have the XML block for qlanRoot, then qCLRoot.  So literally, on my install, if you were to get a list of root queues, it would list them like this:

      qwanRoot
      qlanRoot
      qlanRoot qCLRoot

      I've made this coding error hundreds of times myself in an iterating loop.  The fix is usually something like this:

      [code]
      foreach(@array){
      do something to or with the current value;
      clear the current value going into the next iteration;
      }

      In perl, that "current value" gets referenced by $_ by default.  Where I get into trouble is when I would store that value out into another variable, either because $_ was already in use, or I wanted to be super clear about what value I was messing with.  $_ gets cleared and re-read in on each iteration by default, however any home-grown variables will carry into the next iteration unless I manually init it.  I see areas all over the xmlparse.inc file that isn't just storing a value to a variable, but (ie `$variable = 'newvalue'), but appending ($variable .= 'newvalue').  I'm simply not familiar enough with the codebase or the language to properly debug this.  I've spent several days on it hoping that I would be able to obviate it and either post it here or fix it and post a patch.  Nothing doing.

      Sorry guys.

      1 Reply Last reply Reply Quote 0
      • J
        jeroen234
        last edited by

        will this help ?
        $variable='none'
        $variable = 'newvalue'
        $variable is now newvalue the old variable is lost

        now the other one
        $variable='first'
        $variable .= 'last'
        $variable is now firstlast
        the dot is making that the old value is glued to the new variable

        1 Reply Last reply Reply Quote 0
        • N
          Numbski
          last edited by

          Right I get that.  PHP and perl are syntatically identical in that regard.  I just can't find where it's happening.  That's where I'm looking for help.

          (begs the question how we're getting a space between qlanRoot and qCLRoot though…perhaps it's an array and not a string, but php is being nice and returning it as a space-seperated string?)

          1 Reply Last reply Reply Quote 0
          • N
            Numbski
            last edited by

            Just confirming that this bug remains in RC2g.  Just tested, since it appears parser changes were made.

            1 Reply Last reply Reply Quote 0
            • S
              sullrich
              last edited by

              Traffic shaping is only supported on 2 interfaces.

              1 Reply Last reply Reply Quote 0
              • N
                Numbski
                last edited by

                My apologies for any attempted hijackings.  Since the filter fails to reload with my current config, the upgrade scripts failed until I disabled the pfctl calls.

                Anyhoo.  I understand that it isn't supported, I'm just trying to fix it is all. :)

                Any chance you might point me where in the code this iteration failure might be occuring?  My first guess was /etc/inc/xmlparse.inc, but I'm not seeing at what point rules.debug gets generated. :\

                1 Reply Last reply Reply Quote 0
                • S
                  sullrich
                  last edited by

                  If you get it fixed, we'll glady wil commit changes.  We are taking a break from the shaper as it really kicked our ass in the last 1.5 years.

                  1 Reply Last reply Reply Quote 0
                  • N
                    Numbski
                    last edited by

                    Now you're following me. :D

                    I was only trying to find as polite a way to ask as I could where this happens:

                    Read in config.xml (in particular, the shaper config)
                    Iterate through each queue to put it into the multi-dimensional array.
                    Write out rules.debug.

                    As posted above, if I find where the iterations occur, I'll fix the bug.  I just can't find where in the code this occurs. :)

                    1 Reply Last reply Reply Quote 0
                    • N
                      Numbski
                      last edited by

                      Well, after several hours of screwing around, I have a debug script.  Please try not to laugh at my shoddy php.  It's not my language:

                      
                      require ("shaper.inc");
                      require ("xmlparse.inc");
                      /* include all configuration functions */
                      require_once("globals.inc");
                      require_once("pkg-utils.inc");
                      require_once("notices.inc");
                      require_once ("config.inc");
                      require_once("functions.inc");
                      
                      global $config;
                      $config = parse_config(true);
                      
                      if (!is_array($config['shaper']['rule'])) {
                              print "Huh, not an array.\n";
                      }
                      else {
                              print "An array! w00t!\n";
                      }
                      
                      foreach ($config['shaper']['queue'] as $rule){
                                      $name = $rule['name'];
                                      $bandwidth = $rule['bandwidth'];
                                      $units = $rule['bandwidthtype'];
                                      $priority = $rule['priority'];
                                      $limit = $rule['qlimit'];
                      
                                      print "Name - $name\n";
                                      print "Bandwidth - $bandwidth$units\n";
                                      print "Priority - $priority\n";
                                      print "Limit - $limit\n";
                                      print "----------------\n\n";
                      
                      }
                      ?>
                      
                      

                      I won't dump the entire output, but here's the three lines of interest:

                      Name - qwanRoot
                      Bandwidth - 8000Kb
                      Priority - 0
                      Limit -
                      –--------------

                      Name - qlanRoot
                      Bandwidth - 8000Kb
                      Priority - 0
                      Limit -

                      Name - qCLRoot
                      Bandwidth - 8000Kb
                      Priority - 0
                      Limit -

                      This leaves me with some serious head scratching, because that means the XML parser functions are correct, and that it's the rule writing routines that are really at fault.  Grr.....so several hours down the drain.  Now I get to find the routine that writes rules.debug and see if I can hack that up a bit.

                      1 Reply Last reply Reply Quote 0
                      • S
                        sullrich
                        last edited by

                        Welcome to the hell that we spent 1.0+ year on!

                        1 Reply Last reply Reply Quote 0
                        • N
                          Numbski
                          last edited by

                          Thanks for the warm welcome.  ::)

                          I think I have narrowed it down to the faulty block of code.  Unfortunately it uses an iteration structure totally different than perl.

                          The faulty block starts on line 63 of altq_class.inc.  The iteration structure goes like this:

                          for ($i = '1'; $i < count($rules); $i++)

                          So we start with 1, count the number of elements in the array $rules (that really screwed me up for a while, btw.  in perl that would be @rules), and on each iteration, increment i we'll keep doing it so long as $i is less than the number of elements in $rules.

                          Wordy, but accurate.  I think.

                          So we go

                          switch ($rules[$i])

                          ???  Don't know what this does.  On the first iteration is is $rules[1].  Unless switch means take the $i element, which would be the first element of $rules?

                          (I'm still thinking this loop through.  Saving and coming back to it…)

                          1 Reply Last reply Reply Quote 0
                          • S
                            sullrich
                            last edited by

                            We don't use altq_class.inc!

                            All code is in /etc/inc/filter.inc and /etc/inc/shaper.inc.

                            1 Reply Last reply Reply Quote 0
                            • B
                              billm
                              last edited by

                              @Numbski:

                              Thanks for the warm welcome.  ::)

                              I think I have narrowed it down to the faulty block of code.  Unfortunately it uses an iteration structure totally different than perl.

                              The faulty block starts on line 63 of altq_class.inc.  The iteration structure goes like this:

                              for ($i = '1'; $i < count($rules); $i++)

                              So we start with 1, count the number of elements in the array $rules (that really screwed me up for a while, btw.  in perl that would be @rules), and on each iteration, increment i we'll keep doing it so long as $i is less than the number of elements in $rules.

                              Wordy, but accurate.  I think.

                              So we go

                              switch ($rules[$i])

                              ???  Don't know what this does.  On the first iteration is is $rules[1].  Unless switch means take the $i element, which would be the first element of $rules?

                              (I'm still thinking this loop through.  Saving and coming back to it…)

                              altq_class.inc is a wild goose chase.  You really want shaper.inc and filter.inc.  FWIW, altq_class.inc isn't in releng_1 - it was removed a while back (although you may still have it in your inc directory if you have an old install that's been upgraded ).

                              –Bill

                              pfSense core developer
                              blog - http://www.ucsecurity.com/
                              twitter - billmarquette

                              1 Reply Last reply Reply Quote 0
                              • N
                                Numbski
                                last edited by

                                Thanks for saving me there. :)

                                What send me there is that I did a grep hfsc /etc/inc, and that was the only file that returned a match.

                                Presuming that whatever was generating the broken line had to either assign it to a variable or print it outright, seemed like a logical place to look.

                                I'll go ahead and delete that file so it doesn't throw me off any further…

                                1 Reply Last reply Reply Quote 0
                                • N
                                  Numbski
                                  last edited by

                                  Hey, he done good!  We have progress!

                                  New debug script:

                                  
                                  /* include all configuration functions */
                                  require_once("globals.inc");
                                  require_once("pkg-utils.inc");
                                  require_once("notices.inc");
                                  require_once ("xmlparse.inc");
                                  require_once ("config.inc");
                                  require_once("functions.inc");
                                  require_once ("shaper.inc");
                                  
                                  global $config;
                                  $config = parse_config(true);
                                  $altq_rules  = "";
                                  $queue_names = "";
                                  $is_first = "";
                                  
                                  if(!is_array($config['shaper']['queue'])) {
                                          print "Huh, not an array.\n";
                                  }
                                  else {
                                          print "An array! w00t!\n";
                                  }
                                  
                                  $ifdescrs = array('wan', 'lan');
                                  
                                  for ($j = 1; isset($config['interfaces']['opt' . $j]); $j++) {
                                          $ifdescrs[] = "opt" . $j;
                                  }
                                  
                                  foreach ($ifdescrs as $ifdescr => $ifname) {
                                          $queue_names = "";
                                          $is_first = "";
                                  
                                          print "Evaluating interface $ifname...\n";
                                  
                                          $queue_names = find_root_queue($ifname);
                                  
                                          print "$queue_names\n";
                                  
                                          if($queue_names <> ""){
                                                  $altq_rules .= "altq on {$config['interfaces'][$ifname]['if']} ";
                                                  $bandwidth_arr = get_queue_bandwidth($queue_names);
                                                  $bandwidth = "bandwidth {$bandwidth_arr['bandwidth']}{$bandwidth_arr['bandwidthtype']}";
                                                  $altq_rules .= "{$config['shaper']['schedulertype']}  {$bandwidth} ";
                                                  $altq_rules .= "queue { {$queue_names} }";
                                          }
                                          $altq_rules .= "\n";
                                  
                                  }
                                  return $altq_rules;
                                  
                                  ?>
                                  
                                  

                                  Output is:

                                  
                                  # php test.php
                                  X-Powered-By: PHP/4.4.4
                                  Content-type: text/html
                                  
                                  An array! w00t!
                                  Evaluating interface wan...
                                  qwanRoot
                                  Evaluating interface lan...
                                  qlanRoot
                                  Evaluating interface opt1...
                                  qlanRoot qCLRoot
                                  Evaluating interface opt2...
                                  
                                  

                                  We have a winner!  So our failure is in the find_root_queue() command.  Back to digging…

                                  1 Reply Last reply Reply Quote 0
                                  • N
                                    Numbski
                                    last edited by

                                    Oops.  We get into that command, only to find another failure.

                                    This time, it is in is_subqueue_used_on_interface

                                    Debug script below:

                                    
                                    /* include all configuration functions */
                                    require_once("globals.inc");
                                    require_once("pkg-utils.inc");
                                    require_once("notices.inc");
                                    require_once ("xmlparse.inc");
                                    require_once ("config.inc");
                                    require_once("functions.inc");
                                    require_once ("shaper.inc");
                                    
                                    global $config;
                                    $config = parse_config(true);
                                    $altq_rules  = "";
                                    $queue_names = "";
                                    $is_first = "";
                                    
                                    // Begin pasting find_root_queue function below.
                                    $queue_names = "";
                                    
                                    // Add all interfaces you wish to test to this list.
                                    $interfaces = array("wan", "lan", "opt1");
                                            foreach ($interfaces as $ifname){
                                                    $queue_names = "";
                                    
                                            foreach ($config['shaper']['queue'] as $queue) {
                                                    $rule_interface = "";
                                                    $q = $queue;
                                    
                                                    $name = $q['name'];
                                                    $parentqueue = $q['parentqueue'];
                                    
                                                    /* if we're a parentqueue and aren't attached to another queue we're probably a root */
                                                    if ((isset($q['parentqueue']) && $q['parentqueue'] <> "") && (!isset($q['attachtoqueue']) || $q['attachtoqueue'] == "")) {
                                                            /* Confirm that this is a valid queue for this interface */
                                                            $rule_interface = is_subqueue_used_on_interface($q['name'], $ifname);
                                    
                                                            if ($rule_interface == 1) {
                                                                    // Count the number of characters in $queue_names.
                                                                    if (strlen($queue_names) > 0) {
                                                                            /* If it was greater than 0, it means that $queue_names 
                                                                             had a value set from a previous iteration and we 
                                                                             need to append $q['name'] this time around with a space
                                                                            in front of it. This is due to output from 
                                                                            is_subqueue_used_on_interface. */
                                                                            $queue_names .= " ";
                                                                    }
                                                                    $queue_names .= $q['name'];
                                                            }
                                                    }
                                                    $queue_number++;
                                            }
                                            print "$ifname: $queue_names\n\n";
                                    }
                                    return $queue_names;         
                                    
                                    ?>
                                    
                                    

                                    Outuput is below:

                                    
                                    X-Powered-By: PHP/4.4.4
                                    Content-type: text/html
                                    
                                    wan: qwanRoot
                                    
                                    lan: qlanRoot
                                    
                                    opt1: qlanRoot qCLRoot
                                    
                                    

                                    Getting closer…

                                    1 Reply Last reply Reply Quote 0
                                    • N
                                      Numbski
                                      last edited by

                                      OMFG I am such a LOSER!!!

                                      ::)

                                      Despite what you guys may say, your current code DOES indeed work with multiple interfaces, it's just that I'm such a dork that I couldn't figure it out. :P  My debug scripts finally pointed to the problem, and perhaps this could be a jumping-off point for better error-detection to prevent filter rules that will not load, but I digress.

                                      Here's the final debug script I wound up with when the error became apparent:

                                      
                                      /* include all configuration functions */
                                      require_once("globals.inc");
                                      require_once("pkg-utils.inc");
                                      require_once("notices.inc");
                                      require_once ("xmlparse.inc");
                                      require_once ("config.inc");
                                      require_once("functions.inc");
                                      require_once ("shaper.inc");
                                      
                                      global $config;
                                      $config = parse_config(true);
                                      $altq_rules  = "";
                                      $queue_names = "";
                                      $is_first = "";
                                      
                                      // Begin pasting find_root_queue function below.
                                      $queue_names = "";
                                      
                                      $interfaces = array("wan", "lan", "opt1");
                                              foreach ($interfaces as $ifname){
                                                      print "Testing Infterace $ifname:\n";
                                                      print "=========================\n\n";
                                                      $queue_names = "";
                                      
                                              foreach ($config['shaper']['queue'] as $queue) {
                                                      $rule_interface = "";
                                                      $q = $queue;
                                      
                                                      $name = $q['name'];
                                                      $parentqueue = $q['parentqueue'];
                                      
                                                      /* if we're a parentqueue and aren't attached to another queue we're probably a root */
                                                      if ((isset($q['parentqueue']) && $q['parentqueue'] <> "") && (!isset($q['attachtoqueue']) || $q['attachtoqueue'] == "")) {
                                                              /* Confirm that this is a valid queue for this interface */
                                                              print "Confirming queue $name\n";
                                                              $rule_interface = is_subqueue_used_on_interface_fixed($q['name'], $ifname);
                                      
                                                              if ($rule_interface == 1) {
                                                                      // Count the number of characters in $queue_names.
                                                                      if (strlen($queue_names) > 0) {
                                                                              /* If it was greater than 0, it means that $queue_names 
                                                                               had a value set from a previous iteration and we 
                                                                               need to append $q['name'] this time around with a space
                                                                              in front of it. This is due to output from 
                                                                              is_subqueue_used_on_interface. */
                                      
                                                                              print "I just found an interfaces with 2 root queues.  I was testing queue $name and interface $ifname.\n";
                                                                              $queue_names .= " ";
                                                                      }
                                                                      print "Adding queue $name and interface $ifname, got a return 1 from is_subqueue_used_on_interface.\n\n";
                                                                      $queue_names .= $q['name'];
                                                              }
                                                      }
                                                      $queue_number++;
                                              }
                                              print "\nRoot Queues: $queue_names\n\n";
                                              print "****************************\n\n";
                                      }
                                      return $queue_names;         
                                      
                                      function is_subqueue_used_on_interface_fixed($queuename, $interface) {
                                      //      print "\t is_subqueue_used_on_interface called.\n";
                                              global $config;
                                              $qconfig = $config;
                                              //Tony adds $attachtotemp
                                              $attachtotemp = "";
                                      
                                              if (!is_array($qconfig['shaper']['queue'])) return 0;
                                      
                                              foreach ($qconfig['shaper']['queue'] as $queue) {
                                                      $attachtotemp = $queue['attachtoqueue'];
                                                      if($queue['attachtoqueue'] == $queuename) {   
                                                              print "\t Hmm.  $queuename is attachtoqueue?\n";
                                                              /* recurse if we're a parent queue */
                                                              if ($queue['parentqueue'] == "on") {
                                                                      print "\t ...and parentqueue is set to \"on\".\n";
                                                                      return is_subqueue_used_on_interface_fixed($queue['name'], $interface);
                                                              }
                                      
                                                              /* If we're not a parent check to see if the queue is used on this interface */
                                                              $subqueue_interface = filter_is_queue_being_used_on_interface_fixed($queue['name'], $interface);
                                                              if ($subqueue_interface != ""){
                                                                      print "\t\t\t Subqueue Interface Found: $subqueue_interface\n\n";
                                                                      return 1;
                                                              }
                                                      }
                                              }
                                              return 0;
                                      }
                                      
                                      function filter_is_queue_being_used_on_interface_fixed($queuename, $interface, $direction = 'in') {
                                              global $config;
                                              $lconfig = $config;
                                      
                                              print "\t\tQueue name passed is $queuename\n";
                                              print "\t\tInterface passed is $interface\n";
                                      
                                              if(!is_array($lconfig['shaper']['rule'])) return null;
                                              foreach($lconfig['shaper']['rule'] as $rule) {
                                                      $q  = $direction . 'queue';
                                                      $if = $direction . '-interface';
                                                      if(($rule[$q] == $queuename && $rule[$if] == $interface))
                                                      return $interface;
                                              }
                                              return null;
                                      }
                                      ?>
                                      
                                      

                                      Here is what I saw happening with this script:

                                      _Testing Infterace opt1:

                                      Confirming queue qwanRoot
                                              Hmm.  qwanRoot is attachtoqueue?
                                                      Queue name passed is qwandef
                                                      Interface passed is opt1
                                              Hmm.  qwanRoot is attachtoqueue?
                                                      Queue name passed is qwanacks
                                                      Interface passed is opt1
                                              Hmm.  qwanRoot is attachtoqueue?
                                                      Queue name passed is qVOIPUp
                                                      Interface passed is opt1
                                              Hmm.  qwanRoot is attachtoqueue?
                                                      Queue name passed is qP2PUp
                                                      Interface passed is opt1
                                              Hmm.  qwanRoot is attachtoqueue?
                                                      Queue name passed is qOthersUpH
                                                      Interface passed is opt1
                                              Hmm.  qwanRoot is attachtoqueue?
                                                      Queue name passed is qOthersUpL
                                                      Interface passed is opt1
                                      Confirming queue qlanRoot
                                              Hmm.  qlanRoot is attachtoqueue?
                                                      Queue name passed is qlandef
                                                      Interface passed is opt1
                                              Hmm.  qlanRoot is attachtoqueue?
                                                      Queue name passed is qlanacks
                                                      Interface passed is opt1
                                              Hmm.  qlanRoot is attachtoqueue?
                                                      Queue name passed is qP2PDown
                                                      Interface passed is opt1
                                              Hmm.  qlanRoot is attachtoqueue?
                                                      Queue name passed is qOthersDownH
                                                      Interface passed is opt1
                                              Hmm.  qlanRoot is attachtoqueue?
                                                      Queue name passed is qOthersDownL
                                                      Interface passed is opt1
                                                              Subqueue Interface Found: opt1

                                      Adding queue qlanRoot and interface opt1, got a return 1 from is_subqueue_used_on_interface.

                                      Confirming queue qCLRoot
                                              Hmm.  qCLRoot is attachtoqueue?
                                                      Queue name passed is qCLdef
                                                      Interface passed is opt1
                                                              Subqueue Interface Found: opt1

                                      I just found an interfaces with 2 root queues.  I was testing queue qCLRoot and interface opt1.
                                      Adding queue qCLRoot and interface opt1, got a return 1 from is_subqueue_used_on_interface.

                                      Root Queues: qlanRoot qCLRoot
                                      *********************************_

                                      I know my wording could certainly be better here, but what it is saying is that I have opt1 using the qOthersDownL queue, and thus opt1 is trying to use qlanRoot.  opt1 == CL.  So…..

                                      The fix is simple to do, hard to explain.  Basically every subqueue you create, you can't go crossing over (ie, if I create a qCLRoot, qCLdef) I can't just go and plug traffic from interface CL into qOthersDownL and qOthersDownH, etc.  You have to go in and manually create matching rules to that qCLdef and qCLRoot as appropriate.  THEN the rules will load up as expected and life goes on.

                                      So you guys can now safely say that pfSense works with at least 3 interfaces.

                                      It just isn't supported! ;)

                                      Hopefully my debug script can be tweaked to help others, or, as I said, become a jumping-off point for better error detection.  Whew.  What a day!

                                      1 Reply Last reply Reply Quote 0
                                      • S
                                        sullrich
                                        last edited by

                                        All that we need now is the wizard for traffic shaping to be updated with your logic then we'll be set.  Are you interested in tackling this?

                                        1 Reply Last reply Reply Quote 0
                                        • B
                                          billm
                                          last edited by

                                          Yep, that code was "supposed" to work (and I've made it work before, it's just painful w/out a wizard to frontend it) :)  But it's backassward.  Rules attached to interfaces and queues and trawling the rules to figure out where a queue is attached is stupid.  It started off that way, I stayed on that path and completed the creation of the mess you have in front of you.  Good detective work btw.

                                          What I'd like to see is queues assigned to interfaces and rules assigned to queues (or assigned to interfaces, but only display queues that are part of that interface for selection).  At any rate, the shaper has a LOT of room to grow, consider this a first (really…the third) stab at it - the current wizard came out of that stab due to necessity.

                                          --Bill

                                          pfSense core developer
                                          blog - http://www.ucsecurity.com/
                                          twitter - billmarquette

                                          1 Reply Last reply Reply Quote 0
                                          • N
                                            Numbski
                                            last edited by

                                            LOL  My brain is exhausted from the above.  Please note that I don't code php. ;)

                                            That said, Sullrich, I have to leave to Montreal on Monday, and today is totally booked up.  Perhaps after I return?

                                            I only have one request.  I have felt a bit in the past that you got the impression that I "used" you guys, and didn't do anything to give back.  Imagine me saying this with a big smile on my face:

                                            I don't want to hear it anymore. ;)

                                            Said it before, say it again.  I have a tendency to just think out loud for the benefit of others perhaps seeing something that I've missed.  I don't intend to use anyone.

                                            Glad you like my detective work.  Made my head hurt at the time…. :P

                                            (UPDATED THOUGHT)

                                            ...or, when you create your "down" parts of the rules, just create two.

                                            ie, you can use qUpH, since that's a WAN rule, but you would need to create qlanDownH, and qopt1DownH, etc.

                                            Then way when you specify traffic, at a glance a mistake becomes obvious.  Also makes initial rules creation a bit simpler.

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.