1 WAN, 2 "LAN"
-
Thanks for the warm welcome. ::)
I think I have narrowed it down to the faulty block of code. Unfortunately it uses an iteration structure totally different than perl.
The faulty block starts on line 63 of altq_class.inc. The iteration structure goes like this:
for ($i = '1'; $i < count($rules); $i++)
So we start with 1, count the number of elements in the array $rules (that really screwed me up for a while, btw. in perl that would be @rules), and on each iteration, increment i we'll keep doing it so long as $i is less than the number of elements in $rules.
Wordy, but accurate. I think.
So we go
switch ($rules[$i])
??? Don't know what this does. On the first iteration is is $rules[1]. Unless switch means take the $i element, which would be the first element of $rules?
(I'm still thinking this loop through. Saving and coming back to it…)
altq_class.inc is a wild goose chase. You really want shaper.inc and filter.inc. FWIW, altq_class.inc isn't in releng_1 - it was removed a while back (although you may still have it in your inc directory if you have an old install that's been upgraded ).
–Bill
-
Thanks for saving me there. :)
What send me there is that I did a grep hfsc /etc/inc, and that was the only file that returned a match.
Presuming that whatever was generating the broken line had to either assign it to a variable or print it outright, seemed like a logical place to look.
I'll go ahead and delete that file so it doesn't throw me off any further…
-
Hey, he done good! We have progress!
New debug script:
/* include all configuration functions */ require_once("globals.inc"); require_once("pkg-utils.inc"); require_once("notices.inc"); require_once ("xmlparse.inc"); require_once ("config.inc"); require_once("functions.inc"); require_once ("shaper.inc"); global $config; $config = parse_config(true); $altq_rules = ""; $queue_names = ""; $is_first = ""; if(!is_array($config['shaper']['queue'])) { print "Huh, not an array.\n"; } else { print "An array! w00t!\n"; } $ifdescrs = array('wan', 'lan'); for ($j = 1; isset($config['interfaces']['opt' . $j]); $j++) { $ifdescrs[] = "opt" . $j; } foreach ($ifdescrs as $ifdescr => $ifname) { $queue_names = ""; $is_first = ""; print "Evaluating interface $ifname...\n"; $queue_names = find_root_queue($ifname); print "$queue_names\n"; if($queue_names <> ""){ $altq_rules .= "altq on {$config['interfaces'][$ifname]['if']} "; $bandwidth_arr = get_queue_bandwidth($queue_names); $bandwidth = "bandwidth {$bandwidth_arr['bandwidth']}{$bandwidth_arr['bandwidthtype']}"; $altq_rules .= "{$config['shaper']['schedulertype']} {$bandwidth} "; $altq_rules .= "queue { {$queue_names} }"; } $altq_rules .= "\n"; } return $altq_rules; ?>Output is:
# php test.php X-Powered-By: PHP/4.4.4 Content-type: text/html An array! w00t! Evaluating interface wan... qwanRoot Evaluating interface lan... qlanRoot Evaluating interface opt1... qlanRoot qCLRoot Evaluating interface opt2...We have a winner! So our failure is in the find_root_queue() command. Back to digging…
-
Oops. We get into that command, only to find another failure.
This time, it is in is_subqueue_used_on_interface
Debug script below:
/* include all configuration functions */ require_once("globals.inc"); require_once("pkg-utils.inc"); require_once("notices.inc"); require_once ("xmlparse.inc"); require_once ("config.inc"); require_once("functions.inc"); require_once ("shaper.inc"); global $config; $config = parse_config(true); $altq_rules = ""; $queue_names = ""; $is_first = ""; // Begin pasting find_root_queue function below. $queue_names = ""; // Add all interfaces you wish to test to this list. $interfaces = array("wan", "lan", "opt1"); foreach ($interfaces as $ifname){ $queue_names = ""; foreach ($config['shaper']['queue'] as $queue) { $rule_interface = ""; $q = $queue; $name = $q['name']; $parentqueue = $q['parentqueue']; /* if we're a parentqueue and aren't attached to another queue we're probably a root */ if ((isset($q['parentqueue']) && $q['parentqueue'] <> "") && (!isset($q['attachtoqueue']) || $q['attachtoqueue'] == "")) { /* Confirm that this is a valid queue for this interface */ $rule_interface = is_subqueue_used_on_interface($q['name'], $ifname); if ($rule_interface == 1) { // Count the number of characters in $queue_names. if (strlen($queue_names) > 0) { /* If it was greater than 0, it means that $queue_names had a value set from a previous iteration and we need to append $q['name'] this time around with a space in front of it. This is due to output from is_subqueue_used_on_interface. */ $queue_names .= " "; } $queue_names .= $q['name']; } } $queue_number++; } print "$ifname: $queue_names\n\n"; } return $queue_names; ?>Outuput is below:
X-Powered-By: PHP/4.4.4 Content-type: text/html wan: qwanRoot lan: qlanRoot opt1: qlanRoot qCLRootGetting closer…
-
OMFG I am such a LOSER!!!
::)
Despite what you guys may say, your current code DOES indeed work with multiple interfaces, it's just that I'm such a dork that I couldn't figure it out. :P My debug scripts finally pointed to the problem, and perhaps this could be a jumping-off point for better error-detection to prevent filter rules that will not load, but I digress.
Here's the final debug script I wound up with when the error became apparent:
/* include all configuration functions */ require_once("globals.inc"); require_once("pkg-utils.inc"); require_once("notices.inc"); require_once ("xmlparse.inc"); require_once ("config.inc"); require_once("functions.inc"); require_once ("shaper.inc"); global $config; $config = parse_config(true); $altq_rules = ""; $queue_names = ""; $is_first = ""; // Begin pasting find_root_queue function below. $queue_names = ""; $interfaces = array("wan", "lan", "opt1"); foreach ($interfaces as $ifname){ print "Testing Infterace $ifname:\n"; print "=========================\n\n"; $queue_names = ""; foreach ($config['shaper']['queue'] as $queue) { $rule_interface = ""; $q = $queue; $name = $q['name']; $parentqueue = $q['parentqueue']; /* if we're a parentqueue and aren't attached to another queue we're probably a root */ if ((isset($q['parentqueue']) && $q['parentqueue'] <> "") && (!isset($q['attachtoqueue']) || $q['attachtoqueue'] == "")) { /* Confirm that this is a valid queue for this interface */ print "Confirming queue $name\n"; $rule_interface = is_subqueue_used_on_interface_fixed($q['name'], $ifname); if ($rule_interface == 1) { // Count the number of characters in $queue_names. if (strlen($queue_names) > 0) { /* If it was greater than 0, it means that $queue_names had a value set from a previous iteration and we need to append $q['name'] this time around with a space in front of it. This is due to output from is_subqueue_used_on_interface. */ print "I just found an interfaces with 2 root queues. I was testing queue $name and interface $ifname.\n"; $queue_names .= " "; } print "Adding queue $name and interface $ifname, got a return 1 from is_subqueue_used_on_interface.\n\n"; $queue_names .= $q['name']; } } $queue_number++; } print "\nRoot Queues: $queue_names\n\n"; print "****************************\n\n"; } return $queue_names; function is_subqueue_used_on_interface_fixed($queuename, $interface) { // print "\t is_subqueue_used_on_interface called.\n"; global $config; $qconfig = $config; //Tony adds $attachtotemp $attachtotemp = ""; if (!is_array($qconfig['shaper']['queue'])) return 0; foreach ($qconfig['shaper']['queue'] as $queue) { $attachtotemp = $queue['attachtoqueue']; if($queue['attachtoqueue'] == $queuename) { print "\t Hmm. $queuename is attachtoqueue?\n"; /* recurse if we're a parent queue */ if ($queue['parentqueue'] == "on") { print "\t ...and parentqueue is set to \"on\".\n"; return is_subqueue_used_on_interface_fixed($queue['name'], $interface); } /* If we're not a parent check to see if the queue is used on this interface */ $subqueue_interface = filter_is_queue_being_used_on_interface_fixed($queue['name'], $interface); if ($subqueue_interface != ""){ print "\t\t\t Subqueue Interface Found: $subqueue_interface\n\n"; return 1; } } } return 0; } function filter_is_queue_being_used_on_interface_fixed($queuename, $interface, $direction = 'in') { global $config; $lconfig = $config; print "\t\tQueue name passed is $queuename\n"; print "\t\tInterface passed is $interface\n"; if(!is_array($lconfig['shaper']['rule'])) return null; foreach($lconfig['shaper']['rule'] as $rule) { $q = $direction . 'queue'; $if = $direction . '-interface'; if(($rule[$q] == $queuename && $rule[$if] == $interface)) return $interface; } return null; } ?>Here is what I saw happening with this script:
_Testing Infterace opt1:
Confirming queue qwanRoot
Hmm. qwanRoot is attachtoqueue?
Queue name passed is qwandef
Interface passed is opt1
Hmm. qwanRoot is attachtoqueue?
Queue name passed is qwanacks
Interface passed is opt1
Hmm. qwanRoot is attachtoqueue?
Queue name passed is qVOIPUp
Interface passed is opt1
Hmm. qwanRoot is attachtoqueue?
Queue name passed is qP2PUp
Interface passed is opt1
Hmm. qwanRoot is attachtoqueue?
Queue name passed is qOthersUpH
Interface passed is opt1
Hmm. qwanRoot is attachtoqueue?
Queue name passed is qOthersUpL
Interface passed is opt1
Confirming queue qlanRoot
Hmm. qlanRoot is attachtoqueue?
Queue name passed is qlandef
Interface passed is opt1
Hmm. qlanRoot is attachtoqueue?
Queue name passed is qlanacks
Interface passed is opt1
Hmm. qlanRoot is attachtoqueue?
Queue name passed is qP2PDown
Interface passed is opt1
Hmm. qlanRoot is attachtoqueue?
Queue name passed is qOthersDownH
Interface passed is opt1
Hmm. qlanRoot is attachtoqueue?
Queue name passed is qOthersDownL
Interface passed is opt1
Subqueue Interface Found: opt1Adding queue qlanRoot and interface opt1, got a return 1 from is_subqueue_used_on_interface.
Confirming queue qCLRoot
Hmm. qCLRoot is attachtoqueue?
Queue name passed is qCLdef
Interface passed is opt1
Subqueue Interface Found: opt1I just found an interfaces with 2 root queues. I was testing queue qCLRoot and interface opt1.
Adding queue qCLRoot and interface opt1, got a return 1 from is_subqueue_used_on_interface.Root Queues: qlanRoot qCLRoot
*********************************_I know my wording could certainly be better here, but what it is saying is that I have opt1 using the qOthersDownL queue, and thus opt1 is trying to use qlanRoot. opt1 == CL. So…..
The fix is simple to do, hard to explain. Basically every subqueue you create, you can't go crossing over (ie, if I create a qCLRoot, qCLdef) I can't just go and plug traffic from interface CL into qOthersDownL and qOthersDownH, etc. You have to go in and manually create matching rules to that qCLdef and qCLRoot as appropriate. THEN the rules will load up as expected and life goes on.
So you guys can now safely say that pfSense works with at least 3 interfaces.
It just isn't supported! ;)
Hopefully my debug script can be tweaked to help others, or, as I said, become a jumping-off point for better error detection. Whew. What a day!
-
All that we need now is the wizard for traffic shaping to be updated with your logic then we'll be set. Are you interested in tackling this?
-
Yep, that code was "supposed" to work (and I've made it work before, it's just painful w/out a wizard to frontend it) :) But it's backassward. Rules attached to interfaces and queues and trawling the rules to figure out where a queue is attached is stupid. It started off that way, I stayed on that path and completed the creation of the mess you have in front of you. Good detective work btw.
What I'd like to see is queues assigned to interfaces and rules assigned to queues (or assigned to interfaces, but only display queues that are part of that interface for selection). At any rate, the shaper has a LOT of room to grow, consider this a first (really…the third) stab at it - the current wizard came out of that stab due to necessity.
--Bill
-
LOL My brain is exhausted from the above. Please note that I don't code php. ;)
That said, Sullrich, I have to leave to Montreal on Monday, and today is totally booked up. Perhaps after I return?
I only have one request. I have felt a bit in the past that you got the impression that I "used" you guys, and didn't do anything to give back. Imagine me saying this with a big smile on my face:
I don't want to hear it anymore. ;)
Said it before, say it again. I have a tendency to just think out loud for the benefit of others perhaps seeing something that I've missed. I don't intend to use anyone.
Glad you like my detective work. Made my head hurt at the time…. :P
(UPDATED THOUGHT)
...or, when you create your "down" parts of the rules, just create two.
ie, you can use qUpH, since that's a WAN rule, but you would need to create qlanDownH, and qopt1DownH, etc.
Then way when you specify traffic, at a glance a mistake becomes obvious. Also makes initial rules creation a bit simpler.
-
Yep, that code was "supposed" to work (and I've made it work before, it's just painful w/out a wizard to frontend it) :) But it's backassward. Rules attached to interfaces and queues and trawling the rules to figure out where a queue is attached is stupid. It started off that way, I stayed on that path and completed the creation of the mess you have in front of you. Good detective work btw.
What I'd like to see is queues assigned to interfaces and rules assigned to queues (or assigned to interfaces, but only display queues that are part of that interface for selection). At any rate, the shaper has a LOT of room to grow, consider this a first (really…the third) stab at it - the current wizard came out of that stab due to necessity.
--Bill
What about this for wizard logic then….
Allow the user to choose interfaces to include. WAN and LAN are default, but allow them to choose opts.
Go ahead and create q$ifnameRoot and q$ifname.'def'.
From there in I can see where sanity checking might go a little crazy. The wizard could go ahead an do default shaping between WAN and the interface of your choice. Put a notice to the user not to mix apples and oranges - ie, if you create a rule for an interface, the queues you choose must also be for that interface, and end of in the correct root queue. Perhaps a little error checking during rules creation? (ie, check to see what the root queue is that's in use for the rule, and it the root is associated with an interface that is different from what's selected in the rule, toss at least a warning to the user of what they're doing, and that it could create un-loadable rules?)
Just a thought or two. Going to go rest my head now.
-
What about this for wizard logic then….
Allow the user to choose interfaces to include. WAN and LAN are default, but allow them to choose opts.
Go ahead and create q$ifnameRoot and q$ifname.'def'.
From there in I can see where sanity checking might go a little crazy. The wizard could go ahead an do default shaping between WAN and the interface of your choice. Put a notice to the user not to mix apples and oranges - ie, if you create a rule for an interface, the queues you choose must also be for that interface, and end of in the correct root queue. Perhaps a little error checking during rules creation? (ie, check to see what the root queue is that's in use for the rule, and it the root is associated with an interface that is different from what's selected in the rule, toss at least a warning to the user of what they're doing, and that it could create un-loadable rules?)
Just a thought or two. Going to go rest my head now.
Part of the problem (and we have it today) is shaping between LAN and OPT as well as the incorrect queue creation for the three interfaces. Take this example
512/512 DSL (I'm making my life simple)….If I assign 512 to lan, 512 to opt and 512 to wan, the upstream is the ONLY direction traffic shaping is correct in. The 512 to lan + 512 to opt results in 1024 inbound which is larger than your pipe. I had code to handle this originally, but backed it out because I wasn't thinking about how to solve the "hardware issue" - ie. hardware that doesn't support altq. Also, lan->opt and opt->lan will be shaped at 512 (but that's an issue anyway)... The code as it sits needs a LOT more attention (and I expect another complete rewrite) - I just got burnt out on it, I'll probably get back to it at some point, but right now shaping is kinda mind-numbing ;)
--Bill
-
Oh wow. I thought I had the motivation to tackle this, re-read the thread, and I've chickened out again. No wonder this stagnates. Having messed with it myself, I wouldn't want to touch it either! My own test scripts barely make sense, and I commented the crap out of them….eek!
-
Yep. ALTQ shpaing is a black art and one that will bloody your nose at times :(