Some quirks of LAGG configuration
-
A topic on lagg in the General forum got me looking at lagg on a 2.1 snapshot.
1. The field Parent Interface on the create/edit lagg page would appear to be better described as Member Interfaces since its purpose appears to be to select the member interfaces of the lagg.
2. The interfaces offered include (on my system) usbus0, usbus1, usbus2 and usbus3. Is this a bug? If not, I would be grateful if someone "in the know" would write up a howto for connection of two systems by a lagg over multiple US buses. :)
3. The lacp description includes
Each LAG is composed of ports of the same speed, set to full-duplex operation. The traffic will be balanced across the ports in the LAG with the greatest total speed, in most cases there will only be one LAG which contains all ports.
Since all ports of the lagg are of the same speed it is not clear which ports are those with the greatest total speed. "in most cases …" should be a sentence on its own but its not clear what it is trying to say. For example, I doubt most pfSense installations will have a lagg containing all physical ports. Suggested rewording:
Each LAG is composed of ports of the same speed, set to full-duplex operation. The traffic will be balanced across the ports in the LAG.
4. Among the interfaces offered as lagg members are run0 and ral0.
Both of these have associated "cloned" wireless interfaces (run0_wlan0 and ral0_wlan0 respectively).ral0 has an associated "cloned" wireless interface (ral0_wlan1) If lagg is really supported on WiFi interfaces then we have a more serious variant of the problem discussed in http://forum.pfsense.org/index.php/topic,50444.0.html - a WiFi interface almost always will need some configuration before it will work. To configure it, it must be assigned to pfSense which makes it ineligible to be a lagg member.5. run0 is assigned to pfSense as interface WLAN yet it is still offered as a lagg member.
-
Out of curiosity I configured a lagg with members usbus0 and usbus1. The system log contained this report:
Jun 18 07:53:50 php: /interfaces_lagg_edit.php: The command '/sbin/ifconfig lagg0 laggport usbus0' returned exit code '1', the output was 'ifconfig: SIOCSLAGGPORT: Protocol not supported'
Jun 18 07:53:50 php: /interfaces_lagg_edit.php: The command '/sbin/ifconfig lagg0 laggport usbus1' returned exit code '1', the output was 'ifconfig: SIOCSLAGGPORT: Protocol not supported'
Jun 18 07:53:52 check_reload_status: Syncing firewallI guess lagg on USB will take a bit more effort than writing a howto :-)
-
2. I think the interface listing just defaults to show all interfaces in /dev/ not previously assigned.
1. Yeah. Since most of us will configure a LAGG as LAN or OPTx, it would reduce possible confusion if "Parent Interface" were consistent with other uses of "Parent Interface" in pfSense GUI. "Member Interfaces" makes more sense in LAGG configuration, since one must manually assign the configured LAGG to it's parent interface later.
–--
6. Since the "Choose Member" selection box only lists unassigned interfaces, the user who wants to use his existing LAN or OPTx interface in his new LAGG will not see it displayed.
He must A. select unassigned interface(s) to make a LAGG,
B. add it as (most likely the only) LAGG member,
C. choose protocol and save the new LAGG
D. assign the LAGG as LAN/OPTx, etc
E. assuming A-D were done correctly and the user is now using the LAGG, he must come back to Assign> LAGG page to include the NIC he just de-assigned in D.Not much of a catch-22, but it is a loop that seems better made by the computer than the user. And since it could be done with a GUI selection which invokes a tiny ifconfig unplumb/plumb script…
[ ] Include current LAN interface in LAGGx? (This will assign LAGGx as the LAN interface and keep your existing LAN mac address and IP for the new LAGGx)–---------------------------------------------------------------------------------------------------------------------------
7. Jumbo Frames in Aggregations are currently not persistent by default. Indeed, Jumbo Frames are not persistent in nonaggregated NICs, which makes Jumbo Frames not persistent in pfSense, since there's no user editable /etc/rc.conf or sysctrl hooks, which ifconfig (the underlying command) depends on for persistence.
Although some NIC drivers allow VLAN_MTU options, in general, any MTU settings are lost on reboot in pfSense, and have to be reconfigured by the user. Scripts to do it are easy enough, if the user understands the highlevel command structure, and he understands he's in a "hardened" environment. But I think one of the goals with pfSense is to make this sort of thing doable from the GUI.
One can edit config.xml from the GUI, at his own peril. But this approach encounters some of the same drawbacks of scripting, is not a clean hack, and runs the risk of a small error breaking the xml parser and making pfSense unbootable without some manual debugging. I know this from experience. :D
I guess lagg on USB will take a bit more effort than writing a howto :-)
;D
-
2. I think the interface listing just defaults to show all interfaces in /dev/ not previously assigned.
Maybe. My wireless interfaces (ral0 and run0) that were offered as lagg members don't show up in /dev.
7. Jumbo Frames in Aggregations are currently not persistent by default. Indeed, Jumbo Frames are not persistent in nonaggregated NICs, which makes Jumbo Frames not persistent in pfSense, since there's no user editable /etc/rc.conf or sysctrl hooks, which ifconfig (the underlying command) depends on for persistence.
MTU is configurable through GUI for interfaces assigned to pfSense. Are you reporting MTU settings through the GUI for such interfaces are ignored or don't persist over reboot?
-
2. I think the interface listing just defaults to show all interfaces in /dev/ not previously assigned.
Maybe. My wireless interfaces (ral0 and run0) that were offered as lagg members don't show up in /dev.
My GUI shows USBs in the select field too. But I've attached USB modems and NICs as interfaces, and didn't make much of USB devices being "available" for LAGG. I just "assumed" it was about USB devices with NICs or /dev/ files attached to them, like your Ralink or Alfa.
7. Jumbo Frames in Aggregations are currently not persistent by default. Indeed, Jumbo Frames are not persistent in nonaggregated NICs, which makes Jumbo Frames not persistent in pfSense, since there's no user editable /etc/rc.conf or sysctrl hooks, which ifconfig (the underlying command) depends on for persistence.
MTU is configurable through GUI for interfaces assigned to pfSense. Are you reporting MTU settings through the GUI for such interfaces are ignored or don't persist over reboot?
In previous snaps, ifconfig -a did not always reflect the value I set in the MTU field. I don't know if I can replicate with my current build and config yet, as it's been up with new NICs since install/restore 2 days ago, and it has the LAGG MTU "tweak" in the config besides. :-\
I probably should have :-X on that particular facet of the subject rather than relay it anecdotally. Too much water under the bridge since I saw it last. Even then, I was like, "Huh?"
What WILL happen is that NICs and their BSD drivers have MTU "slots" instead of full ranges, and MTU bit counting convention varies by manufacturer (and maybe BSD driver too ;) ), so that my switch calls a MTU 9014, and pfSense em(4) calls the same MTU 9022 but pfSense will drop packets unless 9014 is entered in the GUI field.
Is pfSense dropping that byte? No. Not according to Pcaps from other hosts. Rewriting headers? No. It's the same number, same packet size, just counted differently, and it's not an Obie Wan.
Nonetheless, ifconfig from the CLI won't allow the switch/pfSense GUI setting of 9014, because the driver/NIC needs 9022, and the GUI has an issue unless set 1 byte lower. -
If lagg is really supported on WiFi interfaces then we have a more serious variant of the problem discussed in http://forum.pfsense.org/index.php/topic,50444.0.html - a WiFi interface almost always will need some configuration before it will work. To configure it, it must be assigned to pfSense which makes it ineligible to be a lagg member.
Just to add to this. A similar problem occurs if you want to use jumboframes over LAGG, as experienced by allpoints, here.
The LAGG interface will default to the lowest MTU of it's member interfaces, therefore you must set all members to use MTU >9000. However currently you can only configure the MTU of assigned/enabled interfaces which then makes them ineligible to be in the LAGG. Problem.Edit: :-[ What you already said!
-
Edit: :-[ What you already said!
[/quote]
Thanks for the post. It reminded me of a correction I needed to make: It seems some WiFi interfaces are offered as lagg members even though they are assigned to pfSense, despite what lagg configuration page says.
