LAGGs + Traffic shaper HFSC + VLANs


  • Hello,

    I would like to report an issue using LAGGs + Traffic shaper HFSC (+ VLANs)

    Model :
    Lanner FW-7573B
    pfSense 2.4.0-RELEASE (amd64) built on Tue Oct 10 06:43:01 CDT 2017
    FreeBSD 11.1-RELEASE-p1

    Design L3 :

    
     --- [WAN] --- [pfSense] --- [SUBNET-CUSTOMER0]
                            |--- [SUBNET-CUSTOMER1]
                            |--- [SUBNET-CUSTOMER2]
                            |--- [SUBNET-CUSTOMER3]
                            |--- [SUBNET-CUSTOMER4]
    
    ``` 
    Design L2 :
    
    

    -[pfSense]--|
                                      ||          ||
                                  TRUNK-WAN   TRUNK-VLANS
                                      ||          ||
                                    LAGG0      LAGG1
                                      ||        ||
    --- [WAN]  ---  [ISP Device] --- [ S W I T C H ] --- [CUSTOMER0]
                                    |  |  |  |--------- [CUSTOMER1]
                                    |  |  |------------ [CUSTOMER2]
                                    |  |--------------- [CUSTOMER3]
                                    |------------------ [CUSTOMER4]

    
    Setup :
    
    LAGG0 - igb2,igb3 -> used for TRUNK-WANs
    LAGG1 - igb0,igb1 -> used for TRUNK-VLANs (interface LAN)
    VLAN 521 on LAGG0 -> ISP1
    LAGG1 -> CUSTOMER0  (interface LAN)
    VLAN 1041 on LAGG1 -> CUSTOMER1  (interface OPT1)
    VLAN 1042 on LAGG1 -> CUSTOMER2 (interface OPT2)
    VLAN 1043 on LAGG1 -> CUSTOMER3 (interface OPT3)
    VLAN 1044 on LAGG1 -> CUSTOMER4 (interface OPT4)
    
    Error message :
    
    

    There were error(s) loading the rules: pfctl: lagg1: driver does not support altq - The line in question reads [0]: @ 2018-02-15 19:46:57

    
    Investigation:
    I've tried to add VLAN1040 on LAGG1 -> CUSTOMER0 but the bandwith is not shared with the other customers.
    
    The is_altq_capable($int) function into the file /etc/inc/interfaces.inc:/ contains this variable.
    
    

    $capable = array("ae", "age", "alc", "ale", "an", "aue", "axe", "bce",
                            "bfe", "bge", "bridge", "cas", "cpsw", "cxl", "dc", "de",
                            "ed", "em", "ep", "epair", "et", "fxp", "gem", "hme", "hn",
                            "igb", "ix", "jme", "l2tp", "le", "lem", "msk", "mxge", "my",
                            "ndis", "nfe", "ng", "nge", "npe", "nve", "ovpnc", "ovpns",
                            "ppp", "pppoe", "pptp", "re", "rl", "sf", "sge", "sis", "sk",
                            "ste", "stge", "ti", "tun", "txp", "udav", "ural", "vge",
                            "vlan", "vmx", "vr", "vte", "vtnet", "xl");

    
    We tried to fake the function by returning true explicitly without success.
    
    The file /tmp/rules.debug (which is the file used by pfSense instead of /etc/pf.conf) contains my configuration :
    
    

    altq on lagg1 hfsc bandwidth 1900Mb queue {  qLink,  qWAN  }
    queue qLink on lagg1 bandwidth 20% qlimit 500 hfsc (  default  )
    queue qWAN on lagg1 bandwidth 47500Kb hfsc (  linkshare 47500Kb  , upperlimit 47500Kb  )  {  q10ACK,  q11RealTime,  q12High,  q13Medium,  q14Low,  q15Choke  }
    queue q10ACK on lagg1 bandwidth 20% hfsc (  linkshare 20%  )
    queue q11RealTime on lagg1 bandwidth 10% hfsc (  realtime 64Kb , linkshare 10%  )
    queue q12High on lagg1 bandwidth 8% hfsc (  linkshare 8%  )
    queue q13Medium on lagg1 bandwidth 6% hfsc (  linkshare 6%  , upperlimit (35000Kb, 5000, 25000Kb)  )
    queue q14Low on lagg1 bandwidth 4% hfsc (  linkshare 4%  , upperlimit (25000Kb, 5000, 15000Kb)  )
    queue q15Choke on lagg1 bandwidth 2% hfsc (  linkshare 2%  , upperlimit 100Kb  )

    
    When I forced the Reload filter I've got the same error :
    
    […]
    Setting up pass/block rules Allow All IPv6
    Creating rule Allow All IPv6
    Creating IPsec rules…
    Creating uPNP rules...
    Generating ALTQ queues
    Loading filter rules
    Setting up logging information
    Setting up SCRUB information
    
    

    There were error(s) loading the rules: pfctl: lagg1: driver does not support altq - The line in question reads [0]

    
    Workaround :
    
    As a workaround, we deleted the lagg1 and used one of the physical port with HFSC applied on the untagged interface and apply VLAN on that physical port.
    
    LAGG0 - igb2,igb3 -> used for TRUNK-WANs
    VLAN 521 on LAGG0 -> ISP1
    no VLAN on igb4 -> CUSTOMER0  (interface LAN)
    VLAN 1041 on igb4 -> CUSTOMER1  (interface OPT1)
    VLAN 1042 on igb4 -> CUSTOMER2 (interface OPT2)
    VLAN 1043 on igb4 -> CUSTOMER3 (interface OPT3)
    VLAN 1044 on igb4 -> CUSTOMER4 (interface OPT4)
    
    Do you have any feedback on this setup?
    Should I create a Redmine ticket as a bug or as a feature request?
    
    Thanks
    
    Florent
  • LAYER 8 Netgate

    altq is not supported directly on a laggX interface. Assign a VLAN and altq on that.

    https://redmine.pfsense.org/issues/4920


  • @Derelict:

    altq is not supported directly on a laggX interface. Assign a VLAN and altq on that.

    https://redmine.pfsense.org/issues/4920

    Thank you for the link. I see that this is not part of the roadmap.

    I don't understand your suggestion "Assign a VLAN and altq on that.", I've already tried that :

    Thx

    EDIT: @Derelict: You were right, your proposition works with the VLAN on the LAGG but the bandwidth is not shared between CUSTOMERs. I've updated my first post, sorry for the mistake.

    The configuration below has the error message.

    LAGG0 - igb2,igb3 -> used for TRUNK-WANs
    LAGG1 - igb0,igb1 -> used for TRUNK-VLANs
    VLAN 521 on LAGG0 -> ISP1
    LAGG1 -> CUSTOMER0  (interface LAN)
    VLAN 1041 on LAGG1 -> CUSTOMER1  (interface OPT1)
    VLAN 1042 on LAGG1 -> CUSTOMER2 (interface OPT2)
    VLAN 1043 on LAGG1 -> CUSTOMER3 (interface OPT3)
    VLAN 1044 on LAGG1 -> CUSTOMER4 (interface OPT4)

    The configuration below has no error message but the bandwidth is not shared among customers.

    LAGG0 - igb2,igb3 -> used for TRUNK-WANs
    LAGG1 - igb0,igb1 -> used for TRUNK-VLANs (interface LAN)
    VLAN 521 on LAGG0 -> ISP1
    VLAN 1040 on LAGG1 -> CUSTOMER0  (interface LAN)
    VLAN 1041 on LAGG1 -> CUSTOMER1  (interface OPT1)
    VLAN 1042 on LAGG1 -> CUSTOMER2 (interface OPT2)
    VLAN 1043 on LAGG1 -> CUSTOMER3 (interface OPT3)
    VLAN 1044 on LAGG1 -> CUSTOMER4 (interface OPT4)

  • LAYER 8 Netgate

    It wouldn't have been anyway.

    A laggX traffic shaper, if it worked, would still be separate from any shaping on the individual VLANs. It would not be hierarchical.


  • @Derelict:

    It wouldn't have been anyway.

    A laggX traffic shaper, if it worked, would still be separate from any shaping on the individual VLANs. It would not be hierarchical.

    I think that I was not clear enough, sorry.

    I can confirm that the bandwidth is shared when you use a "fake" interface to apply QoS on it (igb0), so I don't see any reason despite the fact that the driver doesn't exist for the LAGG.

    igb2 -> used for WAN
    igb0 -> used for VLANs

    no VLAN no IPv4 no IPv6 on igb0 -> VLANS
    VLAN 1040 on igb0 -> CUSTOMER0 (interface LAN)
    VLAN 1041 on igb0 -> CUSTOMER1 (interface OPT1)
    VLAN 1042 on igb0 -> CUSTOMER2 (interface OPT2)
    VLAN 1043 on igb0 -> CUSTOMER3 (interface OPT3)
    VLAN 1044 on igb0 -> CUSTOMER4 (interface OPT4)

    Traffic Shaper applied on igb0(VLANS) for icoming/outgoing

    WAN
    –qWan
    ----q10
    ----q11
    ----q12
    ----q13
    ----q14
    ----q15
    VLANS
    --qLink
    ----q10
    ----q11
    ----q12
    ----q13
    ----q14
    ----q15