Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Speed Interface error in QinQ interface

    Scheduled Pinned Locked Moved L2/Switching/VLANs
    8 Posts 2 Posters 302 Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • P
      prx
      last edited by

      Hi,

      I have 2 Chelsio 4x10Gb SFP+ (T540-LP-CR) and I configured 3 laggs (failover). Then I configured vlan 100 on lagg2 and then I configured several QinQ interface on top vlan 100. Now everything works, but in the dashboard the speed of the QinQ interface is:

      1000baseT <full-duplex>
      

      instead of

      10Gbase-LR <full-duplex>
      

      The clx interfaces of the lagg2 (cxl2,cxl6 ) have the correct media:

      10Gbase-LR <full-duplex>
      

      this is my ifconfig output:

      cxl0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
      	options=8c00b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,LINKSTATE>
      	ether 00:07:43:56:71:f0
      	hwaddr 00:07:43:56:71:f0
      	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
      	media: Ethernet 10Gbase-LR <full-duplex,rxpause,txpause>
      	status: active
      cxl1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1516
      	options=8c00b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,LINKSTATE>
      	ether 00:07:43:56:71:f8
      	hwaddr 00:07:43:56:71:f8
      	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
      	media: Ethernet 10Gbase-LR <full-duplex,rxpause,txpause>
      	status: active
      cxl2: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1516
      	options=8c04b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,LRO,VLAN_HWTSO,LINKSTATE>
      	ether 00:07:43:56:72:00
      	hwaddr 00:07:43:56:72:00
      	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
      	media: Ethernet 10Gbase-LR <full-duplex,rxpause,txpause>
      	status: active
      cxl3: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
      	options=ec07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
      	ether 00:07:43:56:72:08
      	hwaddr 00:07:43:56:72:08
      	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
      	media: Ethernet 10Gbase-LR <full-duplex,rxpause,txpause>
      	status: no carrier
      cxl4: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
      	options=8c00b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,LINKSTATE>
      	ether 00:07:43:56:71:f0
      	hwaddr 00:07:43:56:72:90
      	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
      	media: Ethernet 10Gbase-LR <full-duplex,rxpause,txpause>
      	status: active
      cxl5: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1516
      	options=8c00b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,LINKSTATE>
      	ether 00:07:43:56:71:f8
      	hwaddr 00:07:43:56:72:98
      	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
      	media: Ethernet 10Gbase-LR <full-duplex,rxpause,txpause>
      	status: active
      cxl6: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1516
      	options=8c04b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,LRO,VLAN_HWTSO,LINKSTATE>
      	ether 00:07:43:56:72:00
      	hwaddr 00:07:43:56:72:a0
      	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
      	media: Ethernet 10Gbase-LR <full-duplex,rxpause,txpause>
      	status: active
      cxl7: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
      	options=ec07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
      	ether 00:07:43:56:72:a8
      	hwaddr 00:07:43:56:72:a8
      	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
      	media: Ethernet 10Gbase-LR <full-duplex,rxpause,txpause>
      	status: no carrier
      enc0: flags=0<> metric 0 mtu 1536
      	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
      	groups: enc
      lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
      	options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
      	inet6 ::1 prefixlen 128
      	inet6 fe80::1%lo0 prefixlen 64 scopeid 0xc
      	inet 127.0.0.1 netmask 0xff000000
      	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
      	groups: lo
      pflog0: flags=100<PROMISC> metric 0 mtu 33160
      lagg0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
      	options=8c00b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,LINKSTATE>
      	ether 00:07:43:56:71:f0
      	inet6 fe80::207:43ff:fe56:71f0%lagg0 prefixlen 64 scopeid 0xf
      	inet a.b.c.d netmask 0xfffffff0 broadcast 46.44.246.159
      	inet a.b.c.e netmask 0xfffffff0 broadcast 46.44.246.159 vhid 1
      	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
      	media: Ethernet autoselect
      	status: active
      	carp: MASTER vhid 1 advbase 1 advskew 0
      	groups: lagg
      	laggproto failover lagghash l2,l3,l4
      	laggport: cxl0 flags=5<MASTER,ACTIVE>
      	laggport: cxl4 flags=0<>
      lagg1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1516
      	options=8c00b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,LINKSTATE>
      	ether 00:07:43:56:71:f8
      	inet6 fe80::207:43ff:fe56:71f8%lagg1 prefixlen 64 scopeid 0x10
      	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
      	media: Ethernet autoselect
      	status: active
      	groups: lagg
      	laggproto failover lagghash l2,l3,l4
      	laggport: cxl1 flags=5<MASTER,ACTIVE>
      	laggport: cxl5 flags=0<>
      lagg2: flags=28943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,PPROMISC> metric 0 mtu 1516
      	options=8c04b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,LRO,VLAN_HWTSO,LINKSTATE>
      	ether 00:07:43:56:72:00
      	inet6 fe80::207:43ff:fe56:7200%lagg2 prefixlen 64 scopeid 0x11
      	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
      	media: Ethernet autoselect
      	status: active
      	groups: lagg
      	laggproto failover lagghash l2,l3,l4
      	laggport: cxl2 flags=5<MASTER,ACTIVE>
      	laggport: cxl6 flags=0<>
      lagg1.6: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1516
      	options=80000<LINKSTATE>
      	ether 00:07:43:56:71:f8
      	inet6 fe80::207:43ff:fe56:71f8%lagg1.6 prefixlen 64 scopeid 0x12
      	inet l.m.n.o netmask 0xffffffe0 broadcast 79.135.45.127
      	inet l.m.n.p netmask 0xffffffe0 broadcast 79.135.45.127 vhid 2
      	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
      	media: Ethernet autoselect
      	status: active
      	vlan: 6 vlanpcp: 0 parent interface: lagg1
      	carp: MASTER vhid 2 advbase 1 advskew 0
      	groups: vlan
      lagg1.7: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1516
      	options=80000<LINKSTATE>
      	ether 00:07:43:56:71:f8
      	inet6 fe80::207:43ff:fe56:71f8%lagg1.7 prefixlen 64 scopeid 0x13
      	inet q.r.s.t netmask 0xffffff00 broadcast 91.201.104.255
      	inet q.r.s.t.u netmask 0xffffff00 broadcast 91.201.104.255 vhid 3
      	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
      	media: Ethernet autoselect
      	status: active
      	vlan: 7 vlanpcp: 0 parent interface: lagg1
      	carp: MASTER vhid 3 advbase 1 advskew 0
      	groups: vlan
      lagg2.100: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1516
      	options=80400<LRO,LINKSTATE>
      	ether 00:07:43:56:72:00
      	inet6 fe80::207:43ff:fe56:7200%lagg2.100 prefixlen 64 scopeid 0x14
      	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
      	media: Ethernet autoselect
      	status: active
      	vlan: 100 vlanpcp: 0 parent interface: lagg2
      	groups: vlan QinQ
      lagg2.100: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1516
      	options=80400<LRO,LINKSTATE>
      	ether 00:07:43:56:72:00
      	inet6 fe80::207:43ff:fe56:7200%lagg2.100 prefixlen 64 scopeid 0x14
      	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
      	media: Ethernet autoselect
      	status: active
      	vlan: 100 vlanpcp: 0 parent interface: lagg2
      	groups: vlan QinQ
      lagg2.100.20: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1516
      	options=28<VLAN_MTU,JUMBO_MTU>
      	ether 00:07:43:56:72:00
      	inet6 fe80::207:43ff:fe56:7200%lagg2.100.20 prefixlen 64 scopeid 0x15
      	inet 10.0.0.1 netmask 0xffffffe0 broadcast 10.0.0.31
      	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
      	media: Ethernet autoselect (1000baseT <full-duplex>)
      	status: active
      	groups: QinQ
      lagg2.100.21: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1516
      	options=28<VLAN_MTU,JUMBO_MTU>
      	ether 00:07:43:56:72:00
      	inet6 fe80::207:43ff:fe56:7200%lagg2.100.21 prefixlen 64 scopeid 0x16
      	inet 10.0.1.1 netmask 0xfffffff8 broadcast 10.0.1.7
      	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
      	media: Ethernet autoselect (1000baseT <full-duplex>)
      	status: active
      	groups: QinQ
      lagg2.100.22: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1516
      	options=28<VLAN_MTU,JUMBO_MTU>
      	ether 00:07:43:56:72:00
      	inet6 fe80::207:43ff:fe56:7200%lagg2.100.22 prefixlen 64 scopeid 0x17
      	inet 10.0.1.65 netmask 0xfffffff8 broadcast 10.0.1.71
      	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
      	media: Ethernet autoselect (1000baseT <full-duplex>)
      	status: active
      	groups: QinQ
      lagg2.100.23: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1516
      	options=28<VLAN_MTU,JUMBO_MTU>
      	ether 00:07:43:56:72:00
      	inet6 fe80::207:43ff:fe56:7200%lagg2.100.23 prefixlen 64 tentative scopeid 0x18
      	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
      	media: Ethernet autoselect (1000baseT <full-duplex>)
      	status: active
      	groups: QinQ
      lagg2.100.24: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1516
      	options=28<VLAN_MTU,JUMBO_MTU>
      	ether 00:07:43:56:72:00
      	inet6 fe80::207:43ff:fe56:7200%lagg2.100.24 prefixlen 64 scopeid 0x19
      	inet 10.0.1.73 netmask 0xfffffff8 broadcast 10.0.1.79
      	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
      	media: Ethernet autoselect (1000baseT <full-duplex>)
      	status: active
      	groups: QinQ
      lagg2.100.25: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1516
      	options=28<VLAN_MTU,JUMBO_MTU>
      	ether 00:07:43:56:72:00
      	inet6 fe80::207:43ff:fe56:7200%lagg2.100.25 prefixlen 64 scopeid 0x1a
      	inet 10.0.1.81 netmask 0xfffffff8 broadcast 10.0.1.87
      	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
      	media: Ethernet autoselect (1000baseT <full-duplex>)
      	status: active
      	groups: QinQ
      lagg2.100.26: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1516
      	options=28<VLAN_MTU,JUMBO_MTU>
      	ether 00:07:43:56:72:00
      	inet6 fe80::207:43ff:fe56:7200%lagg2.100.26 prefixlen 64 scopeid 0x1b
      	inet 10.0.1.89 netmask 0xfffffff8 broadcast 10.0.1.95
      	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
      	media: Ethernet autoselect (1000baseT <full-duplex>)
      	status: active
      	groups: QinQ
      lagg2.100.100: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1516
      	options=28<VLAN_MTU,JUMBO_MTU>
      	ether 00:07:43:56:72:00
      	inet6 fe80::207:43ff:fe56:7200%lagg2.100.100 prefixlen 64 scopeid 0x1c
      	inet 10.255.255.1 netmask 0xffffffe0 broadcast 10.255.255.31
      	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
      	media: Ethernet autoselect (1000baseT <full-duplex>)
      	status: active
      	groups: QinQ
      

      the questions are:

      • Why the QinQ interfaces media is 1000BaseT instead 10000Base-SR?

      • Is it correct that lagg0 (WAN) lagg1 (other net) media is "autoselect" and not 10000Base-SR?

      Thank you

      DaddyGoD 1 Reply Last reply Reply Quote 0
      • DaddyGoD
        DaddyGo @prx
        last edited by DaddyGo

        @prx said in Speed Interface error in QinQ interface:

        ifconfig

        Hi,

        Don't you usually disable LRO, TSO stuff?
        These are not good if you are a router or only, if you are an endpoint...
        https://calomel.org/freebsd_network_tuning.html

        although even automatic speed negotion should work, if the setting on the other side is "auto"

        +++edit:

        also important:

        Flow Control (FC) 0=Disabled 1=Rx Pause 2=Tx Pause 3=Full FC

        This tunable must be set according to your configuration. VERY IMPORTANT!

        Set FC to 0 (<x>) on all interfaces

        hw.igb.<x>.fc=0 #Also put this in System Tunables hw.igb.<x>.fc: value=0

        and / or

        Disable Energy Efficiency - set for each igb port in your system
        This setting can cause Link flap errors if not disabled
        Set for every igb interface in the system as per these examples
        dev.igb.0.eee_disabled: value=1
        dev.igb.1.eee_disabled: value=1
        dev.igb.2.eee_disabled: value=1
        dev.igb.3.eee_disabled: value=1

        Cats bury it so they can't see it!
        (You know what I mean if you have a cat)

        1 Reply Last reply Reply Quote 0
        • P
          prx
          last edited by

          Hi,

          as you can see from the attachment, I disabled:

          • Hardware Checksum Offloading
          • Hardware TCP Segmentation Offloading
          • Hardware Large Receive Offloading Advanced_Networking.png

          @DaddyGo
          My NICs are 2 Chelsio T540-LP-CR (every nic has 4x10Gb SFP+ ports), so if I understood well, I have to add the following "lines" in Advanced->System Tunables:

          • hw.cxl0.fc value 0
          • hw.cxl1.fc value 0
          • hw.cxl2.fc value 0
          • hw.cxl3.fc value 0
          • hw.cxl4.fc value 0
          • hw.cxl5.fc value 0
          • hw.cxl6.fc value 0
          • hw.cxl7.fc value 0

          and

          • dev.cxl.0.eee_disabled value 1
          • dev.cxl.1.eee_disabled value 1
          • dev.cxl.2.eee_disabled value 1
          • dev.cxl.3.eee_disabled value 1
          • dev.cxl.4.eee_disabled value 1
          • dev.cxl.5.eee_disabled value 1
          • dev.cxl.6.eee_disabled value 1
          • dev.cxl.7.eee_disabled value 1

          is it right?

          The speed negotiation is set on "auto" on the switches (Cisco nexus 3064-X). This is the running conf of the interface:

          interface Ethernet1/12
            description Internal QinQ Vlans
            switchport mode trunk
            switchport trunk allowed vlan 99-120
            spanning-tree port type edge trunk
          

          And this is the output of show interface ethernet 1/12:

          Ethernet1/12 is up
          admin state is up, Dedicated Interface
            Hardware: 100/1000/10000 Ethernet, address: 0006.f618.91ee (bia 0006.f618.91ee)
            Description: Internal QinQ Vlans
            MTU 1500 bytes, BW 10000000 Kbit , DLY 10 usec
            reliability 255/255, txload 1/255, rxload 1/255
            Encapsulation ARPA, medium is broadcast
            Port mode is trunk
            full-duplex, 10 Gb/s, media type is 10G
            Beacon is turned off
            Auto-Negotiation is turned on  FEC mode is Auto
            Input flow-control is off, output flow-control is off
            Auto-mdix is turned off
            Rate mode is dedicated
            Switchport monitor is off 
            EtherType is 0x8100 
            EEE (efficient-ethernet) : n/a
              admin fec state is auto, oper fec state is off
            Last link flapped 05:21:52
            Last clearing of "show interface" counters never
            5 interface resets
            Load-Interval #1: 30 seconds
              30 seconds input rate 48 bits/sec, 0 packets/sec
              30 seconds output rate 29480 bits/sec, 50 packets/sec
              input rate 48 bps, 0 pps; output rate 29.48 Kbps, 50 pps
            Load-Interval #2: 5 minute (300 seconds)
              300 seconds input rate 32 bits/sec, 0 packets/sec
              300 seconds output rate 29360 bits/sec, 49 packets/sec
              input rate 32 bps, 0 pps; output rate 29.36 Kbps, 49 pps
            RX
              3795925 unicast packets  2832572 multicast packets  4328 broadcast packets
              6632825 input packets  4218958407 bytes
              2039753 jumbo packets  0 storm suppression packets
              0 runts  0 giants  0 CRC  0 no buffer
              0 input error  0 short frame  0 overrun   0 underrun  0 ignored
              0 watchdog  0 bad etype drop  0 bad proto drop  0 if down drop
              0 input with dribble  0 input discard
              10 Rx pause
            TX
              8304992 unicast packets  75937156 multicast packets  814 broadcast packets
              84242962 output packets  12035830617 bytes
              1601477 jumbo packets
              0 output error  0 collision  0 deferred  0 late collision
              0 lost carrier  0 no carrier  0 babble  0 output discard
              0 Tx pause
          

          Please, help me....

          Thank you

          DaddyGoD 1 Reply Last reply Reply Quote 0
          • DaddyGoD
            DaddyGo @prx
            last edited by

            @prx said in Speed Interface error in QinQ interface:

            as you can see from the attachment, I disabled:

            For some reason, this often doesn’t work the way we would expect
            therefore, I would not rely on chance and use loader.conf.local
            (you can also see that the box is checked and the feature is still active)

            like... (loader.conf.local also protects against OP system (FW) changes!):

            592a4dca-327b-4f33-a486-5b7c9f0f1d1a-image.png

            about the 10 Gig:
            Perform a test related to speed and set the switch to fixed speed negotion...
            let's see what happens

            btw:

            with a completely similar problem and we've been talking about this in the forum here for about 6 months now with a college student from the US
            I can't find the posts, plus there was a positive result, so it would be worth looking for....

            One more thing, what CPU you use and what MSI-X
            what does this show:

            dmesg | grep -i msi

            e2818c41-466d-46b5-8952-0731f1452c09-image.png

            Cats bury it so they can't see it!
            (You know what I mean if you have a cat)

            1 Reply Last reply Reply Quote 0
            • P
              prx
              last edited by

              @DaddyGo said in Speed Interface error in QinQ interface:

              dmesg | grep -i msi

              Hi,

              tomorrow I will modify loader.conf.local and I will configure PfSense and switch ports with fixed speed negotion.

              My CPU is AMD EPYC 7272 12-Core and the output of dmesg | grep -i msi is:

              xhci0: Unable to map MSI-X table 
              xhci1: Unable to map MSI-X table 
              bnxt0: msix_init qsets capped at 241
              bnxt0: Using MSI-X interrupts with 2 vectors
              bnxt1: msix_init qsets capped at 241
              bnxt1: Using MSI-X interrupts with 2 vectors
              t5nex0: PCIe gen3 x8, 4 ports, 42 MSI-X interrupts, 140 eq, 41 iq
              xhci2: Unable to map MSI-X table 
              t5nex1: PCIe gen3 x8, 4 ports, 42 MSI-X interrupts, 140 eq, 41 iq
              xhci3: Unable to map MSI-X table 
              

              Thank you very much

              DaddyGoD 1 Reply Last reply Reply Quote 0
              • DaddyGoD
                DaddyGo @prx
                last edited by

                @prx said in Speed Interface error in QinQ interface:

                AMD EPYC 7272

                power machine...😉

                @prx "tomorrow I will modify"
                okay let's see what happens

                btw:
                I'm running on Epyc 3151 (Supermicro) with I350-T4 and I350-F4 (8 interfaces) it takes a long time for a well-tuned NIC setup

                AMD is currently a good choice for a firewall...✋

                a little plus for loader.conf.local...

                Allows NIC to process packets as fast as they are received
                hw.igb.enable_msix=1
                hw.pci.enable_msix=1

                (of course don't put it on "igb")

                Cats bury it so they can't see it!
                (You know what I mean if you have a cat)

                1 Reply Last reply Reply Quote 0
                • P
                  prx
                  last edited by

                  Hi DaddyGo,

                  I followed your advice and now my /boot/loader.conf is:

                  if_bnxt_load="YES"
                  net.inet.tcp.tso=0
                  net.inet.tcp.lro=0
                  hw.cxl.enable_msix=1
                  hw.pci.enable_msix=1
                  dev.cxl.0.fc=0
                  dev.cxl.1.fc=0
                  dev.cxl.2.fc=0
                  dev.cxl.3.fc=0
                  dev.cxl.4.fc=0
                  dev.cxl.5.fc=0
                  dev.cxl.6.fc=0
                  dev.cxl.7.fc=0
                  dev.cxl.0.eee_disabled=1
                  dev.cxl.1.eee_disabled=1
                  dev.cxl.2.eee_disabled=1
                  dev.cxl.3.eee_disabled=1
                  dev.cxl.4.eee_disabled=1
                  dev.cxl.5.eee_disabled=1
                  dev.cxl.6.eee_disabled=1
                  dev.cxl.7.eee_disabled=1
                  

                  Then I have set manually, in the GUI, the speed interface on my QinQ interface to 10GBase-T <full-duplex>. The only thing I couldn't do was set up the the speed of my lagg (0,1,2) interfaces to 10G full-duplex. In the GUI under Assignments -> Interfaces I can only select "autoselect" and this is what I see in the dashboard. For example this is the output of ifconfig of the lagg2 (only media information...)

                  cxl2:
                  	media: Ethernet 10Gbase-LR <full-duplex,rxpause,txpause>
                  cxl6:
                          media: Ethernet 10Gbase-LR <full-duplex,rxpause,txpause>
                  
                  lagg2:
                  	media: Ethernet autoselect
                  
                  lagg2.100:
                          media: Ethernet autoselect
                  
                  lagg2.100.20:
                          media: Ethernet 10Gbase-T <full-duplex>
                  

                  I was able only to force media on the lagg2.100.20 interface but not to its parents interfaces...

                  Thank you

                  DaddyGoD 1 Reply Last reply Reply Quote 0
                  • DaddyGoD
                    DaddyGo @prx
                    last edited by

                    @prx said in Speed Interface error in QinQ interface:

                    In the GUI under Assignments -> Interfaces I can only select "autoselect"

                    Due to the nature of "LAG", both sides require the same - next:

                    -Be of the same speed
                    -Have the same Duplex settings

                    When negotiating the LAG speed (merged), only "auto" can be considered, because of the above.

                    Just think along, two parent interface, which is 1Gig
                    in LAG (with LACP) 2 Gig

                    if one of the parent interfaces becomes detached, the combined speed will again be only 1 Gig, so the speed of the LAG interface cannot be predefined, but depends on the group members (from their number / speed)

                    Cats bury it so they can't see it!
                    (You know what I mean if you have a cat)

                    1 Reply Last reply Reply Quote 0
                    • First post
                      Last post
                    Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.