Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    10gbe acting like 100mbe

    Scheduled Pinned Locked Moved General pfSense Questions
    17 Posts 4 Posters 2.7k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • stephenw10S
      stephenw10 Netgate Administrator
      last edited by

      Wrong on all counts then. 🙄

      F 1 Reply Last reply Reply Quote 0
      • F
        froboz @stephenw10
        last edited by

        @stephenw10

        ifconfig ixl0 -vvvm returns ifconfig: -vvvm: bad value

        ifconfig results for both ports below. I left out ip and mac info

        ixl0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: WAN
        options=e100bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER,RXCSUM_IPV6,TXCSUM_IPV6>
        media: Ethernet autoselect (2500Base-T <full-duplex,rxpause>)
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

        ixl1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: LAN
        options=e100bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER,RXCSUM_IPV6,TXCSUM_IPV6>
        media: Ethernet autoselect (5000Base-T <full-duplex,rxpause>)
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

        Capture1.PNG
        Capture2.PNG

        1 Reply Last reply Reply Quote 0
        • F
          froboz @stephenw10
          last edited by froboz

          @stephenw10

          Thanks for the link I think I will try and switch to auto and then manually set flow control to receive pause. The question is would the entry be hw.ix.flow_control=1 or hw.ixl1.flow_control=1 ? and can I just run that as a temp command to see if it works without editing the file?

          1 Reply Last reply Reply Quote 0
          • stephenw10S
            stephenw10 Netgate Administrator
            last edited by

            Sorry got the switches is the wrong order, clearly not my best day!
            Should be ifconfig -vvvm ixl0

            The 700 series NIC uses the ixl(4) driver, ix is for earlier Intel NICs.

            You should be able to set that value per NIC as a test at the command line using, for example:

            [admin@7100.stevew.lan]/root: sysctl dev.ixl.0.fc=0
dev.ixl.0.fc: 1 -> 0

            Yes, tested that works immediately.

            Steve

            F 1 Reply Last reply Reply Quote 0
            • F
              froboz @stephenw10
              last edited by

              @stephenw10 I tried the command sysctl dev.ixl.1.fc=1 and it worked the 10gbe was now showing rxpause but the bad new is my connection was still only about 100mbps. I am sure I pushing my luck with a beta of pfsense and a brand new switch from QNAP and a newer .. I think intel card. 5gbe is perfectly fine for me and so none of this is a big deal but I am still willing to troubleshoot if anyone finds this stuff interesting.

              ifconfig -vvvm ixl1

              ixl1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
              description: LAN
              options=e100bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER,RXCSUM_IPV6,TXCSUM_IPV6>
              capabilities=f507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,NETMAP,RXCSUM_IPV6,TXCSUM_IPV6>

              media: Ethernet autoselect (5000Base-T <full-duplex,rxpause>)
status: active
supported media:
	media autoselect
	media 10Gbase-T
	media 5000Base-T
	media 2500Base-T
	media 1000baseT
	media 100baseTX
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

              Capture5.PNG

              1 Reply Last reply Reply Quote 0
              • stephenw10S
                stephenw10 Netgate Administrator
                last edited by

                So with the switch set to 5G fixed the pfSense interface auto-negotiates to 5G and you see the full 1.2Gbps across it?

                But if you set both ends to autoselect it tries to comes up at 10G and you see ~100Mbps?
                That sounds a lot like a bad cable.

                Check the interface status when it's not passing as expected, do you see errors or collisions?

                Steve

                F 2 Replies Last reply Reply Quote 0
                • F
                  froboz @stephenw10
                  last edited by

                  @stephenw10 Yes, you are correct in your summary of the problem. I am using a cable matter 5 ft cat 6 550mhz 24 gauge cable currently between the switch and router but I will switch that out and I was thinking of also trying a different port on the switch in case that port is maybe bad. I will check for collisions or errors as well. I have to wait until after 5pm today but I will give it a go and report back what I find. Thanks for the help.

                  1 Reply Last reply Reply Quote 0
                  • F
                    froboz @stephenw10
                    last edited by

                    @stephenw10 To summarize switching cable and ports made no difference at all but .... I did discover that if I force the switch to 5gbe and turn off flow control also from the switch I get the same 100mbps problem. So everything is pointing to flow control being the issue. Is it possible that command sysctl dev.ixl.1.fc=1 made rxpause appear but did not actually turn it on? like display only?

                    Pictures below because who doesn't like photos:

                    So this is internet speed test with switch port forced to 5gbe flow control on:

                    5gbeCapture.PNG

                    Here is after switching physical ports and cable with port set to auto negotiate 10gbe also error checks I did

                    differentportandcable.PNG
                    autoafterspeedtest.PNG

                    1 Reply Last reply Reply Quote 0
                    • stephenw10S
                      stephenw10 Netgate Administrator
                      last edited by

                      Seeing rxpause there does indicate flowcontrol is active. It often would show txpause there too though.
                      Usually it would be disabling flowcontrol that is required if any change is actually required. Flow control is enabled by default on most 10GbE connections.

                      There's probably not that much that can be done here. Setting the switch to 5G fixed seems like reasonable option unless there is something you are trying to achieve beyond that?

                      Steve

                      F 1 Reply Last reply Reply Quote 0
                      • F
                        froboz @stephenw10
                        last edited by froboz

                        @stephenw10 No, not really trying to do much other than future proof. Maybe this will help someone down the road a bit that is running into this problem as everything is really stock here . Thanks again for the help!

                        EDIT: By stock I mean a fresh install with no add ons and very little setting changes.

                        L 1 Reply Last reply Reply Quote 0
                        • L
                          lnguyen @froboz
                          last edited by

                          @froboz

                          • I have a XG-1537 with a X710-T4L running pfSense+ 21.02.2
                          • If ixl0 is connected directly to the 2.5Gbps port on an Arris S33, sub 1Gbps (500Mbps) downstream performance is realized for for all LAN hosts with 1Gbps. 10Gbps LAN hosts are able to saturate the 1440Mbps downstream bandwidth provisioned by Comcast
                          • If ixl0 is connected directly to the 1Gbps port on the Arris S33, both 1Gbps and 10Gbps LAN hosts realize 942Mbps downstream performance.

                          Flow control function seems defunct. Sysctl output shows its disabled:

                          sysctl dev.ixl.0.fc
dev.ixl.0.fc: 0

                          The UI and ifconfig states otherwise:

                          9e245618-6cd9-43af-bdb6-dfa2270bab4c-image.png

                          ixl0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: WAN
        options=e100bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER,RXCSUM_IPV6,TXCSUM_IPV6>
        capabilities=f507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,NETMAP,RXCSUM_IPV6,TXCSUM_IPV6>
        ether "REDACTED"
        inet6 "REDACTED"%ixl0 prefixlen 64 scopeid 0x3
        inet6 "REDACTED" prefixlen 128
        inet "REDACTED" netmask 0xfffffe00 broadcast 255.255.255.255
        media: Ethernet autoselect (2500Base-T <full-duplex,rxpause>)
        status: active
        supported media:
                media autoselect
                media 10Gbase-T
                media 5000Base-T
                media 2500Base-T
                media 1000baseT
                media 100baseTX
        nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
                          F 1 Reply Last reply Reply Quote 0
                          • F
                            froboz @lnguyen
                            last edited by

                            @lnguyen Thanks for the information! if I am understanding correctly you are experiencing the same problem I am and it points to flow control not being set correctly.

                            L 1 Reply Last reply Reply Quote 0
                            • L
                              lnguyen @froboz
                              last edited by lnguyen

                              @froboz

                              Yes. I am trying to update the FW from 3/31/21 on Intel's Support page.

                              https://downloadcenter.intel.com/download/24769/Non-Volatile-Memory-NVM-Update-Utility-for-Intel-Ethernet-Network-Adapter-700-Series

                              There is another thread that another user got this working:

                              https://forum.netgate.com/topic/162333/intel-x710-issues/8?_=1620678897780

                              1 Reply Last reply Reply Quote 0
                              • First post
                                Last post
                              Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.