Navigation

    Netgate Discussion Forum
    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search

    10gbe acting like 100mbe

    General pfSense Questions
    3
    14
    257
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • F
      froboz last edited by

      Let me start by saying I don't normally post so take it easy you bunch of animals ;-) . My knowledge is networks+ certification and a few Cisco classes many many moons ago so not much but let me get to the issues.

      I have a cable modem with 2.5gbe port and 1 gig internet service provisioned to 1.2 from xfinity
      I am using the popular HP 290-p0043w as my hardware with a Intel - X710T2LBLK (dual 10/5/2.5 gbe ports)
      I am using pfsense 2.5.0-DEVELOPMENT
      My switch is 10gbe rj-45 QNAP QSW-M408-4C 10GbE Managed Switch

      The issue: If I set the switch to auto negotiate speed it sets itself to 10gbe and flow control on but on pfsense it is just 10gbe with no note of any flow control and speed tests are showing about 100 mbps and I should be gettings 1.2gbps

      Next if I set switch to force 5gbe then switch shows 5gbe with flow control on and pfsense shows 5000Base-T <full-duplex,rxpause> I assume the rxpause is some sort of flow control.. Now everything is working as it should. When testing with a 2.5gbe hooked to switch I get 1.2 gbps as I would expect.

      So in summary if I set port to 10gbe it acts like 100mbps and if I force 5 gbe everything is as expected.

      I know I should be using iperf and that might be my next step but any thoughts or ideas on this one and does someone have a similar setup hardware wise? Thanks!

      1 Reply Last reply Reply Quote 0
      • stephenw10
        stephenw10 Netgate Administrator last edited by

        The ixl drivers in pfSense 2.4.5 do not have any support for 5G or 2.5G.

        If you are running 2.4.5 I'm quite surprised it links at 5GbE to the switch.

        You can enable/disable flowcontrol using the appropriate sysctls or loader values:
        https://docs.netgate.com/pfsense/en/latest/hardware/tune.html#flow-control

        As far as I know the only way to establish a 2.5GbE link in 2.4.5 is to use one of the newer Realtek cards with an updated driver or one of the Broadcom cards that can be firmware hacked to do so.

        Steve

        1 Reply Last reply Reply Quote 0
        • stephenw10
          stephenw10 Netgate Administrator last edited by

          Nope, I'm wrong. Only at base-T though:
          https://github.com/pfsense/FreeBSD-src/blob/RELENG_2_4_5/sys/dev/ixl/ixl_pf_main.c#L1850

          Which it looks like you're using.

          If you run ifconfig ixl0 -vvvm does it show 2.5 or 5 as avaialable link speeds?

          Steve

          kiokoman F 2 Replies Last reply Reply Quote 0
          • kiokoman
            kiokoman LAYER 8 @stephenw10 last edited by

            @stephenw10
            I am using pfsense 2.5.0-DEVELOPMENT
            wrong RELENG ๐Ÿ˜€

            1 Reply Last reply Reply Quote 0
            • stephenw10
              stephenw10 Netgate Administrator last edited by

              Wrong on all counts then. ๐Ÿ™„

              F 1 Reply Last reply Reply Quote 0
              • F
                froboz @stephenw10 last edited by

                @stephenw10

                ifconfig ixl0 -vvvm returns ifconfig: -vvvm: bad value

                ifconfig results for both ports below. I left out ip and mac info

                ixl0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
                description: WAN
                options=e100bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER,RXCSUM_IPV6,TXCSUM_IPV6>
                media: Ethernet autoselect (2500Base-T <full-duplex,rxpause>)
                status: active
                nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

                ixl1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
                description: LAN
                options=e100bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER,RXCSUM_IPV6,TXCSUM_IPV6>
                media: Ethernet autoselect (5000Base-T <full-duplex,rxpause>)
                status: active
                nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

                Capture1.PNG
                Capture2.PNG

                1 Reply Last reply Reply Quote 0
                • F
                  froboz @stephenw10 last edited by froboz

                  @stephenw10

                  Thanks for the link I think I will try and switch to auto and then manually set flow control to receive pause. The question is would the entry be hw.ix.flow_control=1 or hw.ixl1.flow_control=1 ? and can I just run that as a temp command to see if it works without editing the file?

                  1 Reply Last reply Reply Quote 0
                  • stephenw10
                    stephenw10 Netgate Administrator last edited by

                    Sorry got the switches is the wrong order, clearly not my best day!
                    Should be ifconfig -vvvm ixl0

                    The 700 series NIC uses the ixl(4) driver, ix is for earlier Intel NICs.

                    You should be able to set that value per NIC as a test at the command line using, for example:

                    [admin@7100.stevew.lan]/root: sysctl dev.ixl.0.fc=0
                    dev.ixl.0.fc: 1 -> 0
                    

                    Yes, tested that works immediately.

                    Steve

                    F 1 Reply Last reply Reply Quote 0
                    • F
                      froboz @stephenw10 last edited by

                      @stephenw10 I tried the command sysctl dev.ixl.1.fc=1 and it worked the 10gbe was now showing rxpause but the bad new is my connection was still only about 100mbps. I am sure I pushing my luck with a beta of pfsense and a brand new switch from QNAP and a newer .. I think intel card. 5gbe is perfectly fine for me and so none of this is a big deal but I am still willing to troubleshoot if anyone finds this stuff interesting.

                      ifconfig -vvvm ixl1

                      ixl1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
                      description: LAN
                      options=e100bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER,RXCSUM_IPV6,TXCSUM_IPV6>
                      capabilities=f507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,NETMAP,RXCSUM_IPV6,TXCSUM_IPV6>

                      media: Ethernet autoselect (5000Base-T <full-duplex,rxpause>)
                      status: active
                      supported media:
                      	media autoselect
                      	media 10Gbase-T
                      	media 5000Base-T
                      	media 2500Base-T
                      	media 1000baseT
                      	media 100baseTX
                      nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
                      

                      Capture5.PNG

                      1 Reply Last reply Reply Quote 0
                      • stephenw10
                        stephenw10 Netgate Administrator last edited by

                        So with the switch set to 5G fixed the pfSense interface auto-negotiates to 5G and you see the full 1.2Gbps across it?

                        But if you set both ends to autoselect it tries to comes up at 10G and you see ~100Mbps?
                        That sounds a lot like a bad cable.

                        Check the interface status when it's not passing as expected, do you see errors or collisions?

                        Steve

                        F 2 Replies Last reply Reply Quote 0
                        • F
                          froboz @stephenw10 last edited by

                          @stephenw10 Yes, you are correct in your summary of the problem. I am using a cable matter 5 ft cat 6 550mhz 24 gauge cable currently between the switch and router but I will switch that out and I was thinking of also trying a different port on the switch in case that port is maybe bad. I will check for collisions or errors as well. I have to wait until after 5pm today but I will give it a go and report back what I find. Thanks for the help.

                          1 Reply Last reply Reply Quote 0
                          • F
                            froboz @stephenw10 last edited by

                            @stephenw10 To summarize switching cable and ports made no difference at all but .... I did discover that if I force the switch to 5gbe and turn off flow control also from the switch I get the same 100mbps problem. So everything is pointing to flow control being the issue. Is it possible that command sysctl dev.ixl.1.fc=1 made rxpause appear but did not actually turn it on? like display only?

                            Pictures below because who doesn't like photos:

                            So this is internet speed test with switch port forced to 5gbe flow control on:

                            5gbeCapture.PNG

                            Here is after switching physical ports and cable with port set to auto negotiate 10gbe also error checks I did

                            differentportandcable.PNG
                            autoafterspeedtest.PNG

                            1 Reply Last reply Reply Quote 0
                            • stephenw10
                              stephenw10 Netgate Administrator last edited by

                              Seeing rxpause there does indicate flowcontrol is active. It often would show txpause there too though.
                              Usually it would be disabling flowcontrol that is required if any change is actually required. Flow control is enabled by default on most 10GbE connections.

                              There's probably not that much that can be done here. Setting the switch to 5G fixed seems like reasonable option unless there is something you are trying to achieve beyond that?

                              Steve

                              F 1 Reply Last reply Reply Quote 0
                              • F
                                froboz @stephenw10 last edited by froboz

                                @stephenw10 No, not really trying to do much other than future proof. Maybe this will help someone down the road a bit that is running into this problem as everything is really stock here . Thanks again for the help!

                                EDIT: By stock I mean a fresh install with no add ons and very little setting changes.

                                1 Reply Last reply Reply Quote 0
                                • First post
                                  Last post

                                Products

                                • Platform Overview
                                • TNSR
                                • pfSense Plus
                                • Appliances

                                Services

                                • Training
                                • Professional Services

                                Support

                                • Subscription Plans
                                • Contact Support
                                • Product Lifecycle
                                • Documentation

                                News

                                • Media Coverage
                                • Press
                                • Events

                                Resources

                                • Blog
                                • FAQ
                                • Find a Partner
                                • Resource Library
                                • Security Information

                                Company

                                • About Us
                                • Careers
                                • Partners
                                • Contact Us
                                • Legal
                                Our Mission

                                We provide leading-edge network security at a fair price - regardless of organizational size or network sophistication. We believe that an open-source security model offers disruptive pricing along with the agility required to quickly address emerging threats.

                                Subscribe to our Newsletter

                                Product information, software announcements, and special offers. See our newsletter archive to sign up for future newsletters and to read past announcements.

                                © 2021 Rubicon Communications, LLC | Privacy Policy