• Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login
Netgate Discussion Forum
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login

Mixing different NIC Speeds (1Gb & 10Gb) Performance Problem Question

Hardware
6
166
13.9k
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • N
    ngr2001 @stephenw10
    last edited by Feb 11, 2025, 4:54 PM

    @stephenw10

    Just tried, same diff, speeds quickly taper down to 5-600Mbps.

    I clear my stats on switch, right now at least the dropped packet count is not increasing.

    1 Reply Last reply Reply Quote 0
    • S
      stephenw10 Netgate Administrator
      last edited by Feb 11, 2025, 5:01 PM

      Just to be clear you are running the switch at layer2 only right?

      It's hard to see how adding the switch, or just changing the port speed, can have any effect on the tcp dircetly. It could be that is some how broken upstream such that when introducing some lower lever issue it's very badly effected though. 🤔

      N 2 Replies Last reply Feb 11, 2025, 5:06 PM Reply Quote 0
      • N
        ngr2001 @stephenw10
        last edited by Feb 11, 2025, 5:06 PM

        @stephenw10

        I am not using any layer 3 features that I am aware of right now, thus I guess it should be in layer 2. I do have flow control enabled on the switch though.

        SSH@romulus#show run
        Current configuration:
        !
        ver 08.0.95pT213
        !
        stack unit 1
        module 1 icx7250-48-port-management-module
        module 2 icx7250-sfp-plus-8port-80g-module
        stack-port 1/2/1
        stack-port 1/2/3
        !

        vlan 1 name DEFAULT-VLAN by port
        router-interface ve 1
        !
        !
        symmetrical-flow-control enable
        !
        optical-monitor
        optical-monitor non-ruckus-optic-enable
        aaa authentication web-server default local
        aaa authentication login default local
        enable aaa console
        hostname romulus
        ip dhcp-client disable
        ip dns server-address 10.0.0.1
        ip route 0.0.0.0/0 10.0.0.1
        !
        no telnet server
        !

        clock timezone us Eastern
        !
        !
        ntp
        disable serve
        server time.cloudflare.com
        !
        !
        no web-management http
        !
        manager disable
        !
        !
        manager port-list 987
        !
        !
        interface ethernet 1/1/4
        flow-control neg-on
        !
        interface ethernet 1/1/8
        flow-control neg-on
        !
        interface ethernet 1/1/48
        flow-control neg-on
        !
        interface ethernet 1/2/1
        flow-control neg-on
        !
        interface ethernet 1/2/8
        flow-control neg-on
        !
        interface ve 1
        ip address 10.0.0.3 255.255.255.0
        !

        ip ssh idle-time 120
        !
        end
        SSH@romulus#

        1 Reply Last reply Reply Quote 0
        • N
          ngr2001 @stephenw10
          last edited by ngr2001 Feb 11, 2025, 5:10 PM Feb 11, 2025, 5:10 PM

          @stephenw10

          So I believe I upgraded to the Layer 3 firmware and enabled it from what I recall.

          HW: Stackable ICX7250-48

          UNIT 1: SL 1: ICX7250-48 48-port Management Module

            Software Package: **ICX7250_L3_SOFT_PACKAGE**
            Current License: l3-prem-8X10G
            P-ASIC  0: type B344, rev 01  Chip BCM56344_A0
          
          1 Reply Last reply Reply Quote 0
          • S
            stephenw10 Netgate Administrator
            last edited by Feb 11, 2025, 5:22 PM

            Right. As long as it's not actually routing that shouldn't matter.

            It sure seems like it's failing to negotiate flow control though. What does: ifconfig -vvm ix1 show in pfSense?

            N 2 Replies Last reply Feb 11, 2025, 5:39 PM Reply Quote 0
            • N
              ngr2001 @stephenw10
              last edited by ngr2001 Feb 11, 2025, 5:42 PM Feb 11, 2025, 5:39 PM

              This post is deleted!
              1 Reply Last reply Reply Quote 0
              • N
                ngr2001 @stephenw10
                last edited by ngr2001 Feb 11, 2025, 5:49 PM Feb 11, 2025, 5:41 PM

                @stephenw10

                When connected at 10Gb I get this:
                🔒 Log in to view

                When connected at 1Gb I get this:
                🔒 Log in to view

                L 1 Reply Last reply Feb 11, 2025, 5:58 PM Reply Quote 0
                • L
                  lnguyen @ngr2001
                  last edited by Feb 11, 2025, 5:58 PM

                  @ngr2001 Maybe the 10GBASE-T SFP+ module you have is not passing along the 802.3x FC negotiation to the SFP+ port on the Brocade 7250

                  1 Reply Last reply Reply Quote 0
                  • S
                    stephenw10 Netgate Administrator
                    last edited by Feb 11, 2025, 6:15 PM

                    Yes I would start to suspect the module. It should negotiate that over base-T but....
                    For other media types it's fixed like:

                    ix1: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
                    	description: WAN4
                    	options=4e138bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,MEXTPG>
                    	capabilities=4f53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,NETMAP,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,MEXTPG>
                    	ether 00:08:a2:12:e2:cb
                    	inet6 fe80::208:a2ff:fe12:e2cb%ix1 prefixlen 64 scopeid 0x6
                    	media: Ethernet autoselect (10Gbase-Twinax <full-duplex,rxpause,txpause>)
                    	status: active
                    	supported media:
                    		media autoselect
                    		media 1000Base-KX
                    		media 10Gbase-Twinax
                    	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
                    	drivername: ix1
                    	plugged: SFP/SFP+/SFP28 Unknown (Copper pigtail)
                    	vendor: OEM PN: SFP-H10GB-CU5M SN: CSS51F70287 DATE: 2015-07-01
                    

                    There it's just set to 'enabled' in the switch. I would try setting both ends as enabled rather than negotiated.

                    N 1 Reply Last reply Feb 11, 2025, 6:42 PM Reply Quote 0
                    • N
                      ngr2001 @stephenw10
                      last edited by ngr2001 Feb 11, 2025, 6:54 PM Feb 11, 2025, 6:42 PM

                      @stephenw10

                      I dont see any other options other than negotiate on the brocade side, are you suggesting I tweak the PF Side with a tunable ?

                      https://docs.commscope.com/bundle/fastiron-08095-managementguide/page/GUID-A5971868-1051-4807-8ED2-D3BC6B10AA3B.html

                      I also found this in the manual:

                      🔒 Log in to view

                      So what the heck does that mean, the manual does not offer any means or clues on if you can manually force flow control on a 10Gb port. To me this simply reads as not supported ?

                      1 Reply Last reply Reply Quote 0
                      • S
                        stephenw10 Netgate Administrator
                        last edited by Feb 11, 2025, 6:57 PM

                        @ngr2001 said in Mixing different NIC Speeds (1Gb & 10Gb) Performance Problem Question:

                        https://docs.commscope.com/bundle/fastiron-08095-managementguide/page/GUID-A5971868-1051-4807-8ED2-D3BC6B10AA3B.html

                        Auto-negotiation of flow control is not supported on 10 Gbps and 40 Gbps ports, fiber ports, and copper or fiber combination ports. 
                        

                        So you need to set a 10G port to enabled not neg-enabled.

                        1 Reply Last reply Reply Quote 0
                        • S
                          stephenw10 Netgate Administrator
                          last edited by Feb 11, 2025, 7:02 PM

                          There are multiple options there:

                          SSH@icx7250(config-if-e10000-1/2/1)#flow-control 
                            both            Flow Control in PAUSE generation and honoring mode.
                            generate-only   Flow Control in PAUSE generation only mode.
                            honor-only      Flow Control in  PAUSE honoring (Default) mode.
                            neg-on          Enable Flow Control with negotiation enabled
                            <cr>
                          

                          You probably want both

                          N 4 Replies Last reply Feb 11, 2025, 7:42 PM Reply Quote 0
                          • N
                            ngr2001 @stephenw10
                            last edited by Feb 11, 2025, 7:42 PM

                            @stephenw10

                            I did just see that too.

                            I just set it to both, so far not seeing any change on the PF Side. Going to reboot everything and check again.

                            🔒 Log in to view

                            1 Reply Last reply Reply Quote 0
                            • N
                              ngr2001 @stephenw10
                              last edited by Feb 11, 2025, 7:54 PM

                              @stephenw10

                              Set to Both on Brocade Port

                              No change on PF side after rebooting both PF and Switch.

                              🔒 Log in to view

                              1 Reply Last reply Reply Quote 0
                              • N
                                ngr2001 @stephenw10
                                last edited by Feb 11, 2025, 8:17 PM

                                @stephenw10

                                On the transceiver side I am using:

                                10Gtek 80-Meter, 10GBase-T SFP+ to RJ45 Transceiver
                                https://www.amazon.com/gp/product/B094N9YKN9/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&th=1

                                L 1 Reply Last reply Feb 11, 2025, 9:34 PM Reply Quote 0
                                • N
                                  ngr2001 @stephenw10
                                  last edited by Feb 11, 2025, 8:32 PM

                                  @stephenw10

                                  Found this gem, I think someone suggested its my transceiver, I bet it doesn't support flow control.

                                  🔒 Log in to view

                                  1 Reply Last reply Reply Quote 0
                                  • S
                                    stephenw10 Netgate Administrator
                                    last edited by Feb 11, 2025, 9:14 PM

                                    Urgh, well that would do it I guess.

                                    1 Reply Last reply Reply Quote 0
                                    • L
                                      lnguyen @ngr2001
                                      last edited by Feb 11, 2025, 9:34 PM

                                      @ngr2001 Did you buy the single speed 10GbE model or the quad speed 1/2.5/5/10GbE model?

                                      N 1 Reply Last reply Feb 11, 2025, 9:59 PM Reply Quote 0
                                      • N
                                        ngr2001 @lnguyen
                                        last edited by Feb 11, 2025, 9:59 PM

                                        @lnguyen

                                        The 80M single speed 10GbE model which to what I am researching is unlikely to support flow control, what a kick in the pants.

                                        L 1 Reply Last reply Feb 11, 2025, 10:01 PM Reply Quote 0
                                        • L
                                          lnguyen @ngr2001
                                          last edited by Feb 11, 2025, 10:01 PM

                                          @ngr2001 That is likely the case since the only factor that changes is moving the NIC from RJ45 switchport to SFP+ switchport. When does this Cisco switch you ordered arrive?

                                          N 1 Reply Last reply Feb 11, 2025, 10:20 PM Reply Quote 0
                                          76 out of 166
                                          • First post
                                            76/166
                                            Last post
                                          Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.