• Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login
Netgate Discussion Forum
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login

Throughput problems on 4100

Official Netgate® Hardware
7
28
3.2k
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • A
    ahxcjay @MaxK 0
    last edited by Oct 13, 2022, 10:25 PM

    @maxk-0

    I run a 3100 connected to Verizon Fios provisioned at 200 / 200 Mbps.

    If I was at that speed, I wouldn't even notice these issues. I'm at 940/880. I really dislike speedtest.net, fwiw, it's not remotely an indicator of speed performance. It's incredibly easy to send chunks of 1500 MTUs...this isn't real world performance.

    Anyway, here's a single threaded test:
    login-to-view

    Multi: (again, this has dropped since replacing my ER4 with the 4100)
    login-to-view

    When uploading to YouTube or Google Drive from Google Chrome, Firefox (on Windows or MacOS) the upload speed will hover around ~200Mit/sec. On ER4, it's at ~400-500Mbit/sec every time.

    I deliberately purchased the 4100 because I know the 1100 and 2100 are trash performance wise. To now realise the 4100 is a significant degradation in performance compared to MIPS / ER4 is beyond frustrating as I like the product overall.

    1 Reply Last reply Reply Quote 0
    • A
      ahxcjay @SteveITS
      last edited by ahxcjay Oct 13, 2022, 10:31 PM Oct 13, 2022, 10:28 PM

      @steveits said in Why is the 4100 performance such trash?:

      @ahxcjay Are you testing through the VPN?

      Nope. I have Wireguard and IPsec enabled (with just as many tunnels as ER-4 has [5])) but only very specific traffic goes through those links.

      Is hardware acceleration enabled? What settings?

      I haven't touched the defaults:

      login-to-view

      Per Netgate's store page:
      IPsec VPN
      (AES-GCM-128 w/QAT)

      IPERF3 Traffic: 960 Mbps
      IMIX Traffic: 312 Mbps

      ..those 'tests' are junk, IMO. Did you see how they performed them? They literally bond 4 interfaces together. That is not even remotely 'real world'.

      What gets me most about my issue is that I don't even see the system (single CPU or otherwise) go under any load. It looks like it has plenty of CPU cycles to spare, but here we are. I'm going to disable ALTQ support (probably enabled when I tried out the traffic shaper) and see what that does.

      A S 2 Replies Last reply Oct 13, 2022, 10:33 PM Reply Quote 0
      • A
        ahxcjay @ahxcjay
        last edited by ahxcjay Oct 13, 2022, 10:41 PM Oct 13, 2022, 10:33 PM

        @ahxcjay ..disabling ALTQ has zero effect. I can still only upload to google drive / youtube at ~200Mbit/sec. The moment I switch my cables back to the ER-4, I'm back up to ~500Mbit/sec. Infuriating.

        I want to like this product, I really like some of the features, but it's letting me down on the core fundamentals.

        Here's my software package list:
        login-to-view

        I have no interest in DPI etc..so that's not chugging my perf.

        If I open 2x upload streams in the same browser to GDrive & YouTube my upload speed essentially doubles:
        login-to-view

        .. I just can not get a single upload to sustain past 200-250Mb/sec. A simple cable swap to ER4 and an individual upload is at 500+.

        1 Reply Last reply Reply Quote 0
        • S
          SteveITS Galactic Empire @ahxcjay
          last edited by SteveITS Oct 13, 2022, 10:52 PM Oct 13, 2022, 10:50 PM

          @ahxcjay said in Why is the 4100 performance such trash?:

          Is hardware acceleration enabled? What settings?

          I haven't touched the defaults

          I meant the crypto. But that's irrelevant if you're not going through the VPN. (I was looking for, on the dashboard in the CPU Type section what are the "Crypto" entries? The config settings are under System/Advanced/Miscellaneous.)

          Speed/duplex correct? Change patch cables? Small switch between the 4100 and ISP router? It may seem weird but those things come up on the forum reasonably often.

          Edit: hmm, that doesn't make a lot of sense either if multiple streams are way faster. Does it change if you disable the traffic monitoring?

          We have a client with a 4100 but they have a ~350 Mbps download.

          Pre-2.7.2/23.09: Only install packages for your version, or risk breaking it. Select your branch in System/Update/Update Settings.
          When upgrading, allow 10-15 minutes to restart, or more depending on packages and device speed.
          Upvote 👍 helpful posts!

          A 1 Reply Last reply Oct 13, 2022, 10:54 PM Reply Quote 0
          • A
            ahxcjay @SteveITS
            last edited by ahxcjay Oct 13, 2022, 10:58 PM Oct 13, 2022, 10:54 PM

            @steveits I meant the crypto. But that's irrelevant if you're not going through the VPN. (I > was looking for, on the dashboard in the CPU Type section what are the "Crypto" entries?

            QAT Crypto: Yes (active)
            Hardware crypto AES-CBC,AES-CCM,AES-GCM,AES-ICM,AES-XTS,SHA1,SHA256,SHA384,SHA512

            The config settings are under System/Advanced/Miscellaneous.)

            What settings should I post from here..?

            Speed/duplex correct?

            Yes. 1000, no issues.

            Change patch cables?

            Tried that, and remember, all I need to do is literally swap out the cables to the ER-4 (wan and to my Switch) and the upload speeds to GDrive / YT is in line with expectations.

            Small switch between the 4100 and ISP router?

            Nope. Direct into patch panel.

            It may seem weird but those things come up on the forum reasonably often.

            I understand, but isn't this issue beyond ridiculous? Why is pfSense failing on this? My mate told me to avoid pfSense at all costs and I ignored him (he recommend an SRX).

            Does it change if you disable the traffic monitoring?

            vnstat? Tried disabling that also; makes zero difference.

            S 1 Reply Last reply Oct 13, 2022, 11:01 PM Reply Quote 0
            • S
              SteveITS Galactic Empire @ahxcjay
              last edited by Oct 13, 2022, 11:01 PM

              @ahxcjay said in Why is the 4100 performance such trash?:

              Nope. Direct into patch panel.

              I'm writing too fast. :) I meant add a switch. But that doesn't make sense if multiple streams are 2x fast because it's not being throttled at that network level.

              re: System/Advanced/Miscellaneous, the crypto setting, but never mind, it's not relevant.

              One thing you could try is to back up your config, set to factory defaults, and reproduce that way. If it's fast, it's a config issue. Worst case restore your config to get back to where you are now. And can restore parts of the config to test.

              Pre-2.7.2/23.09: Only install packages for your version, or risk breaking it. Select your branch in System/Update/Update Settings.
              When upgrading, allow 10-15 minutes to restart, or more depending on packages and device speed.
              Upvote 👍 helpful posts!

              A 1 Reply Last reply Oct 13, 2022, 11:05 PM Reply Quote 0
              • A
                ahxcjay @SteveITS
                last edited by Oct 13, 2022, 11:05 PM

                @steveits ..that's one for the weekend.. ;)

                All other (single threaded) web upload activities suffer this issue. Dropbox, iCloud..etc. Once I do something multi-threaded, performance is completely fine.

                To @keyser - where is your gigabit connection with a 4100 against a single threaded upload via a web browser? I'm all eyes...

                K 1 Reply Last reply Oct 14, 2022, 6:30 AM Reply Quote 0
                • S
                  stephenw10 Netgate Administrator
                  last edited by Oct 13, 2022, 11:37 PM

                  What interfaces are you using on the 4100?

                  Try connecting to command line and running top -HaSP whilst you're uploading.

                  Is either core hitting 100% load?

                  What loading from processes is there?

                  Steve

                  A 1 Reply Last reply Oct 13, 2022, 11:40 PM Reply Quote 0
                  • A
                    ahxcjay @stephenw10
                    last edited by ahxcjay Oct 13, 2022, 11:47 PM Oct 13, 2022, 11:40 PM

                    @stephenw10

                    What interfaces are you using on the 4100?

                    WAN1 & LAN4.

                    Try connecting to command line and running top -HaSP whilst you're
                    uploading.

                    ..did that earlier and noticed nothing. Let me try again.

                    login-to-view

                    Is either core hitting 100% load?

                    Barely under any load..

                    What loading from processes is there?

                    Hardly anything. See screenshot.

                    1 Reply Last reply Reply Quote 0
                    • S
                      stephenw10 Netgate Administrator
                      last edited by Oct 13, 2022, 11:47 PM

                      Hmm, that really is almost nothing.

                      I wonder if there's a flow-control issue there.

                      I assume WAN and LAN are linked at 1G?

                      As a test try reassigning the interfaces so both WAN and LAN are using the ix NICs or both using the igc NICs.

                      When you're testing that is the throughput constant or very 'peaky'?

                      Steve

                      A 1 Reply Last reply Oct 13, 2022, 11:55 PM Reply Quote 0
                      • A
                        ahxcjay @stephenw10
                        last edited by ahxcjay Oct 13, 2022, 11:56 PM Oct 13, 2022, 11:55 PM

                        @stephenw10 said in Why is the 4100 performance such trash?:

                        Hmm, that really is almost nothing.

                        I wonder if there's a flow-control issue there.

                        ..I would love to know what it is.. :)

                        I assume WAN and LAN are linked at 1G?

                        Yep. All good there.

                        As a test try reassigning the interfaces so both WAN and LAN are using the ix NICs or both using the igc NICs.

                        Let me go and do that now. I'll use the 'igc' NICs.

                        When you're testing that is the throughput constant or very 'peaky'?

                        Pretty constant I would say...

                        login-to-view

                        1 Reply Last reply Reply Quote 0
                        • A
                          ahxcjay
                          last edited by ahxcjay Oct 14, 2022, 12:03 AM Oct 14, 2022, 12:02 AM

                          O M G

                          IT FIXED IT !

                          login-to-view

                          ..this is a bug, right?

                          B 1 Reply Last reply Oct 14, 2022, 9:34 AM Reply Quote 0
                          • K
                            keyser Rebel Alliance @ahxcjay
                            last edited by Oct 14, 2022, 6:30 AM

                            @ahxcjay said in Why is the 4100 performance such trash?:

                            To @keyser - where is your gigabit connection with a 4100 against a single threaded upload via a web browser? I'm all eyes...

                            Sorry I cannot complete the test for you right now, as I only have SG-4100s at customer sites. My own boxes are SG-2100/SG-6100.
                            But i distinctly remember doing a single stream test and still seeing a 9xxMbps number on a QinQ line that will do about 940Mbps @ 100%.

                            I’m happy Steve found the issue (VERY likely flow control), and that’s exactly my point with my first post. People in here are very very helpfull and know A LOT about these boxes, potential issues and what not.
                            We alle want to help, but it’s not very motivating when posts like yours just flame the product by stating that the specs and performance of these boxes are invented ficticious numbers from netgate.

                            The problem is that google searches from people also finds this post, and some people only read headlines….

                            Love the no fuss of using the official appliances :-)

                            A 1 Reply Last reply Oct 14, 2022, 3:52 PM Reply Quote 1
                            • B
                              bingo600 @ahxcjay
                              last edited by Oct 14, 2022, 9:34 AM

                              @ahxcjay said in Why is the 4100 performance such trash?:

                              O M G

                              IT FIXED IT !

                              login-to-view

                              ..this is a bug, right?

                              I have read this two times now ....
                              What fixed it ????

                              Factory reset or ????

                              If you find my answer useful - Please give the post a 👍 - "thumbs up"

                              pfSense+ 23.05.1 (ZFS)

                              QOTOM-Q355G4 Quad Lan.
                              CPU  : Core i5 5250U, Ram : 8GB Kingston DDR3LV 1600
                              LAN  : 4 x Intel 211, Disk  : 240G SAMSUNG MZ7L3240HCHQ SSD

                              G 1 Reply Last reply Oct 14, 2022, 11:11 AM Reply Quote 0
                              • G
                                Gertjan @bingo600
                                last edited by Oct 14, 2022, 11:11 AM

                                @bingo600 said in Why is the 4100 performance such trash?:

                                I have read this two times now ....
                                What fixed it ????

                                Me 3 times.
                                And I have a 4100 in front of my.
                                I've 4 igc0-1-2-3 interfaces, an ix2 and an ix3. The latter two are combo ports :
                                See here.

                                I presume the fix was : use WAN on igc0 and all the LANs on igc1, 2 and 3 - not using ix2 and ix3.

                                No "help me" PM's please. Use the forum, the community will thank you.
                                Edit : and where are the logs ??

                                A 1 Reply Last reply Oct 14, 2022, 1:47 PM Reply Quote 1
                                • S
                                  stephenw10 Netgate Administrator
                                  last edited by Oct 14, 2022, 11:50 AM

                                  Yeah, I read that as using igc as WAN fixed it. In which case it's probably linking differently than ix to whatever is upstream. Potentially with or without flow-control.
                                  You'd have to check the ifconfig output from each to know more.

                                  Steve

                                  A 1 Reply Last reply Oct 14, 2022, 1:45 PM Reply Quote 1
                                  • A
                                    ahxcjay @stephenw10
                                    last edited by Oct 14, 2022, 1:45 PM

                                    @stephenw10 igc as WAN fixed it. What would cause this behaviour..?

                                    S 1 Reply Last reply Oct 14, 2022, 2:02 PM Reply Quote 0
                                    • A
                                      ahxcjay @Gertjan
                                      last edited by Oct 14, 2022, 1:47 PM

                                      @gertjan correct. Once I removed ix from the ports used my line is at full speed on all uploads again! Amazing.

                                      Thank you everyone!

                                      1 Reply Last reply Reply Quote 0
                                      • S
                                        stephenw10 Netgate Administrator @ahxcjay
                                        last edited by Oct 14, 2022, 2:02 PM

                                        @ahxcjay said in Throughput problems on 4100:

                                        What would cause this behaviour..?

                                        I would think it must be something in the link negotiation there. I would check the ifconfig -vv output for each NIC to start with. It 'feels' like a flow control issue. You would see 'rxpause, txpause' in that output. It can be disabled (or enabled) on ix if it is that.

                                        Steve

                                        A G 2 Replies Last reply Oct 14, 2022, 2:10 PM Reply Quote 1
                                        • A
                                          ahxcjay @stephenw10
                                          last edited by Oct 14, 2022, 2:10 PM

                                          @stephenw10 great info. Thank you! I am so happy this is fixed as I love the product.

                                          1 Reply Last reply Reply Quote 1
                                          15 out of 28
                                          • First post
                                            15/28
                                            Last post
                                          Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.