Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Low routing performance new build, N3160, i211-AT

    Scheduled Pinned Locked Moved Hardware
    15 Posts 8 Posters 5.5k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • w0wW
      w0w
      last edited by

      Did you try tunes for IGB?
      https://doc.pfsense.org/index.php/Tuning_and_Troubleshooting_Network_Cards
      Something else like
      kern.ipc.nmbclusters="1000000"
      hw.igb.rxd="4096"
      hw.igb.txd="4096"
      net.inet.tcp.syncache.hashsize=1024
      net.inet.tcp.syncache.bucketlimit=100
      net.isr.defaultqlimit=4096
      net.link.ifqmaxlen=10240
      hw.igb.rx_process_limit="-1"
      hw.igb.num_queues=2
      dev.igb.0.fc=0
      dev.igb.1.fc=0

      1 Reply Last reply Reply Quote 1
      • H
        heper
        last edited by

        measure thoughput, not from or to pfSense

        1 Reply Last reply Reply Quote 0
        • I
          iphonelover
          last edited by

          I tried adjusting the tunables, one at a time, I would make the setting change in the UI, save and then reboot and retest.  Nothing seemed to make any difference, positive or negative.  I guess, I am just going to have to get all of my devices inline to do a true test of the routing.

          I will keep plugging away, but I have never seen where perf was not able to show what the server/box/router was capable of before, so I am guessing once I get this test done, I will see the same results of only routing in the 300-400 Mbps range, instead of the 900ish I was expecting.

          1 Reply Last reply Reply Quote 0
          • P
            pfBasic Banned
            last edited by

            @iphonelover:

            I can iperf between each client with gigabit speeds, so I feel that I can rule out the clients and the switch as being the issue.

            As for the router, I have tried both 2.3 and 2.4 and have gotten similar results on each.  I am at a loss at what to troubleshoot next.  I have included the system activity while running an iperf test in which the router was only returning 350 Mb/s to the client, in the hope some sees something that is off.

            You're just using the iperf package in the wrong way. If you click the question mark icon on the pfsense page while using the package it will take you to the wiki page for the package.

            https://doc.pfsense.org/index.php/Iperf_package

            Usage
            iperf running on pfSense is NOT a suitable way of testing firewall throughput, as there is a significant difference between performance of traffic initiated or terminated on the firewall and traffic traversing the firewall. There are many suitable uses for iperf running on pfSense, but testing the throughput capabilities of the firewall is not one of them.

            1 Reply Last reply Reply Quote 0
            • P
              pfBasic Banned
              last edited by

              That being said, my box gets gigabit throughput any way i run iperf. From it, to it, across it. I only saw ~5% on top while it was running.

              I don't think the problem is your hardware though. It doesn't take much to just route at gigabit speeds and that's a pretty modern processor.

              Capture.JPG
              Capture.JPG_thumb

              1 Reply Last reply Reply Quote 0
              • P
                pfBasic Banned
                last edited by

                Are you using a sideloaded version of iperf or the iperf package?

                1 Reply Last reply Reply Quote 0
                • I
                  iphonelover
                  last edited by

                  I used both, the perf package, which is version 2, and installed iperf3 from the shell.

                  So I just finished testing this setup-

                  client1 running iperf3 -s connected to router on the wan port
                  client2 running iperf3 -c connected to router on the lan port

                  I didn't have my routes set up correctly, so I couldn't make the full connection, but I was able to iperf from the router lan interface to client1 at full gig speed of about 940 Mbps.

                  While not 100% conclusive it does lead me to believe that I can route at full gig speed.

                  Also I found that the Intel speed step was disabled in the BIOS, so I enabled that, and checked the PowerD settings in Pfsense, and cranked everything up to 11 and I can perf at 600 Mbps, where as before it was 350 Mbps.

                  1 Reply Last reply Reply Quote 0
                  • P
                    pfBasic Banned
                    last edited by

                    yeah that hardware should have no issues routing full gig without any tweaking.

                    1 Reply Last reply Reply Quote 0
                    • T
                      tirsojrp
                      last edited by

                      Do not run iperf on the pfsense box.

                      1 Reply Last reply Reply Quote 0
                      • VioletDragonV
                        VioletDragon
                        last edited by VioletDragon

                        This can be fixed by enabling PowerD and set it to Hiadaptive and applying the following tweaks for network stack. once applied reboot and test.

                        Disable hardware checksum offload

                        Disable hardware TCP segmentation offload

                        Disable hardware large receive offload

                        kern.ipc.nmbclusters="1000000"
                        hw.igb.rxd="4096"
                        hw.igb.txd="4096"
                        net.inet.tcp.syncache.hashsize=1024
                        net.inet.tcp.syncache.bucketlimit=100
                        net.isr.defaultqlimit=4096
                        net.link.ifqmaxlen=10240
                        hw.igb.rx_process_limit="-1"
                        hw.igb.num_queues=2
                        dev.igb.0.fc=0
                        dev.igb.1.fc=0

                        Q 1 Reply Last reply Reply Quote 1
                        • Q
                          quaint-engineer @VioletDragon
                          last edited by

                          @VioletDragon said in Low routing performance new build, N3160, i211-AT:

                          This can be fixed by enabling PowerD and set it to Hiadaptive and applying the following tweaks for network stack. once applied reboot and test.

                          Disable hardware checksum offload

                          Disable hardware TCP segmentation offload

                          Disable hardware large receive offload

                          kern.ipc.nmbclusters="1000000"
                          hw.igb.rxd="4096"
                          hw.igb.txd="4096"
                          net.inet.tcp.syncache.hashsize=1024
                          net.inet.tcp.syncache.bucketlimit=100
                          net.isr.defaultqlimit=4096
                          net.link.ifqmaxlen=10240
                          hw.igb.rx_process_limit="-1"
                          hw.igb.num_queues=2
                          dev.igb.0.fc=0
                          dev.igb.1.fc=0

                          Thanks! This got me from 144Mbits/sec -> 204Mbits/sec -> 940Mbits/sec on my Odyssey X86J4105800. 👍

                          1 Reply Last reply Reply Quote 2
                          • VioletDragonV
                            VioletDragon
                            last edited by

                            That is great. Glad it helped :D

                            B 1 Reply Last reply Reply Quote 0
                            • B
                              BuiltOnSelfSuccess @VioletDragon
                              last edited by

                              @violetdragon

                              I already had:
                              PowerD enabled and set to Hiadaptive ,
                              Disable hardware large receive offload
                              Disable hardware checksum offload

                              So I went and made your suggested changes:
                              Disable hardware TCP segmentation offload
                              and I added all of the network stack tweaks that you suggested but I haven't seen an improvement in speed, are there ay other tweaks that you can suggest or different values for the tweaks that you'll already suggested that I can try please?

                              Thanks in advance.

                              B 1 Reply Last reply Reply Quote 0
                              • B
                                BuiltOnSelfSuccess @BuiltOnSelfSuccess
                                last edited by

                                I found another post (https://hardforum.com/threads/pfsense-2-5-new-configurations-for-igb-performance.2008210/#post-1044942639) and made some additional changes (only to the tunables as I'm not confident and knowledgeable enough to modify the loader.conf) which seemed to improve throughput, which one of these did the trick I have no idea but on my 200/20 speed I was previously getting around 90/12 I'm now achieving around 150/16 but I know that my device has previously been capable of 180/19 so I'll keep playing until I can improve the speed further but I'd be very grateful for any additional ideas/changes/tweaks etc...

                                1 Reply Last reply Reply Quote 0
                                • First post
                                  Last post
                                Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.