Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    2.4.4p2 - 10G Between Physical XG-1537 only getting 1.9Gbps throughput with iPerf3

    Scheduled Pinned Locked Moved General pfSense Questions
    8 Posts 4 Posters 681 Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • C
      comiccactus
      last edited by

      We're troubleshooting a bandwidth issue and narrowed it down to a pair of XG-1537's running 2.4.4p2. Even when running iPerf between the nodes and their 10G links in the same Layer 2 network we're seeing shy of 2Gbps of throughput. The cpu/mem utilization is low, the platform is capable of handling substantially more and there are very few firewall rules set - these are mostly just routers in their current state. We've done iperf tests using parallel streams as well but top out at around 3Gbps.

      We also have setup QoS and adjusted the bandwidth incrementally from 0-10. We get the best performance and lack of retransmits when setting that value to 4Gbps but still don't get the full throughput we're expecting based on the platform and interfaces being used. We've scoured the configuration and can't find anything to explain this behavior.

      Also in the iperf tests we're seeing hundreds of retransmits. When tuning the bandwidth we saw times of very little retransmits but the actual throughput didn't crest above 2Gbps. The cabling is fine and we're also observing this behavior on our virtual pfSense running the same version of code - in two different environments.

      I understand that version of code is legacy but we can't upgrade at this time. Has anyone experienced this before? I did ping TAC but there's no known bugs in this release that would cause this.

      NollipfSenseN stephenw10S 2 Replies Last reply Reply Quote 0
      • AndyRHA
        AndyRH
        last edited by

        If you are seeing re-transmits that indicates a HW problem. Have you verified the NICs?
        If it is optical the likely source of re-transmits is the sender. The laser fails more often than the receiver.

        o||||o
        7100-1u

        C 1 Reply Last reply Reply Quote 0
        • NollipfSenseN
          NollipfSense @comiccactus
          last edited by

          @comiccactus said in 2.4.4p2 - 10G Between Physical XG-1537 only getting 1.9Gbps throughput with iPerf3:

          we can't upgrade at this time.

          I do not understand this statement as the firewall is so outdated that the proper solution is to upgrade to current version...it's really reckless to the principles of network administrating...upgrade now, or late at night (2:00am, etc)...no more excuses.

          pfSense+ 23.09 Lenovo Thinkcentre M93P SFF Quadcore i7 dual Raid-ZFS 128GB-SSD 32GB-RAM PCI-Intel i350-t4 NIC, -Intel QAT 8950.
          pfSense+ 23.09 VM-Proxmox, Dell Precision Xeon-W2155 Nvme 500GB-ZFS 128GB-RAM PCIe-Intel i350-t4, Intel QAT-8950, P-cloud.

          C 1 Reply Last reply Reply Quote 0
          • C
            comiccactus @AndyRH
            last edited by

            @AndyRH - we're also seeing this on virtual pfSense firewalls in the environment in separate locations. We were also able to replicate it in a lab as well.

            1 Reply Last reply Reply Quote 0
            • C
              comiccactus @NollipfSense
              last edited by

              @NollipfSense Is there documentation that the current version has a bug for this issue that a newer version will resolve? Which version?

              The apprehension to upgrade is due to a feature set change between versions that would yield a substantial overhaul. It's understood that we'll eventually have to undertake it but we're trying to solve the short-term issue while planning for the longer term.

              1 Reply Last reply Reply Quote 0
              • stephenw10S
                stephenw10 Netgate Administrator @comiccactus
                last edited by

                @comiccactus said in 2.4.4p2 - 10G Between Physical XG-1537 only getting 1.9Gbps throughput with iPerf3:

                Even when running iPerf between the nodes

                Is that actually running iperf on pfSense? That will always give low results compared to testing through the firewall.

                When testing in VMs were you using current versions? What NIC type?

                Steve

                C 1 Reply Last reply Reply Quote 0
                • C
                  comiccactus @stephenw10
                  last edited by

                  @stephenw10

                  Both iperf on pfsense and through it exposed the symptoms. That’s actually how we narrowed down where the issue was.

                  When running as vm’s we were running the same version as the physicals to rule out the hardware and associated items.

                  We intend to test again with newer versions but I was checking to see if folks were aware of a setting, bug, etc that they have run into before.

                  1 Reply Last reply Reply Quote 0
                  • stephenw10S
                    stephenw10 Netgate Administrator
                    last edited by

                    I'm not aware of anything in 2.4.4p2 but that was a long while ago, I may have just forgotten! There have been thousands of fixes since then.

                    1 Reply Last reply Reply Quote 0
                    • First post
                      Last post
                    Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.