2.4.4p2 - 10G Between Physical XG-1537 only getting 1.9Gbps throughput with iPerf3
-
We're troubleshooting a bandwidth issue and narrowed it down to a pair of XG-1537's running 2.4.4p2. Even when running iPerf between the nodes and their 10G links in the same Layer 2 network we're seeing shy of 2Gbps of throughput. The cpu/mem utilization is low, the platform is capable of handling substantially more and there are very few firewall rules set - these are mostly just routers in their current state. We've done iperf tests using parallel streams as well but top out at around 3Gbps.
We also have setup QoS and adjusted the bandwidth incrementally from 0-10. We get the best performance and lack of retransmits when setting that value to 4Gbps but still don't get the full throughput we're expecting based on the platform and interfaces being used. We've scoured the configuration and can't find anything to explain this behavior.
Also in the iperf tests we're seeing hundreds of retransmits. When tuning the bandwidth we saw times of very little retransmits but the actual throughput didn't crest above 2Gbps. The cabling is fine and we're also observing this behavior on our virtual pfSense running the same version of code - in two different environments.
I understand that version of code is legacy but we can't upgrade at this time. Has anyone experienced this before? I did ping TAC but there's no known bugs in this release that would cause this.
-
If you are seeing re-transmits that indicates a HW problem. Have you verified the NICs?
If it is optical the likely source of re-transmits is the sender. The laser fails more often than the receiver. -
@comiccactus said in 2.4.4p2 - 10G Between Physical XG-1537 only getting 1.9Gbps throughput with iPerf3:
we can't upgrade at this time.
I do not understand this statement as the firewall is so outdated that the proper solution is to upgrade to current version...it's really reckless to the principles of network administrating...upgrade now, or late at night (2:00am, etc)...no more excuses.
-
@AndyRH - we're also seeing this on virtual pfSense firewalls in the environment in separate locations. We were also able to replicate it in a lab as well.
-
@NollipfSense Is there documentation that the current version has a bug for this issue that a newer version will resolve? Which version?
The apprehension to upgrade is due to a feature set change between versions that would yield a substantial overhaul. It's understood that we'll eventually have to undertake it but we're trying to solve the short-term issue while planning for the longer term.
-
@comiccactus said in 2.4.4p2 - 10G Between Physical XG-1537 only getting 1.9Gbps throughput with iPerf3:
Even when running iPerf between the nodes
Is that actually running iperf on pfSense? That will always give low results compared to testing through the firewall.
When testing in VMs were you using current versions? What NIC type?
Steve
-
Both iperf on pfsense and through it exposed the symptoms. That’s actually how we narrowed down where the issue was.
When running as vm’s we were running the same version as the physicals to rule out the hardware and associated items.
We intend to test again with newer versions but I was checking to see if folks were aware of a setting, bug, etc that they have run into before.
-
I'm not aware of anything in 2.4.4p2 but that was a long while ago, I may have just forgotten! There have been thousands of fixes since then.
