How to achieve 10G speeds?
-
Hi all, sorry for the NOOB question.
I have a pfSense box with a Chelsio T540-CR 4 port 10G SFP+
I connected my Win10 Intel NUC via Thunderbolt to SFP+ 10G to one port of my pfsense box.
On win10 side I see: connection: 10/10 (Gbps)
On pfSense interface I see: 10Gbase-LR <full-duplex,rxpause,txpause>But when I run an iPerf3 from Win10 client to pfsense server I get 127Mbits/sec sender and 126MBits/sec receiver
command I use: iperf3 -P 20 -c (ip address of pfSense)
I assume this is not 10G speed.
How can I troubleshoot this?
My CPU :
Intel(R) Xeon(R) CPU E5-2686 v4 @ 2.30GHz
Current: 2300 MHz, Max: 2301 MHz
36 CPUs: 1 package(s) x 18 core(s) x 2 hardware threads
AES-NI CPU Crypto: Yes (inactive)
QAT Crypto: NopfSense:
2.5.2-RELEASE (amd64)
built on Fri Jul 02 15:33:00 EDT 2021
FreeBSD 12.2-STABLEkind regards
-
It takes more than a 10G connection to achieve 10G. Is the rest of the hardware up to the task?
Here's what I get with a 1Gb connection, through a Cisco switch:
[SUM] 0.00-10.00 sec 1.08 GBytes 930 Mbits/sec 7077 sender
[SUM] 0.00-10.00 sec 1.08 GBytes 924 Mbits/sec receiverPfsense is running on the computer described in my sig and the client computer is an i7 CPU & 32GB memory.
Since that's much better than what you have, I suspect you may have a hardware or configuration issue.
Regardless, 10 Gb is a lot. I've only seen it in data centers over fibre.
-
@jknott said in How to achieve 10G speeds?:
I suspect you may have a hardware or configuration issue.
Yes indeed, but how to troubleshoot it and find the bottleneck?
kind regards
-
@gelcom said in How to achieve 10G speeds?:
run an iPerf3 from Win10 client to pfsense server
Netgate posts often to not run a test to pfSense itself as it's not optimized for processing. Run your test to something on the Internet.
Pre-2.7.2/23.09: Only install packages for your version, or risk breaking it. Select your branch in System/Update/Update Settings.
When upgrading, allow 10-15 minutes to restart, or more depending on packages and device speed.
Upvote š helpful posts! -
I got close to a full Gb here, so he's definitely got a problem with his connection somewhere.
-
Yes, that^. Try to test to something other than pfSense directly. Some other local host connected at 10G would be best.
I suspect you're hitting an issue in the NUC though with that sort of restriction. That hardware for pfSense will be capable of multi gigabit even when also running iperf.126Mbps is woefully low for that, there is something fundamentally wrong there.
Steve
-
@steveits said in How to achieve 10G speeds?:
Run your test to something on the Internet.
My internet link is 100/50 so I assume this wouldn't tell me much.
-
@stephenw10 said in How to achieve 10G speeds?:
Try to test to something other than pfSense directly. Some other local host connected at 10G would be best.
I have another port of my T540-CR connected to a Proxmox Host so I run a connection from my Proxmox Host to pfSense and this time with the following results:
[SUM] 0.00-10.00 sec 10.7 GBytes 9.22 Gbits/sec 0 sender
[SUM] 0.00-10.00 sec 10.7 GBytes 9.21 Gbits/sec receiverIt seems that my local PC is unable to get 10g speeds even when connection shows 10G.
I apologize since this is not a pfSense related issue.
kind regards and thanks for the prompt support.
-
@gelcom said in How to achieve 10G speeds?:
It seems that my local PC is unable to get 10g speeds even when connection shows 10G.
That's why I asked if the hardware is up to it. As I mentioned, 10G is a lot and I've only seen it in data centers over fibre. In fact, in between speeds, such as 2.5 and 5 Gb have been introduced, as 10 was too much of a jump.
-
Iām not convinced itās a computer power problem as when I run iperf I get only 15% CPU utilization. Itās a 32Gb i7. O would bet itās something related to network connection or setup as Iām doing a Thunderbolt 3 to sfp+ fiber from PC (Intel NUC) to pfsense. I updated to latest firmware but still no good.
Problem is I have no idea how to troubleshoot it.
Kind regards
-
Well, you apparently have a computer that can handle 10G. Try iperf against it from that Windows box and see how the performance is. You've already ruled out issues with pfsense and it's hardware.
-
@gelcom said in How to achieve 10G speeds?:
I would bet itās something related to network connection or setup as Iām doing a Thunderbolt 3 to sfp+ fiber from PC
Yes, me too. 126Mbps over a 10G link is something fundamentally mismatched.
Can you swap the NIC in the TB adapter for a 1G device and test that?Steve
-
@stephenw10 said in How to achieve 10G speeds?:
Can you swap the NIC in the TB adapter for a 1G device and test that?
Changed both sides to 1G. Sames results:
[SUM] 0.00-10.01 sec 148 MBytes 124 Mbits/sec sender
[SUM] 0.00-10.01 sec 146 MBytes 123 Mbits/sec receiver -
Still over the same hardware though?
I'm not sure what sort of TB adapter you have. Like an external PCIe cage or a dedicated adapter NIC.I assume the NUC itself has a 1G NIC and that can get ~940Mbps?
Steve
-
@stephenw10 said in How to achieve 10G speeds?:
Still over the same hardware though?
Same hardware: from Intel NUC6i7KYK i7 6770hq to pfSense Xeon E5-2686 v4
I'm not sure what sort of TB adapter you have. Like an external PCIe cage or a dedicated adapter NIC.
It's a dedicated Thunderbolt to SFP+ adapter: QNAP T310G1S
I assume the NUC itself has a 1G NIC and that can get ~940Mbps?
You're right. NUC has a 1G NIC too. If I connect to pfSense from this NIC I also get same results:
[SUM] 0.00-10.00 sec 170 MBytes 142 Mbits/sec sender
[SUM] 0.00-10.00 sec 167 MBytes 140 Mbits/sec receiverIt seems problem is not with TB adapter as 1G NIC is also capped at <140Mbits/sec. Maybe a WIN10 config or hardware issue.
kind regards
-
Hmm, well that's good news, it implies the TB 10G adapter is not the cause so probably can be made to work. Unless you have some traffic shaping set in pfSense it's hard to see what might cause that. Something in Windows guess though I'm not sure what.
Steve
