APU2 Slower than expected in iPerf3
-
I'm trying to determine the max speed I can get out of my APU2 but I don't have a very good solution for a real world test at the moment. The good news is that with everything loaded (although it takes a bit to fully boot) it does everything well. Snort, Squid, clam, pfBlocker, etc all performs well with relatively low CPU overhead. The problem is that I can't get iPerf3 to show anything higher than 300-340Mb/s. With nothing running and power set to Maximum I get around 340Mb/s. With everything loaded and power set to Adaptive I only get around 300Mb/s.
Let's assume 2 PCs and the APU2. Here's some stats:
If I set the APU2 to be server and client I get around 1.6Gb/s
If I set the APU2 to be server and connect from either PC I get around 320Mb/s +- 20Mb/s.
If I set a PC to be server and connect from the other PC I get around 933Mb/s +- 10Mb/s.If I use a crossover cable and connect a PC directly to the APU2 I get around 320Mb/s +- 20Mb/s.
The whole time during testing CPU doesn't go over 50%. How can I find what the bottleneck is and what's going on?
Thanks!
-
Using the APU2 as server or client is useless, the only result you need is this:
If I set a PC to be server and connect from the other PC I get around 933Mb/s +- 10Mb/s.
-
But that is PC to PC, without touching the router. I put that in to show that the PCs I were using weren't bottlenecking through the switch. How would going PC to PC show that everything is fine in the router?
-
Ooops, sorry. I thought it was PC to PC through pfsense.
That's how it must be done.
Connect PC server on WAN and PC client on LAN.
-
^That.
Running the server or client and terminating the tcp connection on the box is not a valid test.
Set powerd to hiadaptive if you haven't already.
The actual throughput results will be very dependent on the settings used in iperf also.
Steve
-
I'll try to get these tests run later today or tomorrow.
-
I can attest that iperf on any of my pfSense iterations is always slower than Linux or Windows on the same or similar hardware, just using default settings. Not sure this says anything about pfSense, maybe more about iperf. It's not a bad test, but your true throughput may or may not be indicated by an iperf test. IMO.
-
I finally got those tests run. I ran iperf3 with the switch -t 60. I then took the utilization samples at 30-35 seconds and recorded the throughput shown at the end of the run.
Power Snort Squid pfBlocker CPU % Throughput Mb/s Off On On On Snort 99 Send 505 Darkstat 7 Receive 505 User 24 Nice 0 System 30 Interrupts 24 Idle 23 Power Snort Squid pfBlocker CPU % Throughput Mb/s AdaptiveOn On On Snort 100 Send 501 Darkstat 7 Receive 501 User 26 Nice 0 System 25 Interrupts 17 Idle 30 Power Snort Squid pfBlocker CPU % Throughput Mb/s AdaptiveOff On On Snort 0 Send 540 Darkstat 7 Receive 540 User 2 Nice 0 System 26 Interrupts 21 Idle 51
A couple of notes.
1. I never received more than 550Mb/s. While that is plenty for what we would use it for, I'm surprised it is so low.
2. I always had interrupts at 15-25%. I thought that would go away when we switched from Realtek to Intel NICs.
3. It looks like iperf3 only utilizes 2 cores. Is there a way to get it to use all available resources?Edit: Clear up chart.
-
You can run multiple instances to use more cores.
It would be worth running```
top -aSHSteve
-
In normal the APU2C4 is set up as the following;
- install the mSATA and then pfSense on it
– perhaps activate the TRIM support for the storage
-- high up the mbuf size or numbers if that is really needed
-- activate Powerd (high adaptive)
PC1 (server) to PC2 (client) using two LAN Ports
PC1 (server) tp PC2 (client) using one LAN Port and the WAN Port (with a Switch in front of it)This might be the best option to know what you might be expecting form that kind of hardware
but then please without any kind of installed packet such as Snort, Squid, SuiqdGuard, pfBlockerNG,
and so on and so on, so you see it would be the best to test it with a fresh installed pfSense and not
later after all gets installed, so you will get the raw numbers and the numbers scaled down based on
the installed packets later!!! - install the mSATA and then pfSense on it