Upgrading Realtek with alternate driver - Is it worth it?
-
Ok...so over the weekend, I picked up a used SG-2200 pfSense appliance, which comes with 2 Intel I350 chipset based NIC's. Thought it could serve as redundant/failover box, so may as well have one.
After hours of testing this weekend with the SG-2200, I was disappointed I'm still only averaging ~450 Mbps with the Intel NIC's on the WAN side. On the LAN side, I'm also only able to get ~600Mbps, which is actually quite low. Even the SG-1100's are meant to hit close to 900 Mbps on the LAN side as others have previously reported on this forum.
I re-built my entire network this weekend, so all testing was without any packages/overhead on the pfSense side. All testing was done using iPerf3 for LAN and/or dedicated Speed Test apps on the client side for WAN's. Of interesting note was very high usage (~80-90%) on the CPU on the SG-2200, whereas on my AliExpress box, the CPU has never spiked over 7-10%.
Could my Unifi US-24/US-24 POE switches be the bottleneck here? Considering they are all Gigabit (and no port configurations), I find that case to be unlikely. They were also factory reset so no VLAN tagging or any other overhead on them at the moment.
@DaddyGo At this rate, I'd rather build a custom box which is 10G capable rather than go for any other appliance.
-
Is it running at full speed? Check the CPU frequency shown on the dashboard. You should definietely see faster than 450Mbps in a local iperf test there.
Steve
-
There should not be any differences when testing from the wan or the lan side on iperf3.
I doubt its the unifys for sure..
You really need to establish a testing baseline in order to rule out various subtle (but critical) parameters). Use a third machine (pc) as a reference.
Run iperf from alibox to this machine and verfiy you get 900+mbits., either directly or through your switces. Then put sg2200 into testing and see what happens.
The only way to know its not a faulty cable, a bad lan port , a switch etc, or just plain misconfiguration. -
@sinbox_pfs said in Upgrading Realtek with alternate driver - Is it worth it?:
Unifi US-24
Hi,
I'm glad you found a used unit for your system, this is definitely a good starting point
BTW: this is definitely a configuration issue or a network building issue (physically problem cables or miss connection, etc.)
the Unifi US-24 switches are perfectly suitable not these devices cause the issue, the 10Gig in this environment is unnecessary.
-the SG-2200 knows everything what you want to achieve.
you can begin a step-by-step examination - where does the bottleneck live in your system,
do you have any drawings you can publish?have you already gone through this description?
https://docs.netgate.com/pfsense/en/latest/hardware/tuning-and-troubleshooting-network-cards.htmlit can also help, but treat it carefully:
I350 NIC Tunning loader_conf_local.txt -
@stephenw10 The CPU spikes temporarily to ~80% when running iPerf. Goes back to ~12-20% when idle
-
But what is the reported frequency?
Those devices should have powerd enabled or they can end up running at a much reduced speed.
You should see something like:
CPU Type Intel(R) Atom(TM) CPU C2358 @ 1.74GHz Current: 1411 MHz, Max: 1744 MHz 2 CPUs: 1 package(s) x 2 core(s) AES-NI CPU Crypto: Yes (inactive)Steve
-
@netblues said in Upgrading Realtek with alternate driver - Is it worth it?:
There should not be any differences when testing from the wan or the lan side on iperf3.
I doubt its the unifys for sure..
You really need to establish a testing baseline in order to rule out various subtle (but critical) parameters). Use a third machine (pc) as a reference.
Run iperf from alibox to this machine and verfiy you get 900+mbits., either directly or through your switces. Then put sg2200 into testing and see what happens.
The only way to know its not a faulty cable, a bad lan port , a switch etc, or just plain misconfiguration.I'm a bit challenged at the moment as I have no desktop with ethernet ports except a Mac Mini. I have a Dell XPS, Microsoft Surface and Macbook Pro 13" all of which lack physical Ethernet ports and hence I need to use dongles to test them out. I have had various issues with dongles, so I'm trying to avoid them for the time being until I can get hold of a reliable Thunderbolt > Ethernet Dock/dongle.
For now I re-ran all tests on my a Mac Mini 2018 which does have a 1G ethernet port and this is what I have found. Hope this covers all bases. If there is anything else I can test, please let me know.
LAN iPerf3 Tests:
On Aliexpress Box: [FTTP NTD WAN>Patch Panel>Aliexpress pfSense Box>Mac mini]
With pfSense as Client and Mac Mini as server, the network throughput is 586 Mbps (receiver) and 621 Mbps (sender);
With Mac Mini as Client and pfSense as server, the network throughput is 429 Mbps (receiver) and 436 Mbps (sender)
CPU usage during iPerf3 tests is ~ 22% and Idle is ~2%On SG-2220 [FTTP NTD WAN>US-24>Patch Panel>SG2220>Mac mini]
With pfSense as Client and Mac Mini as server, the network throughput is 865 Mbps (receiver) and 942 Mbps (sender);
However, With Mac Mini as Client and pfSense as server, the network throughput is back to 398 Mbps (receiver) and 398 Mbps (sender)!
CPU usage during iPerf3 tests is ~ 89-92% and Idle is ~12%WAN Speedtest.net tests:
On Aliexpress Box: pfSense > WAN [FTTP NTD WAN>Patch Panel>Aliexpress pfSense Box>Mac mini]
Speedtest ~585 Mbps via SpeedTest native App on Mac Mini to a known server location (~6Kilometers from my property)
Speedtest-CLI on pfSense box itself: Only 306.15 Mbps! (same server location selected) CPU was close to 95%On SG-2220 Appliance: pfs[FTTP NTD WAN>US-24>Patch Panel>SG2220>Mac mini]
Speedtest ~582 Mbps via SpeedTest native App on Mac Mini to a known server location (~6Kilometers from my property)
Speedtest-CLI on pfSense box itself: ~465 Mbps! (same server location selected) CPU was close to 95%Here's is all the things that I think I can rule out. Happy to be corrected:
- As mentioned previously, I have basically setup my network from scratch. There are no Firewall rules apart from the OOTB ones.
- cc: @DaddyGo, It is a new home built less than 2 yrs back and can confirm Cat 6a cables, patch panels etc.
- When I connect the Mac Mini directly to the NTD's ethernet port, I can get close to ~970-980 Mbps, so I think I can rule the Mac Mini as bottleneck
- So, If I have to assume, I'm getting the best possible output off the AliExpress box
So, what baffles me is the LAN results on the SG-2220 with Mac Mini as Client and pfSense as server. Is this where the bottleneck is?
Next steps is to try what @DaddyGo suggests above...
-
@stephenw10 On the SG-2220 it is:
CPU Type Intel(R) Atom(TM) CPU C2338 @ 1.74GHz
Current: 1400 MHz, Max: 2100 MHz
2 CPUs: 1 package(s) x 2 core(s)
AES-NI CPU Crypto: Yes (active)On the AliExpress box it is:
CPU Type Intel(R) Celeron(R) CPU N3160 @ 1.60GHz
Current: 1600 MHz, Max: 1601 MHz
4 CPUs: 1 package(s) x 4 core(s)
AES-NI CPU Crypto: Yes (inactive)powerd was Off on both boxes. You suggest I re-run the tests with this flipped back ok?
-
On the SG-2220 definitely. I imagine it never shows anything other than 1400 MHz? That's what happens on those without powerd enabled. You should see a significant performance improvement with it enabled.
Steve
-
@stephenw10 Tested on the SG-2220 with powerd enabled (followed by a reboot). CPU still spikes ~95% during iPerf (i.e when using SG-2200 as iPerf Server). And with similar throughput may I add...
-
Does the dashboard now show it running at full speed?
Any test where the 2220 is actually running iperf is not a good one. pfSense is not optimised as TCP terminator.
Really you need to test through it, with an iperf3 server on one interface and a client on the other. Running iperf3 on the 2220 will itself use a lot of CPU leaving far less for actually moving traffic.
Steve
