Official Realtek Driver Binary 1.95 For 2.4.4 Release

Griffo

Nevermind, I had so many other issues I've rolled back to 2.4.5 so it's probably moot.

stephenw10

Hmm, looks like you have everything right. You could try this additional line in loader.conf.local, though I did not need it here, the loader looks there anyway:

if_re_name="/boot/modules/if_re.ko"

Steve

NetworkingMicrobe

Hi all,

New to the pfSense world but wanted to follow up on this discussion. I've successfully compiled & installed the latest if_re.ko drivers from Realtek, v1.96.04, for my RTL8111/8168/8411 Gigabit ethernet controller (re0 interface). They are loaded upon boot without issue, as per kldstat and looking at logs in the web UI (Status > System Logs > System > General).
This is an integrated controller on my motherboard (old P7H55-M PRO from Asus) which I'm testing for use as WAN in a homelab setup while I wait for my Intel NICs for LAN. The link is also recognized as 1Gbps as per ifconfig (1000Base-T <full-duplex>).

I am facing performance issues though when benchmarking with iperf3. Using Cat5e or Cat6 cabling all around, with short runs from the firewall to my client, and have tried new cables as well.

Using iperf3, when re0 is the SERVER, I get about 450Mbps. When re0 is the CLIENT, I get a big range, but about 450-600Mbps on average, sometimes peaking at 800Mbps for an entire run if I'm lucky. Retry (Retr column) is almost always all 0 or thereabouts.

The same client can easily get 900+ Mbps with another iperf3 server on the network.

I looked into CPU and memory usage during the tests, plenty of free memory on the system (8GB RAM on system), swap not in use at all. As for the CPU (Intel Core i3-530), usage looks something like this, with lots of idle room...

CPU:  0.7% user,  0.0% nice,  8.0% system, 16.7% interrupt, 74.6% idle

I was wondering if anyone faced similar performance issues, with lack of consistency seeming to occur as well (although never a steady 900Mbps+ or even 800Mbps+)? I saw that @nitewolfgtr and @stephenw10 spoke a bit about performance issues on a 2.5G card, but never came to any conclusions.

Thanks!

Griffo

@networkingmicrobe I had the same issue with my Qotom with the built in realteks. It seemed to have some magic stop around 450mbits, sometimes though it would go up to 600mbits. There appeared to be plenty of CPU left but it just maxed out. I replaced it with a new box with Intel NIC's and hit wire-speed gigabit easily.

stephenw10

@networkingmicrobe said in Official Realtek Driver Binary 1.95 For 2.4.4 Release:

i3-530

That's a 2 core 4 thread CPU so 74% idle could be 1 virtual core at 100%. Potentially the core running iperf.
Try running top -aSH while you test.

Better to test through the firewall rather than to or from it directly.

Steve

Napsterbater

@networkingmicrobe If you want to benchmark pfSense on a particular device and setup. run iperf through pfSense not on it.

It has been said MANY times on the fourms.

Using iperf on the pfsense system itself is not a valid gauge at routing performance.

Cybermaze

@networkingmicrobe I'm not entirely sure, but I suspect that the driver is limited in that one TCP/UDP stream can only utilize one core on the CPU, simply put, 450Mbps is what a single core on your CPU (Intel Core i3-530) can handle if you run iperf with one client thread (should be default).

You can test that by running iperf on the client side with more threads using -P (that's a capital P), something like:

iperf -c 192.168.0.1 -P 4

Will run the client with 4 threads, each opening their own TCP stream (adjust IP address to whatever you are using).

Cybermaze

@napsterbater correct, it is not a valid gauge at routing performance, however I suspect that @networkingmicrobe is attempting to find out wether or not his NIC is capable of sending data close to the 1Gbit linespeed, not the actual routing throughput.

NetworkingMicrobe

@griffo - Thanks for the tip, looking forward to my Intel NICs then! Interesting that this seems to keep happening with integrated (on-motherboard) Realtek NICs, though.

@stephenw10 - You bring up a good point, thanks. That result I pasted above was actually from top -aSH. In that case, I'd expect the same results on an Intel NIC (over PCIe), since it may be an issue with single-core clock speed. It looks like cpu0 (first core) is being used the most during a single-stream iperf3 test, with WCPU% of [idle{idle: cpu0}] dropping down during the test to about 50% and then back to 99% after.
I tried increasing the number of parallel streams to 2, and then 4, like @Cybermaze suggested, but saw same results in terms of throughput (~400Mbps) and core utilization (mostly cpu0, perhaps some of core1 too).

@Napsterbater - I realize benchmarking pfSense is best done by going through the firewall, but what I wanted to simulate here was the maximum theoretical line speed by testing to the WAN port directly. I realize it's not at all indicative of performance through the firewall on other LAN devices, but is a 'sanity check' I'd like to try first, if that makes sense, to see if I can ever get the theoretical ~1Gbps line speeds expected to begin with. Basically what @Cybermaze suggested in their comment above.

Thanks all for your quick and informative replies, perhaps it's just a limitation of my onboard NIC for some reason.

stephenw10

iperf3 is in fact single threaded and will only ever use one CPU core. The -P switch can make it run multiple client streams, which can take advantage of multiqueue NICs for interrupt load, but it will still only use one CPU core on the client and server machine for iperf itself. Which is another good reason to test through the firewall.

If you need to test like that try running multiple iperf servers or clients on different ports simultaneously.

https://fasterdata.es.net/performance-testing/network-troubleshooting-tools/iperf/multi-stream-iperf3/

Steve

NetworkingMicrobe

@Cybermaze - Small update, using multiple client streams into a pfSense iperf3 server yields similar results as a single stream of ~400-450Mbps.
However, I noticed when using multiple client streams when pfSense is the client and another device is the server, I can get steady 800Mbps throughput, much closer to theoretical 1000Mbps max line speed. Interesting...

I'll look into running multiple iperf servers simultaneously as well, thanks @stephenw10.

NetworkingMicrobe

Finally got around to adding some gigabit Intel NICs to the build, and sure enough I can easily obtain 960+ Mbps both to the firewall (to see WAN theoretical max speed) and through the firewall. So definitely something funky going on with the onboard Realtek NIC, like @griffo mentioned, despite using the latest drivers v1.96.04 from Realtek. Not crucial to fix I guess, but still interesting to note for others who may have similar issues in the future.

mikekoke

@rico said in Official Realtek Driver Binary 1.95 For 2.4.4 Release:

Here you go: rtl_bsd12.1_drv_v196.04.zip

-Rico

Hi, I would like to know if this driver also works on the 2.5.0 RC that is about to come out.
Thank you.

stephenw10

As jimp said in the other thread in 2.5 the FreeBSD package for this is in our repo. So you can simply do this at the command line:

pkg install realtek-re-kmod
then
echo 'if_re_load="YES"' >> /boot/loader.conf.local

Then reboot and check the boot logs for output from the new driver loading like:

re0: <Realtek PCIe GbE Family Controller> port 0x1000-0x10ff mem 0xf7a00000-0xf7a00fff,0xf7900000-0xf7903fff irq 16 at device 0.0 on pci1
re0: Using Memory Mapping!
re0: Using 1 MSI-X message
re0: ASPM disabled
re0: version:1.96.04
re0: Ethernet address: 00:0d:b9:37:30:10

Steve

mikekoke

@stephenw10
Ok thanks.

mikekoke

@stephenw10 said in Official Realtek Driver Binary 1.95 For 2.4.4 Release:

As jimp said in the other thread in 2.5 the FreeBSD package for this is in our repo. So you can simply do this at the command line:

pkg install realtek-re-kmod
then
echo 'if_re_load="YES"' >> /boot/loader.conf.local

Then reboot and check the boot logs for output from the new driver loading like:
re0: <Realtek PCIe GbE Family Controller> port 0x1000-0x10ff mem 0xf7a00000-0xf7a00fff,0xf7900000-0xf7903fff irq 16 at device 0.0 on pci1
re0: Using Memory Mapping!
re0: Using 1 MSI-X message
re0: ASPM disabled
re0: version:1.96.04
re0: Ethernet address: 00:0d:b9:37:30:10
Steve

I was forgetting to ask you something, in your opinion it is better to first remove the old driver for realtek that I had installed for 2.4.5, then update to 2.5.0 and finally install the new driver with the procedure you wrote above or I can do upgrade to 2.5.0 and then overwrite the old driver with the new one? Thank you.

stephenw10

I would removed any 2.4.X binaries you have uploaded first. They won't load in 2.5 anyway.

Steve

Allex

@mikekoke said in Official Realtek Driver Binary 1.95 For 2.4.4 Release:

@rico said in Official Realtek Driver Binary 1.95 For 2.4.4 Release:

Here you go: rtl_bsd12.1_drv_v196.04.zip

-Rico

Hi, I would like to know if this driver also works on the 2.5.0 RC that is about to come out.
Thank you.

Thanks. It worked easily for me.

PfSense 2.5.0-RELEASE (amd64)
built on Tue Feb 16 08:56:29 EST 2021
FreeBSD 12.2-STABLE

jgnguyen123

@networkingmicrobe I have the same problem with the built-in RTL8168. In Windows driver, there's an option "Gigabit lite" that is to set to ON by default which reduces its speed to half. I suspect that might be be the default settting on the NIC. Not sure if there's an option to turn it OFF in pfSense.

m4nf47

@mikekoke said in Official Realtek Driver Binary 1.95 For 2.4.4 Release:

@stephenw10 said in Official Realtek Driver Binary 1.95 For 2.4.4 Release:
As jimp said in the other thread in 2.5 the FreeBSD package for this is in our repo. So you can simply do this at the command line:

pkg install realtek-re-kmod
then
echo 'if_re_load="YES"' >> /boot/loader.conf.local

Then reboot and check the boot logs for output from the new driver loading like:
re0: <Realtek PCIe GbE Family Controller> port 0x1000-0x10ff mem 0xf7a00000-0xf7a00fff,0xf7900000-0xf7903fff irq 16 at device 0.0 on pci1
re0: Using Memory Mapping!
re0: Using 1 MSI-X message
re0: ASPM disabled
re0: version:1.96.04
re0: Ethernet address: 00:0d:b9:37:30:10
Steve
I was forgetting to ask you something, in your opinion it is better to first remove the old driver for realtek that I had installed for 2.4.5, then update to 2.5.0 and finally install the new driver with the procedure you wrote above or I can do upgrade to 2.5.0 and then overwrite the old driver with the new one? Thank you.

Hi thread, long time no see! So I recently upgraded from 2.4.5 to 2.5.1 and started getting the dreaded watchdog timeout errors again after a couple of days, would completely hang my connection and force me offline. I noticed that the Realtek driver wasn't loading as before and can confirm that the above simple (pkg install) fix seemed to work for me. The key info to confirm that the driver is loaded is to run the kldstat command and also check the boot logs for the version line as shown above. Many thanks for sharing, fingers crossed this post might help someone else running crappy Realtek hardware on pfSense.