Official Realtek Driver Binary 1.95 For 2.4.4 Release
-
They would be in the system log if you are hitting that issue.
Steve
-
Also, at least in my experience using the driver built in to FreeBSD, the watchdog timeout is pretty easy to provoke by running iperf tests against the LAN interface. I see that the Odroid H2 has dual RTL8111Gs whereas I, and I believe most others using the official Realtek driver, have been running Zotac machines that I believe have RTL8111Es. Both of those use the same driver, but I think it's fair to say that the RTL8111G is less "battle tested" with this driver. Have you also confirmed that the driver is indeed being loaded and used by looking for it in the output of ldstat?
-
I have made FreeBSD 12 package.
https://drive.google.com/file/d/1Ch4Z_w7gpbrpavQ4KhPUXUzYhRyzqnye/view?usp=sharing
I dont consider it my work - I have just packed it in one archive. It is stable for me on 12.0 p3 more than
"sed -i -e 's/TAILQ_FOREACH/CK_STAILQ_FOREACH/g' if_re.c" -
Will pfSense 2.5.0 / FreeBSD 12.0-RELEASE-p3 improve the built-in Realtek drivers situation at all?
-
I would guess no. Not much activity in the FreeBSD repo:
https://github.com/pfsense/FreeBSD-src/commits/RELENG_2_5/sys/dev/re -
I've installed precomplied version of this driver for Nettop Gigabyte GB-BACE-3000 - working good with PFSence 2.4.4-RELEASE-p1. Uptime 31 days.
Thanks for the driver @TheNarc! -
I am currently using a Gigabyte GA-C1007UN-D
It has two realtek NICS either RTL8111E or RTL811F (online reviews conflict) I have never noticed this error, but would use this driver if it would increase stability. using the current 2.4.5 dev build.Any advice on specific iperf test that is known to kick off this problem? traffic to the rtr, from the router, do I need tons of simultaneous streams?
I would like to see it fail before I spent the time.
-
@mervincm In my experience, I didn't need to set any exotic iperf options. Just running a few iperf tests between a host on the LAN and the LAN port of the pfSense machine was enough to reproduce the error.
-
@TheNarc I guess I am fine then as after I stood up iperf as a server on pfSense
I hammered it quite hard from my desktop with 5, 10, 15, 20, 25, 30, 40, 50 parallel threads under iperf2 and never managed to cause the pfSense LAN nic to freeze.
I then shut down the iperf service and then started up iperf3 from the command line iperf3 -s
again, I hammered it quite hard from my desktop with 5, 10, 15, 20, 25, 50, and then even 100 parallel threads a few times under iperf3 and again, never managed to cause the pfSense LAN nic to freeze.The iperf performance wasn't great, say about 700-900 Mbps, but it was stable.
edit I did get 2 occurrences of the alert. I didn't notice the impact while testing, perhaps it came back up quickly?
Mar 27 17:09:23 kernel re1: watchdog timeout
Mar 27 16:38:39 kernel re1: watchdog timeoutIs a few errors like this symptomatic of this bug, or would you see far more of them?
I do see this occasionally in my logs, 35 total of them over a couple of months. I am not sure if that is expected or a problem to fix.
-
Well, I decided to try it out. I added the driver as detailed above, and confirmed it was loaded, and now I beat on it 5 times as hard and I have not been able to get a single more watchdog error. thanks for helping me find and fix a problem I didn't even know I had!
-
@TheNarc said in Official Realtek Driver Binary 1.95 For 2.4.4 Release:
CI323
What I experienced, using the driver built into pfsense on Cl323, is that there will be watchdog timeouts frequently; but eventually the LAN interface will quite working completely until someone reboots the box. After installing the realtek driver 1.95 as described in this thread, those issues are no more and the box is stable.
If you are deploying pfsense on CI323, I'd strongly reccommend this driver and that stress testing the network is part of your checklist.
-
Has anyone experienced a downgrade in performance to around 100mbps? It used to operate at my connection speed 175mbps, but after I had to shutdown and replace a power bar a few days ago, it hasn't been the same.
The dashboard reports the link is 1000mbps but that's not the actual speed when downloading a file.
Thanks for your time.
-
@krackpot I have two systems running with this driver one on a 200mbps link one on a 300mbps, neither have trouble hitting those speeds.
-
That sounds more like something is actually linking at 100Mb. That NIC should pass far more than that with either driver.
Steve
-
Have 2.4.4-RELEASE-p3 the latest driver for RTL8111G or must I manualy update the driver to latest version?
I have problem WAN port stop working, disable the port and enable and it start working again... its happens with low load 10mbit maybe... within 1 hour or so...
-
you need to manually update https://drive.google.com/open?id=1lBg8AIRiRfGBGeXe9kgKtAAma0VjiYfC
copy to /boot/kernel , rename to if_re.ko
kldload if_re.ko
or build it yourselfto load it at boot
edit /boot/loader.conf.local
and paste
if_re_load=“YES"personally i've lost patience with realtek and at the end i bought an intel card
-
@kiokoman said in Official Realtek Driver Binary 1.95 For 2.4.4 Release:
you need to manually update https://drive.google.com/open?id=1lBg8AIRiRfGBGeXe9kgKtAAma0VjiYfC
copy to /boot/kernel , rename to if_re.ko
kldload if_re.ko
or build it yourselfto load it at boot
edit /boot/loader.conf.local
and paste
if_re_load=“YES"personally i've lost patience with realtek and at the end i bought an intel card
/root: kldstat
Id Refs Address Size Name
1 5 0xffffffff80200000 2ddcbe8 kernel
2 1 0xffffffff82fde000 7d290 if_re.ko
3 1 0xffffffff839fa000 10a0 cpuctl.koIs the drive active now ?
-
yes
-
@kiokoman said in Official Realtek Driver Binary 1.95 For 2.4.4 Release:
yes
These should be it ?
Now can I go to bad, and see if internett is still working tomorrow morning :)
Thank you very much !
-
The problem is still present...
Strange it is only a problem with WAN (re1) and not LAN (re0)... -
[2.4.4-RELEASE][root@firewall.test.net]/root: grep re1 /var/run/dmesg.boot
re1: <Realtek PCIe GBE Family Controller> port 0xd000-0xd0ff mem 0xa1104000-0xa1104fff,0xa1100000-0xa1103fff at device 0.0 on pci3
re1: Using Memory Mapping!
re1: Using 1 MSI-X message
re1: ASPM disabled
re1: version:1.95.00
re1: Ethernet address: 00:1e:06:45:02:83
re1: Ethernet address: 00:1e:06:45:02:83[2.4.4-RELEASE][root@firewall.test.net]/root: route -n
route: usage: route [-46dnqtv] command [[modifiers] args][2.4.4-RELEASE][root@firewall.test.net]/root: arp -an | grep re1
? (92.221.80.253) at 00:1e:06:45:02:83 on re1 permanent [ethernet]
? (92.221.80.1) at 00:02:00:01:00:01 on re1 expires in 97 seconds [ethernet][2.4.4-RELEASE][root@firewall.test.net]/root: tcpdump -c 20 -n -i re1
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on re1, link-type EN10MB (Ethernet), capture size 262144 bytes
03:49:11.244621 IP 92.221.80.253.49228 > XX.XX.XX.XX.XXX: 26302+ XXXXXXX (91)
03:49:11.244626 IP 92.221.80.253.49228 > XX.XX.XX.XX.XXX: 26302+ XXXXXXX. (91)
03:49:11.246135 IP 92.221.80.253.25525 > XX.XX.XX.XX.XXX: 28630+ XXXXXXX. (35)
03:49:11.246139 IP 92.221.80.253.25525 > XX.XX.XX.XX.XXX: 28630+ XXXXXXX. (35)
03:49:11.262982 IP 92.221.80.253.25525 > XX.XX.XX.XX.XXX: 42079+ XXXXXXX. (57)
03:49:11.262985 IP 92.221.80.253.25525 > XX.XX.XX.XX.XXX: 42079+ XXXXXXX. (57)
03:49:11.263013 IP 92.221.80.253.25525 > XX.XX.XX.XX.XXX: 23104+ XXXXXXX. (53)
03:49:11.263016 IP 92.221.80.253.25525 > XX.XX.XX.XX.XXX: 23104+ XXXXXXX. (53)
03:49:11.413788 IP 92.221.80.253.25525 > 109.247.114.4.53: 46408+ XXXXXXX. (41)
03:49:11.413793 IP 92.221.80.253.25525 > 92.220.228.70.53: 46408+ XXXXXXX. (41)
03:49:11.413823 IP 92.221.80.253.25525 > 109.247.114.4.53: 30408+ XXXXXXX. (41)
03:49:11.413826 IP 92.221.80.253.25525 > 92.220.228.70.53: 30408+ XXXXXXX. (41)
03:49:11.414619 IP 92.221.80.253.25525 > XX.XX.XX.XX.XXX: 9197+ XXXXXXX. (31)
03:49:11.414622 IP 92.221.80.253.25525 > XX.XX.XX.XX.XXX: 9197+ XXXXXXX. (31)
03:49:11.414695 IP 92.221.80.253.36009 > XX.XX.XX.XX.XXX: 247+ XXXXXXX. (31)
03:49:11.414699 IP 92.221.80.253.36009 > XX.XX.XX.XX.XXX: 247+ XXXXXXX. (31)
03:49:11.490012 IP 92.221.80.253.49228 > XX.XX.XX.XX.XXX: 26302+ XXXXXXX. (91)
03:49:11.490015 IP 92.221.80.253.49228 > XX.XX.XX.XX.XXX: 26302+ XXXXXXX. (91)
03:49:11.491250 IP 92.221.80.253.25525 > XX.XX.XX.XX.XXX: 28630+ XXXXXXX. (35)
03:49:11.491253 IP 92.221.80.253.25525 > XX.XX.XX.XX.XXX: 28630+ XXXXXXX. (35)
20 packets captured
20 packets received by filter
0 packets dropped by kernel -
@kiokoman are you sure the driver has been loaded ?
Just that I facing the same problem after also.... -
@gordon said in Official Realtek Driver Binary 1.95 For 2.4.4 Release:
re1: version:1.95.00
It shows it's loaded there. The FreeBSD in kernel driver doesn't show the version as far as I know. I only have 2.5 dev box up right now but:
[2.5.0-DEVELOPMENT][root@apu.stevew.lan]/root: grep re1 /var/run/dmesg.boot re1: <RealTek 8168/8111 B/C/CP/D/DP/E/F/G PCIe Gigabit Ethernet> port 0x2000-0x20ff mem 0xf7c00000-0xf7c00fff,0xf7b00000-0xf7b03fff irq 17 at device 0.0 on pci2 re1: Using 1 MSI-X message re1: ASPM disabled re1: Chip rev. 0x2c000000 re1: MAC rev. 0x00200000 miibus1: <MII bus> on re1 re1: Using defaults for TSO: 65518/35/2048 re1: Ethernet address: 00:0d:b9:89:3f:11 re1: netmap queues/slots: TX 1/256, RX 1/256
Try commenting out the line in loader.conf.local and rebooting.
Steve
-
@gordon said in Official Realtek Driver Binary 1.95 For 2.4.4 Release:
[2.4.4-RELEASE][root@firewall.test.net]/root: grep re1 /var/run/dmesg.boot
re1: <Realtek PCIe GBE Family Controller> port 0xd000-0xd0ff mem 0xa1104000-0xa1104fff,0xa1100000-0xa1103fff at device 0.0 on pci3
re1: Using Memory Mapping!
re1: Using 1 MSI-X message
re1: ASPM disabled
> re1: version:1.95.00
re1: Ethernet address: 00:1e:06:45:02:83
re1: Ethernet address: 00:1e:06:45:02:83yes it is like stephenw10 say,
version 1.95.00 is the official driver, the one shipped with freebsd does not output the version when it load -
@kiokoman said in Official Realtek Driver Binary 1.95 For 2.4.4 Release:
@gordon said in Official Realtek Driver Binary 1.95 For 2.4.4 Release:
[2.4.4-RELEASE][root@firewall.test.net]/root: grep re1 /var/run/dmesg.boot
re1: <Realtek PCIe GBE Family Controller> port 0xd000-0xd0ff mem 0xa1104000-0xa1104fff,0xa1100000-0xa1103fff at device 0.0 on pci3
re1: Using Memory Mapping!
re1: Using 1 MSI-X message
re1: ASPM disabled
> re1: version:1.95.00
re1: Ethernet address: 00:1e:06:45:02:83
re1: Ethernet address: 00:1e:06:45:02:83yes it is like stephenw10 say,
version 1.95.00 is the official driver, the one shipped with freebsd does not output the version when it loadThank you,
I reinstalled pfsense today and are using WAN on re0 instead of re1 and LAN on re1, so fare so fare, new record in uptime... cross my fingers....
But I cannot understand these shall make any different ?
-
The driver provided by Realtek can commonly help if you're seeing watchdog timeout errors. But it sounds like that's not what you're seeing so it may not make any difference here. You have nothing to lose by trying it though.
Steve
-
@stephenw10
Problem is still present, card stop working and report DOWN in status underRemember both card is identical network card... strange it’s only WAN card who goes down.
Have now tried disabling «Disable hardware checksum offload»
Is it possible to make the card recycle DISABLE and then ENABLE when these happens, way around the problem until I find a fix?
-
«Disable hardware checksum
Was not fixing the problem....Does anybody know how I can auto disable and enable the card when it goes down?
-
try with
/etc/rc.d/netif restart re0 -
@kiokoman said in Official Realtek Driver Binary 1.95 For 2.4.4 Release:
try with
/etc/rc.d/netif restart re0Thank you, I need some script to detect when it shall happens also...
I found some things to try another place on these forum, so I have added these lines to the boot config:
net.inet.tcp.tso=0
hw.pci.enable_msix=0
hw.pci.enable_msi=0
hw.re.tso_enable=0to the boot loader config, and for now I haven`t seen the issue again, need more hours... will report back later.
-
Yes, definitely disable all hardware offloading features. Check what is still enabled using
ifconfig re1
.Did you try re-assigning the NICs? Does the fault follow re1 or remain on WAN?
It's likely you are downloading far more traffic that uploading. Whichever NIC is WAN is doing the receiving of that traffic and a number of those hardware offloading features are receive only.Steve
-
@stephenw10 said in Official Realtek Driver Binary 1.95 For 2.4.4 Release:
Yes, definitely disable all hardware offloading features. Check what is still enabled using
ifconfig re1
.Did you try re-assigning the NICs? Does the fault follow re1 or remain on WAN?
It's likely you are downloading far more traffic that uploading. Whichever NIC is WAN is doing the receiving of that traffic and a number of those hardware offloading features are receive only.Steve
I reinstalled the whole pfsense yesterday and then I swapped so re0 is WAN and re1 is LAN.
The problem have always followed the card I have assigned to be WAN.The problem has not accour after I disabled al the things mention above from the HW i the boot loader. But there is a strange thing the uptime on the box is never 1 hour and 47 minutes, but I have not restarted the box today..... I think there are something wrong with the clock... its not going correct... it`s deviate more and more from the "real time"....
Maybe there is a releation here ? lead time on the DHCP maybe got problem...
I will look into getting the clock correct first.
-
Make sure NTP is configured and working. All APUs are old now, the CMOS battery may be dead.
Edit: Is this an APU? Not sure where I pulled that from..
Steve
-
This post is deleted! -
I just subscribed to thank you for your work,
I was going mad trying to understand the issue and finally this morning I found a post that directed me here.The issues I had on my ci323 nano seems all solved now.
thanks again
-
@TheNarc, just wanted to thank you and post an interesting update.
Apparently your fix also solves an issue I had been experiencing with passing the realtek card through to a VM running Pfsense on a Centos 8 host.
Running on J4105-ITX motherboard.
The error occured with pretty much any realtek card I tried including the onboard one.
Until now, it would give an nasty DMAR error and the card would always stop working after pfsense VM reboot.
Sometimes, it would show a watchdog timeout, sometimes would just quetly stop responding.
With this driver, the card is stable and there is no DMAR error in Host's dmesg. -
I've had three Zotac CI32x boxes deployed, running pfsense for a few years now, and I had not run into the dreaded watchdog timeout error until after upgrading to pfSense 2.4.4. Now the lan interface of two out of three boxes goes silent about once a month. Fortunately, since it it is the lan interface, I just reboot them remotely.
Thanks for doing the work building this driver. This saved me quite a bit of time.
Question:
With this new driver, can/should I re-enable hardware offloading under Advanced-> Networking?I've also got the following set in loader.conf.local. These were suggested as workarounds in another thread. Is it safe to remove these?
hw.re.msi_disable=1
hw.pci.enable_msix=0
hw.pci.enable_msi=0 -
@knnniggett I'm glad to hear the driver has been useful. I'm sorry to hear about the LAN interface flakiness; that's not something I've seen (yet) in on the two CI323s I run. With regard to your questions:
I would not recommend re-enabling hardware offloading. I believe that Realtek claims to support it, but from what I understand it can be prone to bugginess and you're unlikely to see any tangible benefit from it. Also, it's the sort of thing that (I think) you wouldn't necessarily know right away if it works. So you might try it and it would seem fine, but some time down the line you'd have a problem and, if you're anything like me, probably forget that you'd changed that setting a while back and not immediately realize that it's related.
As for the loader.conf.local entries, at least based on my experience it should be fine to get rid of them. I don't have them set on either of my boxes right now and haven't had any issues.
-
@knnniggett said in Official Realtek Driver Binary 1.95 For 2.4.4 Release:
I've had three Zotac CI32x boxes deployed, running pfsense for a few years now, and I had not run into the dreaded watchdog timeout error until after upgrading to pfSense 2.4.4. Now the lan interface of two out of three boxes goes silent about once a month. Fortunately, since it it is the lan interface, I just reboot them remotely.
Thanks for doing the work building this driver. This saved me quite a bit of time.
Question:
With this new driver, can/should I re-enable hardware offloading under Advanced-> Networking?I've also got the following set in loader.conf.local. These were suggested as workarounds in another thread. Is it safe to remove these?
hw.re.msi_disable=1
hw.pci.enable_msix=0
hw.pci.enable_msi=0I do not have these anymore either with the new driver.
Should be good to remove.
-
I made myself a new router after my previous one failed(power supply fried).
I got myself a GIGABYTE GA-N3160N-D3V motherboard with an embedded CPU and DUAL NIC's perfect for making a nice small compact router with pfsense. Everything was hunky-dory for a few months with no issues until one day I started encountering this "watchdog timeout" issue.
I thought that the CPU had overheated and bent the solder like how the Xbox 360 did with the RROD. The CPU was only at 65 degrees Celcius...
I installed fans in the case and ensured that the temp never grew past 40 degrees Celcius. The issue still persisted and I thought I would now need to RMA the board.
As I cannot go without internet for longer periods of time due to working from home I needed to get my original pfsense router up and running before sending this board in.
Whilst in the process of doing so I was doing some uploading of work to Jira when one gif file would repeatedly kill the router. I was stunned, how on earth could a 2mb file kill the router?!?!?!
I thought I would do one last search online to see if I could find anything software related that might be the issue and I stumbled upon this thread.
I can confirm that applying the Realtek driver has now solved my issue!
Thank you!