Zotac ZBOX CI323 nano

trottoluccia

Sorry guys but I'm new to freeBSD (actually I'm just a user with no experience in compiling stuff)… is there any guide to put this driver in my pfsense installation?

I've got the Zotac CI323 and I can only go up to 100tx on my LAN despite everything else is gigabit... hope the diver can help..

thanks, Max

TheNarc

Apologies to bgbird03; I don't check the forums as often as I should. But the WinSCP solution referred to by him/her is probably the path of least resistance for getting the if_re.ko file to the /boot/kernel/ location on your pfSense machine from a Windows machine. Here's a decent write-up with a link to download WinSCP that should still work even though it was written in 2010: https://dave.harris.uno/using-winscp-to-connect-to-pfsense/. Once you have the file up, you may need to set its permissions and/or ownership; in any case, it's a good idea. The easiest way to do this from a Windows machine is probably using putty. Here's a decent guide on that: https://turbofuture.com/internet/How-to-Access-pfSense-Remotely-Using-SSH. Note that you only need to follow it as far as "The Console Menu." Once you have a command prompt after entering 8 at the options menu, execute the following two commands:
chown root:wheel /boot/kernel/if_re.ko
chmod 0555 /boot/kernel/if_re.ko
After this, if you execute:
ls -al /boot/kernel/if_re.ko
You should see output like this:
-r-xr-xr-x 1 root wheel 522968 Oct 12 19:51 /boot/kernel/if_re.ko

After that, you just need to tell the system to load and use this module. To do that, go to the normal pfSense web interface and navigate to Diagnostics > Edit File. Type in /boot/loader.conf.local and click Load. At the end of anything already there, add the line:
if_re_load="YES"

I also have these lines, which I found some time ago in this thread: https://forum.pfsense.org/index.php?topic=101587.0. Full disclaimer, I don't know whether they're really necessary or helpful, but I've had them forever on my CI323 and they certainly don't cause any trouble . . .
hw.re.msi_disable=1
hw.pci.enable_msix=0
hw.pci.enable_msi=0

So that ought to do it. Obviously you'll need to reboot your pfSense machine for the changed to take effect. If you want to verify the new module is being used, SSH in again using putty and run:
kldstat

You'll probably get a few lines of output from this, but one of them should list if_re.ko.

bgbird03

@TheNarc:

Apologies to bgbird03; I don't check the forums as often as I should. But the WinSCP solution referred to by him/her is probably the path of least resistance for getting the if_re.ko file to the /boot/kernel/ location on your pfSense machine from a Windows machine. Here's a decent write-up with a link to download WinSCP that should still work even though it was written in 2010: https://dave.harris.uno/using-winscp-to-connect-to-pfsense/. Once you have the file up, you may need to set its permissions and/or ownership; in any case, it's a good idea. The easiest way to do this from a Windows machine is probably using putty. Here's a decent guide on that: https://turbofuture.com/internet/How-to-Access-pfSense-Remotely-Using-SSH. Note that you only need to follow it as far as "The Console Menu." Once you have a command prompt after entering 8 at the options menu, execute the following two commands:
chown root:wheel /boot/kernel/if_re.ko
chmod 0555 /boot/kernel/if_re.ko
After this, if you execute:
ls -al /boot/kernel/if_re.ko
You should see output like this:
-r-xr-xr-x 1 root wheel 522968 Oct 12 19:51 /boot/kernel/if_re.ko

After that, you just need to tell the system to load and use this module. To do that, go to the normal pfSense web interface and navigate to Diagnostics > Edit File. Type in /boot/loader.conf.local and click Load. At the end of anything already there, add the line:
if_re_load="YES"

I also have these lines, which I found some time ago in this thread: https://forum.pfsense.org/index.php?topic=101587.0. Full disclaimer, I don't know whether they're really necessary or helpful, but I've had them forever on my CI323 and they certainly don't cause any trouble . . .
hw.re.msi_disable=1
hw.pci.enable_msix=0
hw.pci.enable_msi=0

So that ought to do it. Obviously you'll need to reboot your pfSense machine for the changed to take effect. If you want to verify the new module is being used, SSH in again using putty and run:
kldstat

You'll probably get a few lines of output from this, but one of them should list if_re.ko.

TheNarc: you're amazing! You've already done a ton by compiling the driver and posting it, so yeah, I don't blame you if you don't live on the forum 24/7. Your debt to society has been paid in my opinion…

Putty'd in no problem and changed the permissions like you said (my time stamp is slightly different, but whatever -- good enough for me). Edit file was super easy also. I had seen most of those lines before -- seems like every Zotac box of this type needs them? -- and added the hw.re.msi_disable=1 because I didn't have that one. And...of course it caused my boot to hang at some random thing...atrtc0, right above some "event timer". I rebooted, pushed option 3, set hw.re.msi_disable=0, and everything booted fine. So henceforth, I deleted that from mine. No idea why my box didn't like it.

Rebooted and verified with kldstat that if_re.ko is indeed running. Also did a dmesg and after hunting around for it, found my re0 & re1 version was 1.94.01, so I'm assuming things went as planned.

Will this driver work with future version of pfSense? Like when they move to FreeBSD 12.0 or whatever (in the future?) Just kind of curious. I think the solution is to probably move away from a Realtek NIC, but for the moment I wanted a cheap project to play around with and learn simple firewall/unix stuff, and this box really fit the (low power) bill.

It seems to have fixed my watchdog timeout error. Now if only I could get my PS4 to stop having massive lag! Again -- thanks for your help on this very specific issue.

TheNarc

Nice! Thanks for the kind words and glad this worked for you. Not sure about that hw.re.msi_disable option; honestly I haven't really looked into exactly what it does. But of course the important thing is getting rid of the timeout errors. As far as forward-compatibility, I would sort of expect we'll need to (or at a minimum should) recompile again for the next major FreeBSD release (e.g. 12.0) but that what we have now should work for any minor releases of FreeBSD 11. That kind of compatibility question really depends on how OS releases are managed, and I'm not a BSD guru. But I think it's a reasonable expectation, in roughly the same way that you would expect a driver for Windows 7 to work regardless of what service pack you're on, but not necessarily that the same driver would work for Windows 10. Although I'll be glad to recompile for FreeBSD 12.0 whenever pfSense transitions to it.

bgbird03

Interesting followup to the hw.re.msi_disable=1 that you had in the loader.conf.local. If you remember, I put that in, it "broke" the boot, so took it out and it worked fine. Well…when I upgraded to 2.4.2 today, it for some reason got stuck in the exact same spot (even though hw.re.msi_disable=1) wasn't in the loader.conf.local. So...I manually set it to hw.re.msi_disable=0 before the boot process using option 3, and that fixed it. I've now added hw.re.msi_disable=0 to the loader.conf.local permanently; just interesting that it is the exact opposite setting of what you have.

TheNarc

That is strange, especially since the default value is supposed to be 0 based on on information here: https://www.freebsd.org/cgi/man.cgi?query=re&sektion=4&manpath=FreeBSD+11.1-RELEASE. Are you also running a CI323 or the newer CI327? Maybe they use slightly different Realtek chipsets. Although since they both use the same Realtek driver you'd expect that driver setting to behave the same on both. Anyway, whatever works I suppose, though it's always nice to understand why things work when possible :)

bgbird03

I've got a 3127NANO-U, so maybe that is the difference. Then again, I have no idea what I'm doing and my pfSense experience so far has been like fighting (against myself) in the contested cyber domain. Sure, I can sort of do some things. But then other things break (like my BBC News App), and it takes me hours to figure out how to fix the firewall rules and why in the heck these things are being blocked (don't get me started on my VPN & certificate issues). So yeah…

My experience is what I think fighting China must be like in cyberspace. Except I am my own enemy.

rek5

Update: new module does not with with 2.4.2 I was able to load a fresh version of 2.4.1 and the module loaded correctly. I am still having issues with the WAN throughput for the device. Internet is 120/10, I can only get 50/10 after the pfsense box. iperf shows 500mbit speeds.

bgbird03

I was able to get it working with 2.4.2, unless you're talking about something else. And I only have 50/50 internet, but I can usually pull that through my Zotac. My PS4 doesn't work through it though, and my VPN is slower than molasses. But I still have 2.4.2 working…

Warhol

Thanks a lot to TheNarc for posting the compiled module!

I tried it with pfSense 2.4.2 on a ZBOX CI327:

The module does load! :)
However, I had to set loader options opposite to what TheNarc suggested:
- hw.re.msi_disable=0
- hw.pci.enable_msi=1
- hw.pci.enable_msix=1

With the options set otherwise, I experienced weird behaviour - either the boot hangs completely, or the system comes up (VGA console visible, bootup-tune plays), but then neither USB-keyboard nor network I/O is working, preventing all access to the running system.

rek5

@Warhol:

Thanks a lot to TheNarc for posting the compiled module!

I tried it with pfSense 2.4.2 on a ZBOX CI327:

The module does load! :)

However, I had to set loader options opposite to what TheNarc suggested:

hw.re.msi_disable=0

hw.pci.enable_msi=1

hw.pci.enable_msix=1

With the options set otherwise, I experienced weird behaviour - either the boot hangs completely, or the system comes up (VGA console visible, bootup-tune plays), but then neither USB-keyboard nor network I/O is working, preventing all access to the running system.

I will try out these settings on my 327 and report back.

TheNarc

I have the older CI323, so it definitely sounds like the CI327 uses a different Realtek chipset than the CI323. I found a post on a different forum that seems to confirm this, but not what the two chipsets actually are: https://forums.untangle.com/hardware/38992-zotac-zbox-nano-intel-n3150-3160-3450-comments.html.

If you run the command pciconf -l on pfSense, either via SSH or the web configuration page (Diagnostics > Command Prompt), you'll get a bunch of output that includes two lines that begin with re (one each for re0 and re1). On those lines there should be a chip= field. On my CI323 this field is chip=0x816810ec, which I have to assume means that the chipset in the CI323 is the RTL8168. Running this on a CI327 should reveal its chipset.

Mind you, I don't claim that knowing the chipset will necessarily provide any useful or actionable information, but at least it's good to know ;)

bgbird03

Yeah, I guess I've got the same thing. My output is (from the web command prompt):

re0@pci0:2:0:0: class=0x020000 card=0x012310ec chip=0x816810ec rev=0x0c hdr=0x00
re1@pci0:3:0:0: class=0x020000 card=0x012310ec chip=0x816810ec rev=0x0c hdr=0x00

So…good to know that when TheNarc posts a driver update/compilation, it will work with mine :)

Warhol

With TheNarc's driver module, my CI327 box is now up and stable since 5 days. I'll call this a huge success! Before, it used to crash within 24 hours. :)

Also I can confirm bgbird03's hardware ID for the CI327:

# pciconf -l | grep ^re
re0@pci0:2:0:0: class=0x020000 card=0x012310ec chip=0x816810ec rev=0x0c hdr=0x00
re1@pci0:3:0:0: class=0x020000 card=0x012310ec chip=0x816810ec rev=0x0c hdr=0x00

What would be a good place / good way for a concise write-up about Zotac boxes? Including the driver and loader options CI323 vs CI327, harware ID etc. Right now it's bits and pieces scattered through this thread. A sticky somehow?

The doc wiki sounds like a natural place for this kind of write-up, but it seems not to be as open as this forum for signing up to edit.

xciter327

Just reporting my CI323 nano is also stable for many days now with latest from TheNarc. Also experiences issues when fiddling with the MSI/MSIX parameters. It either did not work and had no adapters or only one was working. This prevented the config from applying. Currently sticking with defaults.

Side questions:
I am guessing people are running this box with the offloading features disabled?

TheNarc

I've definitely always run with the offloading features disabled. From what I've heard anecdotally (though I don't claim to be any sort of expert on the subject) Realtek chipsets in particular are notorious for incorrectly implementing the functionality that handles that offloading. Now, I also don't know whether the offloading occurs in silicon, the driver, or some combination of the two. So it may be worth trying using the latest 1.94 driver (the binary I posted before). But only if you're not running anything mission-critical and realize that it will most likely cause problems.

jovimon

After reading through this thread and looking up some information on the Internet, I compiled a little "for dummies" guide to compile the Realtek Driver in FreeBSD, so you can use it in pfSense.

You can find it here:

https://gist.github.com/jovimon/524e116471f249626fd2ccd141f3fe05

duren

Yeah I had 2.3.2 embedded on my 323 for a long time and it was rock solid, then I decided to upgrade, upgraded itself to 2.3.4 and was botched so I loaded the key with a fresh 2.3.4 and restored config, worked like a charm except I started to experience the instability mentioned in this thread. I went to 2.4.2 on an SSD, instability continued, until replacing the Realtek driver.

I'm not sure what changed since 2.3.2 but this driver appears to be a requirement now.

Big thanks to TheNarc for saving me compilation time.

NewJoker

Just needed to give a huge "Thank you", to OP TheNarc and the following support throughout this thread. this was the biggest issue I have had since starting with pfsense.

thekorn

Another thank-you to TheNarc! Upgraded my internet connection from 155Mbps to 350Mbps and immediately started having resets. So far so good, happy pushing 350Mbps through with 1.94! (pfsense 2.4.2)

Didn't do any of the msi options at all, just loaded the new driver and hit reboot.