Zotac ZBOX CI323 nano

IggyB

After a few weeks of trying to fix it i think i finally have my zotac box stable, no "watchdog timeout" and loss of WAN IP under heavy load in about 4 days. Longest run ever.

Hi Doboy,

Could you describe this issue more to me? I'm not sure i understand, how would one know if connection is dropping out? I haven't experienced anything like this. However i do experience a weird issue sometimes where network traffic would come to a halt for couple of seconds, the WAN doesn't seem to be dropping. I think this issue started appearing after i installed pfblocker and added some lists, i'm also running av scan on traffic through squid on 850 evo SSD. Without these packages i haven't had these issues.

Can you tell me how to check for this watchdog timeout dropout issue and how you compiled a new driver etc. Thanks.

IggyB

Have you guys updated the BIOS yet?

https://www.zotac.com/us/files/download/by_product?p_nid=501278&driver_type=238&os=All

I don't understand the latest changes.

Can anyone try to to explain?

Version 2K161012

Changed memory slot power (when AC Power attached)
Fixed ME / TXE disabled (when Deep Sleep S5 enabled)

Doboy

You would see the "watchdog timeout" error on your console for the pfsense, for me i got those repeated errors as well as a dropped WAN Interface ( no ip) only thing to bring it back was a hardware reboot.

Hard setting my WAN to 100Mbps(my WAN was 120Mbps) causes less errors but still happened almost daily if I pushed too much traffic through the WAN interface.

if you search " watchdog timeout" + zotac CI323 on google you will see plenty of threads on it.

So far I'm past 5 days with no errors since i compiled latest driver

@IggyB:

Have you guys updated the BIOS yet?

https://www.zotac.com/us/files/download/by_product?p_nid=501278&driver_type=238&os=All

I don't understand the latest changes.

Can anyone try to to explain?

Version 2K161012

Changed memory slot power (when AC Power attached)

Fixed ME / TXE disabled (when Deep Sleep S5 enabled)

Doboy

Yes I updated the BIOS no changes for me as far as performance or stability

@IggyB:

Have you guys updated the BIOS yet?

https://www.zotac.com/us/files/download/by_product?p_nid=501278&driver_type=238&os=All

I don't understand the latest changes.

Can anyone try to to explain?

Version 2K161012

Changed memory slot power (when AC Power attached)

Fixed ME / TXE disabled (when Deep Sleep S5 enabled)

IggyB

Thanks Doboy for making that clear.

I'm not having any problems like that. My WAN is only 10mbit not enough to push it hard. However i might be switching over to Docsis 3.0 cable which would put me just above 100mbit maybe then i will start experiencing same problems. Can you tell me how you compiled the new driver for it? Is there a guide for it? Thanks.

Doboy

I compiled the driver using a VM with freeBSD 10.3 on it, then copied over the .ko file to the pfsense box and load it at startup in the the boot.loader

flyride

Is there any way you could post the compiled .ko file and explain the procedure for adding it to the boot? I have the same problem only on LAN (RE0) if I run our IP security cameras it overloads it, I have to disconnect them to keep the LAN running more then a few minutes.

Any assistance would be greatly appreciated!

IggyB

Yeah sure soon as I get back home and have the PC up and running.

flyride

@IggyB:

Yeah sure soon as I get back home and have the PC up and running.

Thank you, that would be awesome! So I assume you got it working as well?

thlunk

Just ordered one of these with 4GB Crucial RAM and a Sandisk 120GB SSD. Should get it all Sunday

flyride

For anyone interested I stumbled across some of DoBoY's experience figuring some of this out while Googling: https://forums.freebsd.org/threads/55861/

IggyB

Hey sorry, just got my main desktop up and running. I had some trouble after draining my watercooling loop, had to go out and buy a new block. then my pumps reservour cracked. etc.. lol

1. Install Kernel FreeBSD 10.3 with kernel sources.
1. Get kernel driver 1.91 from realtek site. You don't need the patch for driver, only if you're using FreeBSD 11+
2. Modify /usr/src/sys/amd64/conf/GENERIC and delete line "device re" - so if_re no longer be included in kernel itself - module will be build anyway.
3. Edit /boot/loader.conf adding line if_re_load="YES", so external module will be loaded.
4. Go to /usr/src and run "make buildkernel KERNCONF=GENERIC && make installkernel KERNCONF=GENERIC"
5. Reboot with new kernel. System should boot with open-source driver as module. kldstat will show if_re in list of modules.
6. Put Realtek's if_re.c and if_rereg.h into /usr/src/sys/dev/re
7. Build driver (cd /usr/src/sys/modules/re && make clean && make)
8. Backup /boot/kernel/if_re.ko, then cp /usr/src/sys/modules/re/if_re.ko /boot/kernel/if_re.ko && chmod 0555 /boot/kernel/if_re.ko && chown root:wheel /boot/kernel/if_re.ko
9. Reboot

Now i wasn't sure how to upload files to internet using lynx or fetch so i installed xorg server and kde on vm FreeBSD. This took a while and i had some issues with timeouts etc.
Basicly i uploaded new compiled if_re.ko to some small file hosting sites, used a tinyurl to shorten it with direct link

SSH into pfSense and used fetch to download it into /boot/kernel/
chmod 0555 /boot/kernel/if_re.ko && chown root:wheel /boot/kernel/if_re.ko

edit /boot/loader.conf in pfSense. Can do through web browser or through ssh client with vi
add line if_re_load="YES"

That should be it. Reboot and your 1.92 should be loaded. If you want to unload it just put # infront of if_re_load="YES" in /boot/loader.conf
If you're having issues compiling or uploading to web/usb stick i will be happy to upload it for you or send you somehow a already compiled one

thlunk

Just got mine all up and running. I must say, I am truly impressed with pfSense. This was my first build with pfSense and it's by far the best firewall I've used. I'm not using snort/squid or anything (yet), but I'm connected to VPN and getting ~90-100Mb. I usually get ~180Mb not on VPN, so I'm very happy with that given the cost of this box with a SSD/RAM is around ~$250. It took me a little bit to learn the GUI (which is very intuitive I must say) and figure out my firewall rules to force all traffic (except one device my wife uses for Hulu because Hulu doesn't allow connections on VPN) through the VPN, but it's all configured and working properly now.

Danrbball

I just ordered a CI323 to run pfsense and wanted to make sure everyone running it would still buy one of these devices again. I know there have been driver issues, but it appears if you limit the wan or run in a VM it's a great device.

rightnow

@IggyB:

Hey sorry, just got my main desktop up and running. I had some trouble after draining my watercooling loop, had to go out and buy a new block. then my pumps reservour cracked. etc.. lol

1. Install Kernel FreeBSD 10.3 with kernel sources.
1. Get kernel driver 1.91 from realtek site. You don't need the patch for driver, only if you're using FreeBSD 11+
2. Modify /usr/src/sys/amd64/conf/GENERIC and delete line "device re" - so if_re no longer be included in kernel itself - module will be build anyway.
3. Edit /boot/loader.conf adding line if_re_load="YES", so external module will be loaded.
4. Go to /usr/src and run "make buildkernel KERNCONF=GENERIC && make installkernel KERNCONF=GENERIC"
5. Reboot with new kernel. System should boot with open-source driver as module. kldstat will show if_re in list of modules.
6. Put Realtek's if_re.c and if_rereg.h into /usr/src/sys/dev/re
7. Build driver (cd /usr/src/sys/modules/re && make clean && make)
8. Backup /boot/kernel/if_re.ko, then cp /usr/src/sys/modules/re/if_re.ko /boot/kernel/if_re.ko && chmod 0555 /boot/kernel/if_re.ko && chown root:wheel /boot/kernel/if_re.ko
9. Reboot

Now i wasn't sure how to upload files to internet using lynx or fetch so i installed xorg server and kde on vm FreeBSD. This took a while and i had some issues with timeouts etc.
Basicly i uploaded new compiled if_re.ko to some small file hosting sites, used a tinyurl to shorten it with direct link

SSH into pfSense and used fetch to download it into /boot/kernel/
chmod 0555 /boot/kernel/if_re.ko && chown root:wheel /boot/kernel/if_re.ko

edit /boot/loader.conf in pfSense. Can do through web browser or through ssh client with vi
add line if_re_load="YES"

That should be it. Reboot and your 1.92 should be loaded. If you want to unload it just put # infront of if_re_load="YES" in /boot/loader.conf
If you're having issues compiling or uploading to web/usb stick i will be happy to upload it for you or send you somehow a already compiled one

Would it be ossible to get your compiled version? I cant get it to compile :(

all5n

@IggyB:

Hey sorry, just got my main desktop up and running. I had some trouble after draining my watercooling loop, had to go out and buy a new block. then my pumps reservour cracked. etc.. lol

1. Install Kernel FreeBSD 10.3 with kernel sources.
1. Get kernel driver 1.91 from realtek site. You don't need the patch for driver, only if you're using FreeBSD 11+
2. Modify /usr/src/sys/amd64/conf/GENERIC and delete line "device re" - so if_re no longer be included in kernel itself - module will be build anyway.
3. Edit /boot/loader.conf adding line if_re_load="YES", so external module will be loaded.
4. Go to /usr/src and run "make buildkernel KERNCONF=GENERIC && make installkernel KERNCONF=GENERIC"
5. Reboot with new kernel. System should boot with open-source driver as module. kldstat will show if_re in list of modules.
6. Put Realtek's if_re.c and if_rereg.h into /usr/src/sys/dev/re
7. Build driver (cd /usr/src/sys/modules/re && make clean && make)
8. Backup /boot/kernel/if_re.ko, then cp /usr/src/sys/modules/re/if_re.ko /boot/kernel/if_re.ko && chmod 0555 /boot/kernel/if_re.ko && chown root:wheel /boot/kernel/if_re.ko
9. Reboot

Now i wasn't sure how to upload files to internet using lynx or fetch so i installed xorg server and kde on vm FreeBSD. This took a while and i had some issues with timeouts etc.
Basicly i uploaded new compiled if_re.ko to some small file hosting sites, used a tinyurl to shorten it with direct link

SSH into pfSense and used fetch to download it into /boot/kernel/
chmod 0555 /boot/kernel/if_re.ko && chown root:wheel /boot/kernel/if_re.ko

edit /boot/loader.conf in pfSense. Can do through web browser or through ssh client with vi
add line if_re_load="YES"

That should be it. Reboot and your 1.92 should be loaded. If you want to unload it just put # infront of if_re_load="YES" in /boot/loader.conf
If you're having issues compiling or uploading to web/usb stick i will be happy to upload it for you or send you somehow a already compiled one

Thanks. I installed FreeBSD 10.3 in a virtualbox VM with the source code, and was able to re-build the kernel based on your instructions, and then build the if_re module and load that as well.

Was the purpose of re-building the kernel just to make sure the if_re.ko actually loaded/worked? Because it seems like you could just build the module (if_re.ko, steps 6-7) and copy it over to the pfsense box, add the line to loader.conf, and that's all it would take. Is there really a need really to re-build the whole kernel, now that you have proven that the re-built works?

Is it correct to assume that loading if_re.ko as a module on the pfsense box will override the version that is built into the kernel? Or does the pfsense kernel load all interfaces as modules? That would make sense. My CI323 is on the way, so dont have it to play with at the moment.

EDIT: Was able to test by installing pfsense in a virtualbox VM. The kernel seems to have loaded the module just fine, and i didnt see any other modules for the intel NICs that the virtualbox hypervisor emulates. So i think i can assume (uh oh) that the newly built if_re.ko module would override the realtek open source driver built into the kernel.

EDIT2: kldstat -v only lists the one if_re from the module, so have to assume thats the one its actually going to use.

all5n

@all5n:

@IggyB:

Hey sorry, just got my main desktop up and running. I had some trouble after draining my watercooling loop, had to go out and buy a new block. then my pumps reservour cracked. etc.. lol

1. Install Kernel FreeBSD 10.3 with kernel sources.
1. Get kernel driver 1.91 from realtek site. You don't need the patch for driver, only if you're using FreeBSD 11+
2. Modify /usr/src/sys/amd64/conf/GENERIC and delete line "device re" - so if_re no longer be included in kernel itself - module will be build anyway.
3. Edit /boot/loader.conf adding line if_re_load="YES", so external module will be loaded.
4. Go to /usr/src and run "make buildkernel KERNCONF=GENERIC && make installkernel KERNCONF=GENERIC"
5. Reboot with new kernel. System should boot with open-source driver as module. kldstat will show if_re in list of modules.
6. Put Realtek's if_re.c and if_rereg.h into /usr/src/sys/dev/re
7. Build driver (cd /usr/src/sys/modules/re && make clean && make)
8. Backup /boot/kernel/if_re.ko, then cp /usr/src/sys/modules/re/if_re.ko /boot/kernel/if_re.ko && chmod 0555 /boot/kernel/if_re.ko && chown root:wheel /boot/kernel/if_re.ko
9. Reboot

Now i wasn't sure how to upload files to internet using lynx or fetch so i installed xorg server and kde on vm FreeBSD. This took a while and i had some issues with timeouts etc.
Basicly i uploaded new compiled if_re.ko to some small file hosting sites, used a tinyurl to shorten it with direct link

SSH into pfSense and used fetch to download it into /boot/kernel/
chmod 0555 /boot/kernel/if_re.ko && chown root:wheel /boot/kernel/if_re.ko

edit /boot/loader.conf in pfSense. Can do through web browser or through ssh client with vi
add line if_re_load="YES"

That should be it. Reboot and your 1.92 should be loaded. If you want to unload it just put # infront of if_re_load="YES" in /boot/loader.conf
If you're having issues compiling or uploading to web/usb stick i will be happy to upload it for you or send you somehow a already compiled one

Thanks. I installed FreeBSD 10.3 in a virtualbox VM with the source code, and was able to re-build the kernel based on your instructions, and then build the if_re module and load that as well.

Was the purpose of re-building the kernel just to make sure the if_re.ko actually loaded/worked? Because it seems like you could just build the module (if_re.ko, steps 6-7) and copy it over to the pfsense box, add the line to loader.conf, and that's all it would take. Is there really a need really to re-build the whole kernel, now that you have proven that the re-built works?

Is it correct to assume that loading if_re.ko as a module on the pfsense box will override the version that is built into the kernel? Or does the pfsense kernel load all interfaces as modules? That would make sense. My CI323 is on the way, so dont have it to play with at the moment.

EDIT: Was able to test by installing pfsense in a virtualbox VM. The kernel seems to have loaded the module just fine, and i didnt see any other modules for the intel NICs that the virtualbox hypervisor emulates. So i think i can assume (uh oh) that the newly built if_re.ko module would override the realtek open source driver built into the kernel.

EDIT2: kldstat -v only lists the one if_re from the module, so have to assume thats the one its actually going to use.

Quick follow up on this. Got pfsense installed and the realtek driver module installed and working as per the instructions. Has worked without a hitch so far.

Don't think i am stressing the NICs too much with my 50/5 connection.

IggyB

hi
sorry haven't checked the forums in a while
for the guy who asked for it

http://s000.tinyupload.com/?file_id=67950169242632494971

that's the compiled version i use on my pfSense 2.3.2

compiled on FreeBSD 10.3

krackpot

@IggyB:

hi
sorry haven't checked the forums in a while
for the guy who asked for it

http://s000.tinyupload.com/?file_id=67950169242632494971

that's the compiled version i use on my pfSense 2.3.2

compiled on FreeBSD 10.3

I tried following your instructions, but couldn't locate the driver on Realtek's site.

I think it's the one listed under:

Network Interface Controllers
10/100/1000M Gigabit Ethernet
PCI Express <–- Is this correct?

http://www.realtek.com.tw/downloads/downloadsView.aspx?Langid=1&PNid=13&PFid=5&Level=5&Conn=4&DownTypeID=3&GetDown=false

Unix (Linux)
Apply to RTL8111H(S)/RTL8118/RTL8119i as well.
Description Version Update
FreeBSD 7.x and 8.0
1.93 2017/2/13 83k

Is the above mentioned 1.93 file under Unix/FreeBSD 7.x and 8.0 what I need to get to continue with the rest of the compile steps?

EDIT: Was able to get some help compiling the 1.93 Realtek PCI-E driver above. Seems to work without issue (so far) and I verified it was loaded.

Forsaked

There is a new generation out with an Apollo Lake processor, but still Realtek-NICs.

https://www.zotac.com/us/product/mini_pcs/ci327-nano