Zotac ZBOX CI323 nano

OK

Hi all,

being new to pfSense and BSD (but not to firewalling and computers in general), I thought this little box might be a good idea. Well, not so much at the moment.

With the Celeron N3150 being quite new, there is a serious lack of support for the hardware, leading to a situation where most OS did not install correctly. Adding a BIOS that only has a broken "Legacy Boot" option (blank screen and nothing happens), this leaves me with the following results:

Windows 10 works flawlessly, but who wants that
MINT Linux installed, but X does not work. Didn´t fiddle around with the settings, but I assume the internal display is not yet supported. Same for CentOS.
FreeBSD installed only when using the "FreeBSD-10.2-RELEASE-amd64-uefi-memstick.img" All ISO were said to be unbootable (Rufus)

With no Legacy Boot and no UEFI image for pfSense I´m kind of stuck at the moment. The hardware itself seems to be very capable of running FreeBSD, so I assume it´s worth still chasing that.

Unless Zotac release a BIOS update that permits Legacy Boot, I wonder what is needed to create a UEFI image for pfSense. It worked for FreeBSD, so in theory it´s possible for pfSense as well (correct me if I´m wrong).

If you have any other ideas, please let me know. I strictly refuse to install Win 10 with Virtualbox :D (but tested that and it worked, although the windows driver for the realteks seems to strip away the VLAN tags).

Cheers !

EDIT:

Maybe the BSD gurus can help here. When I boot into the EFI shell, I can navigate to "fs0:" then "cd boot" and "ls" gives me two executables, "boot1.efi" and "loader.efi". When starting "loader.efi", I get this (see attachment).

Can someone explain what the issue is ?

![zotac boot uefi loader 1.jpg](/public/imported_attachments/1/zotac boot uefi loader 1.jpg)
![zotac boot uefi loader 1.jpg_thumb](/public/imported_attachments/1/zotac boot uefi loader 1.jpg_thumb)

OK

Never underestimate the danger of assumptions….

When I said that Legacy Boot is broken, this is true as long as there is a HD TFT display attached, at least the one I used via Displayport. Instant blank screen. BUT:

Using a good old analog VGA Monitor, everything worked like a charm. Using the memstick to run live or install to SSD.

The baby is now up and running, interfaces assigned, the VLANs work great (not like under the windows I had running, where the realtek driver looked like a pre-alpha PITA).

So happy !

The base config with 4 VLANs, manual outbound natting, proxyarp was setup in no time, I really start enjoying this.

CI321 compared to CI323:
2013 vs 2015
2 cores vs 4
11W vs 6W
16GB vs 8GB (only drawback)
no fans :)

Cheers !

almost the same price, currently

tazmo

Hi-

I too am looking at purchasing this box for my first pfsense router… 2 NICs, 4 cores, AES-NI support for OpenVPN, and inexpensive, make it very attractive (at least to me).

What about Wifi?

Have you tested OpenVPN throughput?

Anything more you could provide on the Zotac ZBOX CI323 would be greatly appreciated.

Thanks,
Bob

OK

I bought the box with something different in mind, so I haven´t built and measured VPNs yet. So sorry, no direct answer :)

Wireless does not seem to be supported at the moment, but I honestly didn´t try very much and will pull the wireless cards out anyway - in my case I use it cable based only.

What I did though is run a hypervisor on it and then have pfsense in a vm. Reason for this is twofold: First, instant firewall recovery by using a cloned VM, second: Utilizing the hardware (8GB, 4 cores) to run a second or third VM as syslog server or a dedicated separate security box. Just in case something is not available as a package yet or for trying stuff out, like the Sophos UTM for example, while keeping the main firewall running and untouched.

ESXi is a nightmare and soooo picky about hardware, it didnt´t install and I spent quite some time. That does not say much, but I gave up after trying most tips I found online.

Xen seems to work so far, 2 days uptime with no issues. However, I am unable to pass VLANs into pfsense, as the NICs are seen as xn0 instead em0. Not a big issue as long as we´re talking 7 VLANs or less, as one creates one xn interface per vlan on the hypervisor, so the pfsense box sees just native, untagged frames.

So yes, running pfsense on this box is very well possible minus the wifi, but that´s from someone who has no interest in fixing the wireless part, there may be ways to get there.

tazmo

Thanks for the response… very interesting. One more question:

I can't seem to find what chipset the Gigabit ethernet cards use. I've looked a lot of places. It seems to be a Realtek card but I can't find a model number. I see you mention xn0 but that's a FreeBSD Xen NIC driver if I'm not mistaken...

Do you know what model it is?

Regardless, I just ordered one and of course, it's on backorder >:(

Regardless, thanks for the response and I hope you Xen install continues to be a solid one...

Bob

OK

Yes it´s a realtek, using lspci form the xenserver console i see that they are recognized as

Realtek RTL8111/8168/8411

Xen uses kernel driver r8169 to access these devices.

While we´re at it, lspci also finds the wireless card as an intel wireless 3160 but doesn´t use it as a NIC from a xenserver perspective.

I will double check that from a bare metal pfsense installation and see what BSD makes of the hardware, but that will happen a few days down the road.

securvark

Hey guys,

I'm about to take the plunge on this little box to build my own router and replace a TP-Link N750.

Any news on the wifi, usb 3, bluetooth and cardreader?

Thanks!

OK

To my shame I never continued to work in that direction, as my setup does not use anything but the cable-based NICs.

What I can say is that a config with Xenserver as the HostOS and pfsense on one of the VMs runs stable for about 4 weeks now, so unless you need WiFi, I´d recommend the box.

interfasys

The card reader does not work on 10.2 or 11
I've installed the OS from a USB3 stick, so that works
It's got enough power to compile your packages from ports or a kernel

There is one big caveat though: The NICs give up under load if you're using netmap. So you can't use that box as-is if you want to do IPs with Suricata until Suricata gets fixed to work with drivers which don't support netmap.

rajl

@interfasys:

The card reader does not work on 10.2 or 11

I've installed the OS from a USB3 stick, so that works

It's got enough power to compile your packages from ports or a kernel

There is one big caveat though: The NICs give up under load if you're using netmap. So you can't use that box as-is if you want to do IPs with Suricata until Suricata gets fixed to work with drivers which don't support netmap.

As an alternative, you can wait until the drivers do support netmap. From the netmap website:

Netmap-aware device drivers are needed to use netmap at high speed on ethernet ports. To date, we have support for Intel ixgbe (10G), ixl (10/40G), e1000/e1000e/igb (1G), Realtek 8169 (1G) and Nvidia (1G). FreeBSD has also native netmap support in the Chelsio 10/40G cards.

I'm not sure what all is required for netmap support for Realtek RTL8111/8168/8411 versus Realtek 8169, but this could be a "simple" coding project for someone with the time if there are enough similarities between the 8168 and the 8169 drivers.

rajl

I may stand corrected. I just browsed the Realtek driver and netmap driver code in the FreeBSD stable branch. It looks like all versions of the Realtek gigabit chipsets are are supported by netmap in FreeBSD.

interfasys

;)
That's correct, it's been "supported" from the start. If you read the code, you can see that performance is more akin to a half-gigabit NIC.

The lock up problem has been reported upstream and since there is sort of a workaround by using jumbo frames, I have hope it's something which can be fixed.

rajl

@interfasys:

;)
That's correct, it's been "supported" from the start. If you read the code, you can see that performance is more akin to a half-gigabit NIC.

The lock up problem has been reported upstream and since there is sort of a workaround by using jumbo frames, I have hope it's something which can be fixed.

Unfortunately, my working knowledge of BSD driver code is good enough to tell if a feature is enable, but not good enough to see that performance with netmap is around ~500 Mb/s instead of 1Gb/s. Why is there such a performance hit for the Realtek using netmap?

A second question – why does the Realtek lock up? And what is it about jumbo frames the keeps the Realtek from locking up? And, more importantly, would I have to enable jumbo frames for my entire network, or just on the Realtek interface of this particular device in order to prevent the lockup from happening?

interfasys

@rajl:

Why is there such a performance hit for the Realtek using netmap?

I don't think it's related to netmap, it's either the chip or the driver, because of badly designed chips, bad documentation, bad original driver (because of bad documentation), etc.

@rajl:

A second question – why does the Realtek lock up? And what is it about jumbo frames the keeps the Realtek from locking up? And, more importantly, would I have to enable jumbo frames for my entire network, or just on the Realtek interface of this particular device in order to prevent the lockup from happening?

My theory is that netmap overfills the card's buffer and at some point the card can't cope any more and we end up with interrupts piling up.
By turning on jumbo frames, the total number of mbufs is split equally between the 2 types of frames and the card only almost dies (dropping from 350kpps to less than a 100).
Ideally, you'd need all your network to support 9k frames to be able to see the benefits, but if you just want the fix, you can just turn it on for the LAN interface. There will be side effects and so, you should read about what happens when using large frames with equipment which doesn't support it.

interfasys

If jumbo frames don't work for you, you can use the emulated mode by setting "dev.netmap.admode" to 2. In my tests, I get the same throughput, but use a lot more CPU.

rajl

Interesting. Have you tested the same chip with another OS that supports netmap? I know Linux has different drivers and supports netmap. If Linux exhibits the same or similar behavior, then the problem is with the chip itself most likely. If Linux runs better, then the problem is probably with the BSD code (and likely fixable).

interfasys

@rajl:

Interesting. Have you tested the same chip with another OS that supports netmap? I know Linux has different drivers and supports netmap. If Linux exhibits the same or similar behavior, then the problem is with the chip itself most likely. If Linux runs better, then the problem is probably with the BSD code (and likely fixable).

Not yet. I need to boot into IPFire or something and apply the same pkt-gen test

rajl

@interfasys:

@rajl:

Interesting. Have you tested the same chip with another OS that supports netmap? I know Linux has different drivers and supports netmap. If Linux exhibits the same or similar behavior, then the problem is with the chip itself most likely. If Linux runs better, then the problem is probably with the BSD code (and likely fixable).

Not yet. I need to boot into IPFire or something and apply the same pkt-gen test

Well, if you end up testing it out, let me know. I would be interested in the results. I just bought one of these things to use as my first PFSense box, but am a little concerned after reading this thread. But maybe I shouldn't be since I was planning on using Snort instead of Suratica (unless Snort uses netmap also and I am just unaware).

interfasys

netmap is the future, for IPS or just packet forwarding with netmap-fw. I'm sure the problem will be fixed eventually. It could simply be a problem with the 8111G revision. FreeNAS users had similar issues a few years back and were forced to use the Realtek drivers while waiting for a fix, so I think it will just be a matter of being patient (or paying someone to fix the problem).

PjotterThisIs

I've bought the same Zbox (CI323). I would like to do the same thing: Install XenServer and install pfSense or rather Sophos UTM in a VM.

However, I've a probably simple (noob) question: When you install XenServer, you need to specify an IP address etc. But the VM inside this machine is going to be my router, so how is that going to work?

Can anybody help me?

rajl

Not sure. But this question is probably better answered by the guys who hang out in the Virtualization sub-thread. Those guys use Xen-Server and VMWare all the time.

PjotterThisIs

Thanks a lot! I've a look at that!

ddarlington36

Really following this thread for the updates, I'm about to purchase one of these boxes for the same purpose as most here 'pfsense' guess I'll be using xenserver if exsi 6.0 doesn't work with the hardware yet was hoping it would

Main role will be openvpn client/pia plus a PBX voice server with the use of a vlan switch and exclude the VPN to just a certain IP range

had hopes of using wireless without having to bridge another router
I don't have any issues with replacing the actual WiFi card with one that works I just need advice on which wireless card to use looking to use dual band ac 1200/ I see a lot of people can't get theirs to work either. Are most of the conflicts just driver support not up to date yet..

rajl

I'm waiting for PFSense 2.3 to hit release before putting this Zotac through it's paces. However, PFSense 2.3 probably won't hit release until FreeBSD 10.3 is released at the end of March. I'll be happy to report my experiences on this thread when I do. However, I would not anticipate any problems. The hardware is well supported and the only potential issue is using netmap with Suricata on this device (I am more of a Snort guy).

Hugh Jorgan

Hey guys! I bought a Zotac CI-323 and have had the same experience as OP. Need to make this into a wireless router/dns sinkhole. I can boot pfsense daily without issue, I just dont know how to set this up entirely.

Rango

Hi guys. I just purchased this box as well. How long did you guys wait before it got to your house?
As far as ESXi 6 and realtek driver it seem this has been fixed by injecting net55-r8168 driver into esxi iso image before installing. If you can confirm that this works would be great
.
http://www.v-front.de/2015/03/vsphere-6-is-ga-ultimate-guide-to.html

Realtek 8168 and VMware 6.0 :

net55-r8168-8.039.01-napi.x86_64.vib

I've also read some threads of some random disconnect in other thread on pfsense forum but that also seem to be fixed. I'm assuing you it's smooth sailing for you guys since it's been a while since you guys posted.

Looking forward to feedback on this box and hoping to get it soon. Hope you guys can assist me if i run into trouble he he.

movax

Hi.

So, did anybody done some performance tests on this little thing? (pfsense, FreeBSD, Linux, doesn't matter for me).

I am thinking about getting one, but I would like for it to be able to do IPv4 NAT + IPv6 at full gigabit speed (at ~1KB packets) between two ports. Was somembody able to do this? What was the CPU load?

Thanks!

Rango

@movax:

Hi.

So, did anybody done some performance tests on this little thing? (pfsense, FreeBSD, Linux, doesn't matter for me).

I am thinking about getting one, but I would like for it to be able to do IPv4 NAT + IPv6 at full gigabit speed (at ~1KB packets) between two ports. Was somembody able to do this? What was the CPU load?

Thanks!

Throughput testing even with openvpn is tested in this thread. It will easily do 100Mbps on openvpn and 400Mbps on unencrypted lan, although i think intel nic are better but for that you would need to spend $360 plus unless u wanna do applicence but then no vga card. I think this will improve once the drivers get worked out maybe in 2.3 release?

https://forum.pfsense.org/index.php?topic=87217.15

movax

@Rango:

@movax:

Hi.

So, did anybody done some performance tests on this little thing? (pfsense, FreeBSD, Linux, doesn't matter for me).

I am thinking about getting one, but I would like for it to be able to do IPv4 NAT + IPv6 at full gigabit speed (at ~1KB packets) between two ports. Was somembody able to do this? What was the CPU load?

Thanks!

Throughput testing even with openvpn is tested in this thread. It will easily do 100Mbps on openvpn and 400Mbps on unencrypted lan, although i think intel nic are better but for that you would need to spend $360 plus unless u wanna do applicence but then no vga card. I think this will improve once the drivers get worked out maybe in 2.3 release?

https://forum.pfsense.org/index.php?topic=87217.15

I seen openvpn results.

I must have missed 400Mbps on unencrypted lan however. I am somehow disappointed about the performance then. Still it doesn't tell if it was due cpu being fully loaded, or something else being a bottleneck, or whatever it was actually trying to load it to the limits.

If you say the drivers might be an issue, would Linux behave possibly better here?

Thanks again!

(I already have a router that is doing about 900Mbps routing and nat, on a almost 10 year old machine. I am just searching for something much smaller and power efficient).

Rango

@movax:

@Rango:

@movax:

Hi.

So, did anybody done some performance tests on this little thing? (pfsense, FreeBSD, Linux, doesn't matter for me).

I am thinking about getting one, but I would like for it to be able to do IPv4 NAT + IPv6 at full gigabit speed (at ~1KB packets) between two ports. Was somembody able to do this? What was the CPU load?

Thanks!

Throughput testing even with openvpn is tested in this thread. It will easily do 100Mbps on openvpn and 400Mbps on unencrypted lan, although i think intel nic are better but for that you would need to spend $360 plus unless u wanna do applicence but then no vga card. I think this will improve once the drivers get worked out maybe in 2.3 release?

https://forum.pfsense.org/index.php?topic=87217.15

I seen openvpn results.

I must have missed 400Mbps on unencrypted lan however. I am somehow disappointed about the performance then. Still it doesn't tell if it was due cpu being fully loaded, or something else being a bottleneck, or whatever it was actually trying to load it to the limits.

If you say the drivers might be an issue, would Linux behave possibly better here?

Thanks again!

(I already have a router that is doing about 900Mbps routing and nat, on a almost 10 year old machine. I am just searching for something much smaller and power efficient).

Don't quote me on 400Mbps as this is what i've seen on different website and could be untrue. I know Realtek will have less throughput then intel nics but i don't really know by how much. I've only looked into openvpn honestly. I may do higher then what i've posted. Freebsd will have an update in april so this may be resolved by then.
Don't think linux will be any different as this is driver related, meaning realtek driver and some better hardware capability of intel nic design.
If you're looking for true 1Gbs performace i would focus on intel nics but you will pay 3x more or you will have to wait and build yourself PC applience with only console access and no vga. That will be in same price range as NUC but you will only have access via console port. I don't like this personally and i don't care for 1GB peformace yet.
You're always limited by ISP speed anyway unless you're doing your own LAN and care about that but not sure what application at home you would need for true 1gb performace.

If you don't care about spending $360-$450 go with atom cpu and one of those boards with intel nic posted somewhere in this forum.

movax

@OK:

Never underestimate the danger of assumptions….

When I said that Legacy Boot is broken, this is true as long as there is a HD TFT display attached, at least the one I used via Displayport. Instant blank screen. BUT:

Using a good old analog VGA Monitor, everything worked like a charm. Using the memstick to run live or install to SSD.

The baby is now up and running, interfaces assigned, the VLANs work great (not like under the windows I had running, where the realtek driver looked like a pre-alpha PITA).

So happy !

The base config with 4 VLANs, manual outbound natting, proxyarp was setup in no time, I really start enjoying this.

CI321 compared to CI323:
2013 vs 2015
2 cores vs 4
11W vs 6W
16GB vs 8GB (only drawback)
no fans :)

Cheers !

almost the same price, currently

I am probably going to risk, and try and put 16GB in CI323. :)

I found one comment on amazon.de, claiming that it works on this very particulate model, with Linux, few VMs, and all. I also found that there is some other single mini-ITX board from some vendor (forgot now), that is using same CPU, and also claims 16GB support. Most of the other sites, including Intel Ark, claims 8GB on all different board, but, maybe, just maybe, it is per-channel/socket limit.

If it doesn't work, I will just return 2x8GB, and get 2x4GB (which are almost same price anyway).

And if it doesn't perform very well for me as a router, I still kind of want to have mid range Windows desktop, because reasons.

Rango

Yeah let me know if 16GB worked. I ordered 8 only although it does not makes sense to me that 2013 release was 16gb and 2015 is 8gb.
The cpu spec sheet says it support 8gb but maybe it will work. Let me know.

btw even 2GB with pfesnes is enough providing you're not running packages. If 5-10 packages 4GB will suffice and 8gb is overkill but if you make a pc later then 16GB would be better obviously.

movax

@Rango:

Yeah let me know if 16GB worked. I ordered 8 only although it does not makes sense to me that 2013 release was 16gb and 2015 is 8gb.
The cpu spec sheet says it support 8gb but maybe it will work. Let me know.

btw even 2GB with pfesnes is enough providing you're not running packages. If 5-10 packages 4GB will suffice and 8gb is overkill but if you make a pc later then 16GB would be better obviously.

Will do, just ordered one.

2GB is ok for just networking (I almost forgot that this is a pfsense forum, but still this is a hardware specific thread).

But it is always good for stuff like webcaching, or if I turn it into desktop box (even just for browsing internet, 8gb is not enough for me these days), some developement box, or a backups storage server, where it will nicely act as a file system cache. Price difference is very small. This is why I would prefer 16GB a lot over 8GB.

movax

First report - 16GB works just fine. I have put HyperX Impact (2x, 8GB, DDR3L-1600 (PC3-12800), SODIMM 204), product numebr HX316LS9IBK2/16
, and it shows as 16GB in BIOS/UEFI. Booted Linux and it shows 16GB in dmesg and in free. Done some tests in python on livecd, and I can use about 15GB until the process is killed (there are some other processes).

Nice.

movax

Quiet powerful. With Linux, frequency governor changes to performance, and no other tweaks, I am getting 900Mbps with NAT from eth0 to vlan11@eth1.Tested both way, but not at the same time, but the cpu usage looks ok. The IRQ load is crazy high, and ethtool shows no options for interrupts coalescing. :( With powersave frequency governor from time to time CPU usage would go to 80%, and some strange things would happen. With performance it is getting very low.

cores temps around 52,53,56,60 deg C, in a room of about 25 deg C.

The power supply is tiny, and is not hot at all.

Pretty impressive.

Love.IT.Again

@movax:

Quiet powerful. With Linux, frequency governor changes to performance, and no other tweaks, I am getting 900Mbps with NAT from eth0 to vlan11@eth1.Tested both way, but not at the same time, but the cpu usage looks ok. The IRQ load is crazy high, and ethtool shows no options for interrupts coalescing. :( With powersave frequency governor from time to time CPU usage would go to 80%, and some strange things would happen. With performance it is getting very low.

cores temps around 52,53,56,60 deg C, in a room of about 25 deg C.

The power supply is tiny, and is not hot at all.

Pretty impressive.

Sir, How about the power consumption of this unit? It was advertised with 6W consumption. How about pfsense running on it? It seems there are so many people waiting for this, it was sold out at Newegg yesterday, but they got some again in the afternoon.

g1bson

@OK:

Using a good old analog VGA Monitor, everything worked like a charm. Using the memstick to run live or install to SSD.

The baby is now up and running, interfaces assigned, the VLANs work great (not like under the windows I had running, where the realtek driver looked like a pre-alpha PITA).

So happy !

The base config with 4 VLANs, manual outbound natting, proxyarp was setup in no time, I really start enjoying this.

Did you try to install Pfsense on ZBOX CI323 directly? Is it possible? Or only in hypervisor in Linux?

Love.IT.Again

@g1bson:

@OK:

Using a good old analog VGA Monitor, everything worked like a charm. Using the memstick to run live or install to SSD.

The baby is now up and running, interfaces assigned, the VLANs work great (not like under the windows I had running, where the realtek driver looked like a pre-alpha PITA).

So happy !

The base config with 4 VLANs, manual outbound natting, proxyarp was setup in no time, I really start enjoying this.

Did you try to install Pfsense on ZBOX CI323 directly? Is it possible? Or only in hypervisor in Linux?

I bought one of this ZBOX C1323 at eBay last week. I got a $15 off on Paypal so I just paid about $135 more. However, the RAM was purchased at Newegg and after a week now, the package is still on its way. Although, I received the ZBOX after three days, I can't make use at the moment. I should have bought it from Amazon to utilize my Prime account's 2 day delivery. It's my first time that Newegg failed to impress me. Usually, I get the package 3-4 days.

Same as you, I was also looking for some feedbacks regarding this ZBOX's performance using pfSense but I can't seems to find any here in the forum. What triggered me to get one like this was the reviews and feedbacks from Amazon.com and they are telling that this ZBOX is good for pfSense. Hopefully those reviews at Amazon were true.

I will post my views regarding this ZBOX once I get the whole unit together.

DigitalPackrat

@g1bson:

Did you try to install Pfsense on ZBOX CI323 directly? Is it possible? Or only in hypervisor in Linux?

Walked my father through installing Pfsense 2.3 on a CI323 earlier today over the phone. Just set the BIOS to legacy and installed everything off the USB stick to a 60GB SSD. The wireless card that came with the unit doesn't show up not that I expected it to or cared about it.

hazza

Just ordered a Zotac CI323 based on the opinions in this thread. Been looking for a suitable dual-nic mini PC to replace my existing netbook+usb ethernet dongle setup for a while now; the Zotac appears to tick all the boxes. :)