Intel NUC DN2820FYKH, bogus ACPI, USB boot

stig.voss · Jun 18, 2014, 10:59 PM

Hello,

I am currently working on getting a low-power/low-footprint pfSense box up and running at a smaller gas station to provide customer WiFi and company network via VLANs.
Myself, I have an Intel NUC D34010WYK2 (Haswell i3) running pfSense via a VLAN-enabled switch, crushing some 250-300 Mbps on OpenVPN.

As the bigger version handles pfSense plenty fine, I thought that the smaller version would fit the lower usage at the gas station, but no.

I have run into an issue concerning the ACPI. It seems from research that it is caused by a bogus ACPI table on the DN2820 NUC and FreeBSD*, which causes a kernel panic upon boot.

I can of course choose to disable ACPI during the boot selection screen, but as I am running pfSense from a USB-drive, the booting process does not manage to complete due to the lack of a boot delay.

I have not found a way to edit the loader.conf.local before having a fully booted system. I failed horribly at my attempt of mounting the filesystem within a FreeBSD VM to edit the loader.conf.local, or in other words, I didn't manage to get it up running anyways.

Long story short, does anyone have an idea of how I could manage to pass this issue or is my box doomed as a pfSense device?

Issue described and fixed in FreeBSD bugzilla: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=187965

Thanks in advance.

stephenw10 · Jun 19, 2014, 12:28 AM

Ok, similar to the bug already reported here:
https://redmine.pfsense.org/issues/3678
That is solved on at least one board by using an updated BIOS, are you running the latest version?
Another user booted successfully by disabling all but one CPU core in the BIOS.

What do you want to add to loader.conf.local? You can always interrupt the boot loader and set things manually.

Steve

stig.voss · Jun 20, 2014, 2:42 PM

Thank you for your reply and sorry for the late response, an exam in software testing came up. ;)

Yes it does seem like a similar bug as reported there.

I updated the BIOS and testest both with pfSense 2.1.3-RELEASE and 2.2-DEVELOPMENT, none of them would boot and ended up with an error message similar to "panic: bogus interrupt polarity". I have now checked for the possibility for disabling CPU-cores in the BIOS of my DN2820FYKH, but it does not seem to be a possibility.

Using the method of mounting the pfSense filesystem within a FreeBSD VM, added the following line to "/boot/device.hints" as stated here: https://doc.pfsense.org/index.php/Disable_ACPI

hint.acpi.0.disabled="1"

But as I am writing this, I am starting the question the validity of my actions due to the fact that this seems to be targeted towards an older version of pfSense, namely 1.2-BETA.

To continue my list of actions, I also added the following line to "/boot/loader.conf.local" as stated here: https://doc.pfsense.org/index.php/Boot_Troubleshooting#Booting_from_USB

kern.cam.boot_delay="10000"

Though, I might want to try the following commands upon boot next time, unless someone has alternative suggestions:

unset acpi_load
set hint.acpi.0.disabled=1
set kern.cam.boot_delay="10000"
boot

stephenw10 · Jun 21, 2014, 5:45 PM

It's possible to load an alternative DSDT table at boot time so if you know where the error is you could correct it and load that instead. I have done it once a long while ago so I can't remember the details but I think you have to extract the table from the BIOS decompile it, alter it then recompile it. Then load it with a line in loader.conf.

Steve

charliem · Jun 22, 2014, 11:36 AM

@stephenw10:

It's possible to load an alternative DSDT table at boot time so if you know where the error is you could correct it and load that instead. I have done it once a long while ago so I can't remember the details but I think you have to extract the table from the BIOS decompile it, alter it then recompile it. Then load it with a line in loader.conf.

That process for FreeBSD is described here: https://forum.pfsense.org/index.php?topic=66601.msg366359#msg366359

I've used a similar process with a linux kernel, but haven't had a need to use it yet with FreeBSD. It's a little more complicated for pfSense as the required tools are not included, you have to use a FreeBSD image to generate the modified table.

stephenw10 · Jun 22, 2014, 4:31 PM

Yep, that's it. Re-reading though the error appears to br in the MADT table not the DSDT. I'm not really familiar enough to know whether this might be possible. I note it hasn't been suggested in other bug reports so perhaps not.

Steve

stig.voss · Jun 22, 2014, 5:00 PM

To report my findings from attempting the following.

@stig.voss:

unset acpi_load
set hint.acpi.0.disabled=1
set kern.cam.boot_delay="10000"
boot

I did not receive kernel panic upon using these settings, but another issue araised. pfSense continued nicely to initiate the 10 second wait for USB controller and devices to appear, but this failed. No warning upon the matter, just a failure to boot due to no mountable file system.

Concerning what Steve and Charlie is suggesting, I think it may be a bit over my level of competence. Unless I'd be able to obtain a somewhat detailed guide or set of instructions, I guess I'd be quite lost and end up spending more times worth than another piece of hardware would cost.

Unless there is an idea of why my pfSense does not find the USB device on boot, I guess this device may as well get another use and it's older version (DCCP847DYE, Sandy Bridge-based Celeron NUC), which I have gotten a confirmation on as functioning for pfSense, may be acquired for a pfSense installation.

Again, thank you for your assistance and if there would be an idea to make it recognize my USB, I would of course like to hear suggestions.

stig.voss · Dec 14, 2014, 12:24 PM

I am writing to inform that the system in question is booting using pfsense 2.2-RC, more specific, the following image: pfSense-2.2-RC-1g-i386-nanobsd-vga-20141213-1326.img

firewalluser · Dec 14, 2014, 2:54 PM

I have run into an issue concerning the ACPI. It seems from research that it is caused by a bogus ACPI table on the DN2820 NUC and FreeBSD*, which causes a kernel panic upon boot.

Is this problem related to just that model of NUC?

Its just I got a NUC Celeron 847 http://www.intel.co.uk/content/www/uk/en/nuc/nuc-kit-dccp847dye.html with 8GB ram + 128GB mSATA drive back in July 2013 and its been running very well with pfsense on it. I think back then it was pfsense 2.1 that was available at the time, but apart from the built in AMT back door into every Intel system its met expectations.

If need to compare HW settings drop a pm.

FWIW.

stig.voss · Dec 14, 2014, 4:07 PM

Yes, it only concerns the Bay Trail-based Celeron NUC. I also have the Celeron 847-based and a i3-4010U-based NUC running pfsense 2.1.5, both working perfectly.

Today I retested the Bay Trail-based NUC using the release candidate of 2.2 to verify if the issue had been resolved in the latest version, as it stated in the bug report that it would have been resolved in FreeBSD 10.1, and it does seem to be the case for pfsense 2.2 too as it's based on FreeBSD 10.1.

firewalluser · Dec 14, 2014, 4:31 PM

Thats handy to know, thanks for that!

mzarrugh · Dec 16, 2014, 4:16 PM

This is really awesome to know as I'm planning to setup a similar config on my home network.

Did the Realtek ethernet cause you problems with VLANs though or did it work smoothly so far?

stig.voss · Dec 17, 2014, 3:04 PM

@mzarrugh:

This is really awesome to know as I'm planning to setup a similar config on my home network.

Did the Realtek ethernet cause you problems with VLANs though or did it work smoothly so far?

I didn't actually take it that far as I didn't have any VLAN equipment at the location, I just installed pfsense 2.2-RC on the device. During the coming weekend, I should be at a site with VLAN equipment and I will try it for you.

Personally, upon encountering the issue I bought a replacement, namely the Celeron 847-based NUC mentioned earlier. Here in Denmark the DN2820FYKH and the DCCP847DYE are priced somewhat similar and I can confirm that the DCCP847DYE works flawlessly with both pfsense 2.1.5 and VLAN.

But I do very much see your point about the need for testing VLAN capabilities. I will return with an answer to this question after the coming weekend.

stig.voss · Dec 17, 2014, 6:18 PM

Plans change and I got access to VLAN equipment way earlier than expected. I have now tested the device using VLAN and it does seem to work as shown in the picture below.

https://drive.google.com/file/d/0Byf_N3Zxj7ytbnZmdFRqZlVpQXM/view?usp=sharing

mzarrugh · Dec 17, 2014, 6:35 PM

@stig.voss:

Plans change and I got access to VLAN equipment way earlier than expected. I have now tested the device using VLAN and it does seem to work as shown in the picture below.

https://drive.google.com/file/d/0Byf_N3Zxj7ytbnZmdFRqZlVpQXM/view?usp=sharing

Wow thanks much appreciated :D

stig.voss · Jan 5, 2015, 10:36 PM

This is not a request for help or an attempt to revive the thread for a new question. But I would like to inform potential users of this device a bit about my experience it.

Since around Christmas I attempted to implement this device as the firewall appliance in my home network. At first it did show some symptoms of being unstable under heavy load and upon that cause some "re0: watchdog timeout" errors. I left home for some weeks and let it run as my gateway and firewall. During the weeks with low activity on the network, it showed no signs of this problem. Now, as I got home and have started using my network again, the device is doing the same tricks again. From time to time after pushing a couple of megabytes per second on my connection or on VPN, it will crash the connection. When streaming it will crash the connection. A basic synchronization with Google Drive seems to be able to do it. I have yet not found a specific pattern in the issue. There might be a solution out there, but I have not experimented further.

To visualize my experience, I will provide a chart of ping logging.

Performance wise, ignoring the timeouts, it easily handles my 150/150 Mbps connection and it pushes some 90-110 Mbps on OpenVPN.

firewalluser · Jan 6, 2015, 12:51 PM

Who is your isp? I wonder if you are seeing problems simply by testing the speed limits. Was this with the I3 or Celeron 847?

stephenw10 · Jan 6, 2015, 1:39 PM

Have you tried the loader tunables available in the driver:
https://www.freebsd.org/cgi/man.cgi?query=re&apropos=0&sektion=0&manpath=FreeBSD+10.1-RELEASE&arch=default&format=html#LOADER_TUNABLES

I'd definitely try disabling msi/msi-x if you haven't already.

Steve

stig.voss · Jan 6, 2015, 5:16 PM

@firewalluser:

Who is your isp?

The Danish optic fibre provider named Syd Energi in cooperation with their ISP company named Stofa.

@firewalluser:

I wonder if you are seeing problems simply by testing the speed limits.

It seems to happen mostly during load at several megabytes per second.

@firewalluser:

Was this with the I3 or Celeron 847?

It is the Bay Trail-based Celeron version with Intel Celeron N2820, product link: http://www.intel.com/content/www/us/en/nuc/nuc-board-dn2820fykh.html.

@stephenw10:

Have you tried the loader tunables available in the driver:
https://www.freebsd.org/cgi/man.cgi?query=re&apropos=0&sektion=0&manpath=FreeBSD+10.1-RELEASE&arch=default&format=html#LOADER_TUNABLES

I'd definitely try disabling msi/msi-x if you haven't already.

Steve

I have tried disabling both MSI and MSI-X, but I am not sure that I did it properly as my commands seemed to concern PCI devices not Realtek-specific. I shall attempt this and report back.

Guest · Jan 6, 2015, 5:33 PM

Have you checked the log for apinger if it starts to panic with some load on the connection (Status - Systemlogs - Gateways)? Increasing the time for latency and package loss (System - Gateway - edit Gateway) might stop it from restarting interfaces/killing states on "gateway failure" (System - Advanced) due to apinger freaking out…