[Solved] pfSense doesn't see passed-through NIC
-
Hello all! I've been having issues getting virtualized pfSense 2.3.4 to see a NIC I'ved passed through to it, and I'm hoping someone with more experience with PCIe pass-through might be able to help me.
I am trying to pass-through the on-board NIC of an R410 (BMC5716) to a pfSense guest running on Hyper-V Server 2016.
However, pfSense does not see the NIC at all. Neither port on the NIC are enumated in ifconfig or pciconf, and nothing in dmesg stands out. I compared the dmesg output to another pfSense VM running on Hyer-V 2016 (but with virtualized NICs) and nothing stands out. (Logs linked at the end.)
I've also tried:
-
A FreeBSD 10.3 LiveCD => No NIC
-
pfSense 2.4 RC (Rescue shell) => No NIC
-
An Ubuntu LiveCD => The NIC is present
-
Booting in verbose mode (-v) to get more detailed dmesg logs => Seemingly nothing relevant
The fact that Ubuntu sees the NIC, but not pfSense/BSD leads me to believe this is probably a FreeBSD issue. One thing that did stand out on Ubuntu, the NIC's ports got some pretty crazy names (enP16070p0s0f0 / enP16092p0s0f0), but maybe that is normal. The PCI bus number is also seemingly weird (pci@3ec6:00:00.0), but that could be normal too.
pfSense used to run on this exact hardware on the bare metal, so I know the NIC works with pfSense.
Does anyone have any suggestions for additional things to check before I throw in the towel? All of my FreeBSD experience comes from using pfSense, so I may be missing something obvious.
Here's logs and extra info that may be relevant:
-
List of PowerShell commands I ran on the host to set up the pass-through (From my notes, but it should be complete.)
-
dmesg output from the pfSense guest that can't see the NIC (hn0 is a temporary virtual switch I added to get the output)
-
dmesg output from a different pfSense guest that uses virtual NICs instead (for comparison. Different host, so different CPU.)
-
Verbose dmesg output from pfSense guest that can't see the NIC (With the -v parameter at boot)
-
-
I finally figured out what the issue was. It turns out that PCIe pass-through in Hyper-V requires guest support. Guest support for FreeBSD was added in 11.1. I foolishly assumed that pfSense 2.4-rc was using the latest FreeBSD so I never tested FreeBSD 11.1 until now. I did, and it works. So it looks like I am waiting for pfSense 2.4.1 for this feature.
If you want to check for yourself, here's the code reviews that were done for adding Hyper-V DDA support to FreeBSD:
-
https://reviews.freebsd.org/D8332
-
https://reviews.freebsd.org/D8410
-
https://reviews.freebsd.org/D8409
If you compare the dates on the reviews to the FreeBSD release dates, you'll notice that they all happened after 11.0 was released.
Oh well, at least I can rest easy knowing it wasn't a weird configuration issue!
-
-
I have same issue with Hyper-V 2016 and PFSense 2.4 (based on Freebsd 11.1), do you resolve it?
-
pfSense 2.4.0-RELEASE is based on FreeBSD 11.1
-
I know it, but passed through NICs didn't work.
-
Then it's not a FreeBSD 11.1 issue.
-
I have tried Ubuntu 17.04 on same VM configuration and it's works perfect, so - it's Freebsd 11.1 issue. :(
-
Hey Gektor,
I finally got a chance to try this again in my lab environment. I can confirm that pfSense 2.4 release properly detects and can use the passed-through NIC for me. However, I will note that things are a little wonky, I suspect the pass-through is confusing the Broadcom driver, as it doesn't put the interfaces up on boot. (However, manually putting them up and running dhclient worked for me.) I did not test very extensively, so I imagine they won't come up when booting after install either (I did my testing from the recovery shell.)
Looking at your screenshot, it looks like you're having some similar driver problems (which is different from my original issue.) Your Intel NIC is seen by pfSense, but it fails to initialize properly. Do they appear in your ifconfig output? If so, you might manually putting them up manually (ifconfig <device-name>up) and configure them (dhclient <device-name>) to see if they work. However, based on the fact that it kept using en0 for each port, I suspect they won't appear.
My gut says that the PCIe passthrough is confusing the Intel driver. Hopefully it is something the FreeBSD developers are aware of and it will be fixed in FreeBSD 11.2. I don't have enough time to investigate my specific issue further. Since you have an error in your logs, you might be able to find out more about the root cause of yours.
(EDIT: Removed brain fart involving FreeBSD versions.)</device-name></device-name>
