PfSense within FreeBSD Jail - It still kernel panics, so don't do it.

[Thrae]

After doing more research, I've found why no one seems to talk about using pfSense within a FreeBSD jail over using it within a Type-2 Hypervisor:

https://bugs.freebsd.org/bugzilla/buglist.cgi?bug_id=143621%2C%20176112%2C%20161094%2C%20176992%2C%20143808%2C%20148155%2C%20165252%2C%20178480%2C%20178482
(Thanks to: http://www.a1poweruser.com/35.00-Jails_guide_article.php#16.11%20Vnet/Vimage)

Basically, there are lots of nasty kernel-panicking bugs, so in short: don't do it (yet).

In fact, even trying to run pfctl commands will kernel panic the entire system: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=188018

You can view all applicable bugs here: https://bugs.freebsd.org/bugzilla/buglist.cgi?email1=virtualization&emailtype1=substring&f1=short_desc&f2=short_desc&f3=resolution&f4=short_desc&o1=anywordssubstr&o2=anywordssubstr&o3=nowordssubstr&query_format=advanced&v1=pf%20ipfw&v2=vnet%20vimage%20jail&v3=FIXED

Hopefully posting this as a reference will save someone at least a few minutes and avoid a possibly disastrous situation.

~~On my home server I thought I'd give server consolidation a try, since I have a Core i7 (Nehalem) with lots of extra power for my needs, but no ULP hardware lying around; and my install of pfSense onto a Netburst Celeron has seen better days (PSU just died and it's running a spare).

So I was looking through the pfSense code and doing some research on various virtualization solutions to get pfSense running on my FreeBSD server. And a thought many have had – but oddly not many have voiced on these forums -- is why can't we put pfSense in a jail again? If I compile my FreeBSD kernel including the same modules and patches as pfSense, I should have a kernel that includes all the features it needs. And if I'm worried about pfSense behaving badly and using up all the host system's resources, there are new resource controls added into 9 and 10.

I've also read about problems using certain plugins within pfSense not working right under Type-2 hypervisors (like squid).~~

[cmb]

A jail isn't viable for a full-blown OS. Our kernel isn't the same as FreeBSD's. It's not very practical to run our kernel on a stock FreeBSD. Bhyve is a vastly better solution than jails for what you're looking to do.

[Thrae]

@cmb:

A jail isn't viable for a full-blown OS. Our kernel isn't the same as FreeBSD's. It's not very practical to run our kernel on a stock FreeBSD. Bhyve is a vastly better solution than jails for what you're looking to do.

Except Bhyve doesn't support early Nehalem's implementation of EPT (also on some Atom processors) and has a low priority for a fix. (I did try Bhyve with various workarounds, but couldn't get it to stop kernel panicking even before I did anything with it.)