Installation trouble on XEN

lloydsmart

Hi Guys, first post here so be kind!

I'm trying to install pfSense as a HVM guest on Xen, but I haven't had any success so far.

I've got the hypervisor installed, with Alpine Linux as my Dom0. No GUI - everything's console based, but I do use a graphics card to give a VGA connection to view the console.

All that's working just fine so far.

The problems start when I try to install pfSense as a HVM. I've tried both the CD ISO version and the usbstick serial version. With the serial version, I can see the system booting to a certain point, then the serial connection just disconnects. With the ISO version, I can't even get the thing to boot.

Here's the Xen configuration I've been using:

builder = "hvm"
name = "pfsense"
memory = 2048
maxmem = 2048
vcpus = 2
acpi = 1
apic = 1

vif = [ 'bridge=br0, mac=00:3e:16:02:02:99' ]

disk = [ 'phy:/dev/vg0/pfsense,xvdb,w', 'phy:/dev/sdk,xvda,w' ]
boot = 'c'

sdl = 0
vnc = 0
nographics = 1
serial = 'pty'

on_poweroff = 'destroy'
on_reboot = 'destroy'
on_crash = 'destroy'

By watching the PTY interface with Screen, I can see the system start, load the kernel, and start identifying hardware devices, but it gets no further than that. After a while, the PTY device just disappears, which is weird because the VM continues to run - I just can't access it. Any ideas what's going on?

I've tried sending the serial to TCP, too. When I do this, I can see it booting up from PuTTY on another computer, but it still cuts off in the same place.

Please help!
Many thanks.

johnpoz

And did you read the STICKY on XEN and pfsense?

https://forum.pfsense.org/index.php?topic=88467.0

IMPORTANT: Xen/KVM networking will not work using default hypervisor settings!

lloydsmart

Thanks for the reply.

I did read the sticky. Unfortunately that's not the issue I'm having.

My problem is not to do with ethernet but rather the serial console cutting off during boot. I haven't even got to the stage of configuring ethernet or pinging interfaces yet. I'm just trying to get through the installer using the serial console via a pty.

lloydsmart

Ok, so today I added```
xen_platform_pci=0


![](https://i.imgur.com/lrMlwFb.jpg)

It looks to me like a kernel panic. Is that what it is? If so, what could be causing it?

lloydsmart

bump

Please? Any hints as to what I'm doing wrong here?

cougarmaster

You should give more hardware details. Also if you use hvm you cannot use console and you need to use vnc to view it.

lloydsmart

Thanks for your reply.

The server is based on the SuperMicro X10SAE motherboard, with an Intel Xeon E3 CPU. 16GB DDR3 ECC RAM @ 1600MHz. It has Vt-x and VT-D if that's what you're wondering.

@cougarmaster:

if you use hvm you cannot use console and you need to use vnc to view it.

That's not actually true. You're right that you can't use Xen's built in "xl console" command, but you don't have to resort to vnc. If your VM produces serial output (as pfSense does), you can have it output that serial interface to a virtual pty port. To do this, just include the line

serial = 'pty'

in your Xen HVM config. This will cause the machine's serial ports to be re-directed to files under /dev/pts on the host machine, which you can then view and interact with using your favourite terminal emulator. I use GNU Screen for this purpose, so to see the output from my pfSense HVM, I issue:

screen /dev/pts/1 115200

That's actually how I got the screenshot in my second post. The screenshot is of a BSD kernel panic that happened when I ran my pfSense HVM under Xen, and viewed it using Screen via the pty.

cougarmaster

This error usually reside with your cpu not having vt-d enabled. The bios has nothing to do with cpu. Make sure your cpu has it. I never used intel for virtualization as I mainly use amd for all my servers.

Why not try using xenserver and see if it has problems first

lloydsmart

Hmm,

The processor is an Intel Xeon E3-1230 v3 Haswell, which supports VT-d according to this page: http://ark.intel.com/products/75054/Intel-Xeon-Processor-E3-1230-v3-8M-Cache-3_30-GHz.

Is there something specific I need to do to enable it?

cougarmaster

I don't think its your hardware problem maybe the way you configured your OS. I still suggest using xenserver will make your life way easier. Also you still can do pci pass thru as there are lots of tutorials. I have been running xen for many years and didn't have much problems. I usually used Debian for the base and xen. Most of the cpus I used are AMD 8350 with 16/32GB ram and raid 1. I pci pass thru 1 nic for wan to pfsense. Other vms i use are zentyal and consolidate the windows servers they have into xen. Now I mainly use freenas for all storage and iSCSI and xenserver for virtualization. The xenserver boots of a raid1 with pfsense in it and all other vm runs from iSCSI.

lloydsmart

Well.. I've found the problem, and boy do I feel stupid! :-[

Turns out I hadn't allocated enough RAM to my Dom0. Whenever I launched the pfSense HVM, the Dom0 would run out of RAM and crash xl.

Changed the allocation on the boot line from 256M to 2G, and it works beautifully. 8)