Installation trouble on XEN

  • Hi Guys, first post here so be kind!

    I’m trying to install pfSense as a HVM guest on Xen, but I haven’t had any success so far.

    I’ve got the hypervisor installed, with Alpine Linux as my Dom0. No GUI - everything’s console based, but I do use a graphics card to give a VGA connection to view the console.

    All that’s working just fine so far.

    The problems start when I try to install pfSense as a HVM. I’ve tried both the CD ISO version and the usbstick serial version. With the serial version, I can see the system booting to a certain point, then the serial connection just disconnects. With the ISO version, I can’t even get the thing to boot.

    Here’s the Xen configuration I’ve been using:

    builder = "hvm"
    name = "pfsense"
    memory = 2048
    maxmem = 2048
    vcpus = 2
    acpi = 1
    apic = 1
    vif = [ 'bridge=br0, mac=00:3e:16:02:02:99' ]
    disk = [ 'phy:/dev/vg0/pfsense,xvdb,w', 'phy:/dev/sdk,xvda,w' ]
    boot = 'c'
    sdl = 0
    vnc = 0
    nographics = 1
    serial = 'pty'
    on_poweroff = 'destroy'
    on_reboot = 'destroy'
    on_crash = 'destroy'

    By watching the PTY interface with Screen, I can see the system start, load the kernel, and start identifying hardware devices, but it gets no further than that. After a while, the PTY device just disappears, which is weird because the VM continues to run - I just can’t access it. Any ideas what’s going on?

    I’ve tried sending the serial to TCP, too. When I do this, I can see it booting up from PuTTY on another computer, but it still cuts off in the same place.

    Please help!
    Many thanks.

  • And did you read the STICKY on XEN and pfsense?

    IMPORTANT: Xen/KVM networking will not work using default hypervisor settings!

  • Thanks for the reply.

    I did read the sticky. Unfortunately that’s not the issue I’m having.

    My problem is not to do with ethernet but rather the serial console cutting off during boot. I haven’t even got to the stage of configuring ethernet or pinging interfaces yet. I’m just trying to get through the installer using the serial console via a pty.

  • Ok, so today I added```

    It looks to me like a kernel panic. Is that what it is? If so, what could be causing it?

  • bump

    Please? Any hints as to what I’m doing wrong here?

  • You should give more hardware details. Also if you use hvm you cannot use console and you need to use vnc to view it.

  • Thanks for your reply.

    The server is based on the SuperMicro X10SAE motherboard, with an Intel Xeon E3 CPU. 16GB DDR3 ECC RAM @ 1600MHz. It has Vt-x and VT-D if that’s what you’re wondering.


    if you use hvm you cannot use console and you need to use vnc to view it.

    That’s not actually true. You’re right that you can’t use Xen’s built in “xl console” command, but you don’t have to resort to vnc. If your VM produces serial output (as pfSense does), you can have it output that serial interface to a virtual pty port. To do this, just include the line

    serial = 'pty'

    in your Xen HVM config. This will cause the machine’s serial ports to be re-directed to files under /dev/pts on the host machine, which you can then view and interact with using your favourite terminal emulator. I use GNU Screen for this purpose, so to see the output from my pfSense HVM, I issue:

    screen /dev/pts/1 115200

    That’s actually how I got the screenshot in my second post. The screenshot is of a BSD kernel panic that happened when I ran my pfSense HVM under Xen, and viewed it using Screen via the pty.

  • This error usually reside with your cpu not having vt-d enabled. The bios has nothing to do with cpu. Make sure your cpu has it. I never used intel for virtualization as I mainly use amd for all my servers.

    Why not try using xenserver and see if it has problems first

  • Hmm,

    The processor is an Intel Xeon E3-1230 v3 Haswell, which supports VT-d according to this page:

    Is there something specific I need to do to enable it?

  • I don’t think its your hardware problem maybe the way you configured your OS. I still suggest using xenserver will make your life way easier. Also you still can do pci pass thru as there are lots of tutorials. I have been running xen for many years and didn’t have much problems. I usually used Debian for the base and xen. Most of the cpus I used are AMD 8350 with 16/32GB ram and raid 1. I pci pass thru 1 nic for wan to pfsense. Other vms i use are zentyal and consolidate the windows servers they have into xen. Now I mainly use freenas for all storage and iSCSI and xenserver for virtualization. The xenserver boots of a raid1 with pfsense in it and all other vm runs from iSCSI.

  • Well… I’ve found the problem, and boy do I feel stupid!  :-[

    Turns out I hadn’t allocated enough RAM to my Dom0. Whenever I launched the pfSense HVM, the Dom0 would run out of RAM and crash xl.

    Changed the allocation on the boot line from 256M to 2G, and it works beautifully.  😎


© Copyright 2002 - 2018 Rubicon Communications, LLC | Privacy Policy