Hello,
Did you had success install on Softlayer/Bluemix env?

best regards,
Carlos

Derelict your Xen NIC step seems to have fixed the issue. It looks like it is working perfectly now, thank you.

Thank you as well KOM for you assistance.

If your hypervisor crashes, that is 100% a problem with your hypervisor or hardware. A problem in the guest OS should never cause the hypervisor to crash.

Thanks very much for the reply.  I did that and it's working fine.  Very happy with how this is running as a VM.  Much better than when I tried the x86 version on 2008R2 with its legacy adapters and all. Seamless install and config restored from my old 32bit box and everything works great.

What type of throughput did you see on this setup? I'm looking at doing something similar.

You are suggesting to migrate to another virtualization platform just in order to get some disk space?
There must be an easier solution, right?

Edit:
Of course there is. Removed the disk in KVM guest, removed the underlying lvm logical volume and started over from scratch.
Works now very well. Used this howto:
https://www.freebsd.org/doc/handbook/disks-adding.html

Hi.
In a past I'm using Xen as hypervisor too. But Proxmox VE (kvm hypervisor) more and more better.

P.s. Even Amazon migrate to KVM.

@jimp:

There is no limit I know of BUT ESX does wonky things with the NICs after 4. They get interleaved in the probe order.

So you end up with:

1
5
2
3
4

or

1
5
2
6
3
7
4
8

So you have to be careful to line up the detected NICs, MAC addresses, interface assignments in pfSense and so on.

According to https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=198406#c9 this can be fixed by editing ethernetX.pciSlotNumber lines respectively the vmx file directly on the ESXi.host.
Up to 8 vmxnet3 interfaces they should look like:

ethernet0.pciSlotNumber = "160" ethernet1.pciSlotNumber = "1184" ethernet2.pciSlotNumber = "192" ethernet3.pciSlotNumber = "1216" ethernet4.pciSlotNumber = "224" ethernet5.pciSlotNumber = "1248" ethernet6.pciSlotNumber = "256" ethernet7.pciSlotNumber = "1280"

Yep I can do that.. plenty of spare.

You could use hardware passthrough for the WAN interface to the pfSense VM instance.

Slightly more complicated to configure but that way the interface isn't even visible in ESXI networking, so less risk for administrative mistakes.

So your running traffic from physical to vm.. across the lagg.  How many physical, how many vms?  What algorithm did you select for the load balance.. Keep in mind that this can only be done for outgoing traffic.

You will need to check the counters on the actual interfaces on the switch to see what kind of distribution your getting across the physical paths, etc..

Good luck..

@Georget27:

That helped A lot.  Did you use PPPoE by accident to authenticate the WAN ?

Glad to help. My ISP does not use PPPoE, but rather RFC 1483 via DHCP. "Authentication" is done by only allowing RA after DHCP solicit. I am one of the users who tested that feature while it was being developed. (Thanks again, marjohn56!) Good luck wiht pfsense. Welcome to the club.

Been running running pfsense for YEARS under esxi.. Never had a problem with it. Even before freebsd added native drivers for the vmx3 virtual nics, etc.  Before that under old vmware server version 1 and 2.. And even on virtualbox for a tinybit, etc.

I would stay away from usb nics to be honest.  With vlans and smart switch you could get away with 1 nic on your esxi host.  But better the more nics you have available in your esxi host, etc.