@jimp: There is no limit I know of BUT ESX does wonky things with the NICs after 4. They get interleaved in the probe order. So you end up with: 1 5 2 3 4 or 1 5 2 6 3 7 4 8 So you have to be careful to line up the detected NICs, MAC addresses, interface assignments in pfSense and so on. According to https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=198406#c9 this can be fixed by editing ethernetX.pciSlotNumber lines respectively the vmx file directly on the ESXi.host. Up to 8 vmxnet3 interfaces they should look like: ethernet0.pciSlotNumber = "160" ethernet1.pciSlotNumber = "1184" ethernet2.pciSlotNumber = "192" ethernet3.pciSlotNumber = "1216" ethernet4.pciSlotNumber = "224" ethernet5.pciSlotNumber = "1248" ethernet6.pciSlotNumber = "256" ethernet7.pciSlotNumber = "1280"

Yep I can do that.. plenty of spare.

You could use hardware passthrough for the WAN interface to the pfSense VM instance. Slightly more complicated to configure but that way the interface isn't even visible in ESXI networking, so less risk for administrative mistakes.

So your running traffic from physical to vm.. across the lagg.  How many physical, how many vms?  What algorithm did you select for the load balance.. Keep in mind that this can only be done for outgoing traffic. You will need to check the counters on the actual interfaces on the switch to see what kind of distribution your getting across the physical paths, etc.. Good luck..

@Georget27: That helped A lot.  Did you use PPPoE by accident to authenticate the WAN ? Glad to help. My ISP does not use PPPoE, but rather RFC 1483 via DHCP. "Authentication" is done by only allowing RA after DHCP solicit. I am one of the users who tested that feature while it was being developed. (Thanks again, marjohn56!) Good luck wiht pfsense. Welcome to the club.

Been running running pfsense for YEARS under esxi.. Never had a problem with it. Even before freebsd added native drivers for the vmx3 virtual nics, etc.  Before that under old vmware server version 1 and 2.. And even on virtualbox for a tinybit, etc. I would stay away from usb nics to be honest.  With vlans and smart switch you could get away with 1 nic on your esxi host.  But better the more nics you have available in your esxi host, etc.

RESOLVED***** Upgraded MS-Server 2012 to MS-Server2012R2 and everything now works as expected. I have attempted to update a few MS-Server2012 Hyper-V PFSense firewalls and all have failed. I will now update all HOST OS to 2012 R2.

https://doc.pfsense.org/index.php/Connectivity_Troubleshooting It could be lots of things.

UPDATE I upgraded from 2.3.2 to 2.3.4 and I still have vtnet interfaces (and full speed). Now I am trying to go up to 2.4.1 and check if it still works. I will keep you posted (if someone will ever read this post  ;) ) EDIT: Upgrading to 2.4.1 kept the virtnet driver. So it seems the issue is present only when starting from 2.4 ISO… meh. EDIT2: Found the culprit: when I create a VM on cloudstack I can choose the "OS Type" and I choose a generic "FreeBSD (64 bit)". The 2.3.2 instead was created with OS Type = FreeBSD 10 (64 bit). It seems that does the difference... two days lost, lesson learnt :)

Anybody have any ideas? Or can anyone even confirm the behavior I'm seeing is expected? (i.e. tcpdump -i <int>host <carp-neighbor>– and look for ping/response to from that member on the primary firewall) I've gone as far as to damn near tear out my entire networking stack in esxi... Still the problem persists. I've upgraded to the latest patch levels, VM hardware levels, etc... I've tried manually setting MAC Addresses. Nothing working.  My next step is to try swapping E100E interfaces for VMXNET3 ones... But I'd love to at least know if I'm completely wasting my time. EXTMember1      EXTMember2         \                      /           \                  /               \            /                 \        /                     VIP                     |                     |                     VIP                   /                      /                        /                          / <------    \        -------- Traffic from INT Member 2 to EXT Gateway is seen on this interface INT Member1        INT Member 2</carp-neighbor></int>

Your pfSense has internet access? You can verify this at the dashboard in the Version section. If it is able to search for updates, it has access to the internet. If pfSense has internet access, do your LAN rules also allow access to the clients?

Yes! I even reinstalled both of the hosts (and their datastores)! Even on 2.4.0 the issue still persists!

@heper: I see no need to automatically add packages. Those who want to use it, can install it from GUI I agree. If someone needs/wants it, it only takes a few clicks to install. Someone may not have connectivity to the Internet when they spin up a VM so automatically fetching a package could fail.