Thank you for posting this, it's extremely helpful. I'm hopeful changes in 2.4 will benefit virtio performance? I'm not in a position where I can just pass through a nic dedicated to pfsense and so am at the mercy of virtio.

1. I need to setup the 3 physical NICS (LAN, WAN, MGT or maybe DMZ) with static IPs You only need virtual switching.  Just add as many network adapters as youd like through hyper-v manager and your vm settings. 2. IP address(s) for virtual switch(s) and what types (ext, int or private?) for outbound(wan) traffic, use an external switch and create at least one external network adapter for your pfsense vm.  You don't have to share this with management os, but take not that your management os wont have access to it.  in your pfsense configureation, this will be assigned an ip either by dhcp or staticely to match the external network.  create an internal switch for all other vms and even your host.  create adapters for all your vms and configure vms with pfsense internal ip address as default gateway. 3. IP addresses within pfsense (LAN, WAN etc). pretty much answer to 2. 4. Endstate:  I have a WAN link with firewall rules applied and isolated from everything else, LAN link for filtered internet access and a LINK for management of pfsense (web interface and isolated to a workstation only). I would suggest keeping it in an isolated environment until you are comfortable with it.  Then when you are sure of your abilities to manage it, put it into production.

heper - it's [iperf client] <-> [pfsense VM] <-> [iperf server] that all sit on the same switch; "iperf client", "pfsense VM" and "iperf server" are each on their own hardware. I don't this would be considered as multiple L3 setup right?

Been through all those suggestions but I appreciate the responses. I currently have 4 cores assigned, none seem to pin even under heavy load. The I350 nic settings I've left at their defaults. With the Broadcom's I had VMQ's disabled. The IPSec offload is enabled on the virtual NIC's. I'm starting to think that some how the Layer3 configuration is playing a role in the issue. I'm going to do a bit more research and follow up.

Hello heper. I have looked on status/system log/gateway and there are messages like these: Feb 9 13:24:48 dpinger WANGW 192.168.9.254: sendto error: 64 The 192.168.9.254 is the ISP's modem address. On the other hand, this logs are real-time? the last message has date Feb 9 13:24:48, however, if this error message is related with my trouble, should there be messages from today's date? the pfSense firewall is currently operating and failing at every moment. For the rest,  I have not found anything relevant in other logs options. Thank you. Luis

so my emX interfaces will not be changed in vmxX ? No, of course not.

Does anyone know what the underlying issue of this was? Or if it is going to be resolved? I had 2.3.1 in Hyper-V on 2012R2 experiencing HEAVY packet loss when approaching 5mbps on our MPLS. Once I downgraded to 2.2.6 everything was fine again. I couldn't find any bug referencing this issue. I'm glad to find this problem is more widespread then just me.

Tenacity prevails! If you assign the bridge to the LAN interface and add both your previous lan interface and the tap0 interface to it, then you can attain access to the internets for your vms. 1. Create a bridge in Interfaces:(assign):Bridges 2. Go to Interfaces:(assign) and determine which device is assigned to LAN 3. Go to Disgnostics:Command Prompt 4. Enter ifconfig bridge0 addm <device assigned="" to="" lan="">5. Go back to Interfaces:(assign) and set BRIDGE0 as the LAN 6. Add the previous LAN device 7. Enable that device 8. Go to Bridges:BRIDGE0 and assign your new device (OPT1?) to BRIDGE0 When you add the device to the bridge with the command prompt, you make certain that the bridge has access to the wire.  When you add the device to the bridge in the GUI, you make that persist between reboots. It works!  If anybody want to use Netflix with a Hurricane Electric IPv6 Tunnel you can use a VM to provide a barebones BIND install to filter out Netflix's IPv6 addresses. If someone besides me shows interest in this thread, I could make the instructions more cohesive  ;D, but for now I'll leave this as is and hope it's useful to anyone else who wants to do this.</device>

Modern OSes like Windows Server boot in 5 seconds VMware Workstation 12.5.2, running on an Intel i7-2600, takes about 15 seconds to boot a fully-patched Windows Server 2012R2 VM stored on an SSD.

its solved.. i just change another physical NIC and plug the trunk interface into new NIC. set encapsulation dot1q on cisco switch, i've tried (it works on 2900 & 3550) it doesnt work on 2950,2960(may be im wrong config). old one NIC doesnt support for trunking. Thanks.

As long as you set up the vswitch so that only the pfsense box has a LAN port on it, and its running to a dedicated esxi NIC your fine. That's not as uncommon as you think. I run into that all the time when I work on networking chassis or firewalls, anytime I change out a line card or module it regenerates the ssh keys when it restarts.

The problem is with the ESXi. I can put an ESXi management interface on the LAN and add a NAT, however I can't change the default gateway, it's a global configuration to all VMKernels. And so the packages are unable to get out!

Well I didn't bother to wait for an answer here… not sure why it failed after the update, maybe because it's been updated so many times from previous versions something was messed up. I downloaded 2.3.2 ISO, did a fresh installation, then updated to patch 1 and loaded my backup everything is working great. So I guess case closed. Cheers