Are you planning on doing upwards of 50K Packets Per second? If so ok then maybe you need to save some cpu cycles and throw out all the benefits of just letting esxi handle the nics and give your vms virtual nics.
To be honest if you were setting up a network that would come close to moving that many packets over a vm infrastruce you sure an the hell would not be buying your nics on ebay for $40 nor would you be here asking questions ;) If you were you got promoted into the wrong job or lied like crazy on your resume and not going to last in your position for more than a couple of weeks ;) hehehe
No I don't, this is just for my home network. I've also never really worked with ESXi before that much, so I'm pretty much a newbie mostly. I've used Proxmox before but when I decided to set up my own router with pfSense I thought I'd switch to ESXi because I thought a baremetal hypervisor would feel better, which it does, no more annoying kernel updates.
There is obviously a lot I have to learn now. ;D But I'm doing it to learn something and it's a fun hobby for me. I'm not employed as network admin of course, I'm just a software developer. :D
As to more nics - its is good thing. Well worth a $40 cost.. I would suggest you break out your vmkern to its own nic. This will for sure give you a performance increase moving files to and from your datastore from your physical network.
Having more nics allow you to create more networks and not have to vlan and hairpin connections which allows you better performance, etc. I have 4 physical nics in mine, I would much rather have a couple of more. You see my wlan vswitch is doing vlan tagging all sharing the same physical nic. If I had more could break those out to their own physical nics.. Prob not all that big of deal because that traffic is mostly wifi. But makes for simpler setup and for sure no hairpins, etc.
Yeah I've also thought about using the NIC for more networks, I wanna separate my WiFi and the devices in my living room, so I'll do that. For now I've disabled DirectPath for all the NICs and I'm now using a similar configuration like yours, it works and the speed is fine so I'm happy with that result, thanks for your help and patience! :)