Well, I'm no expert but I got it working on my end, esxi 6.7, but using layer 2 'lite' layer 3 switches from Netgear though. I kept the management network on my default LAN, thinking that if there were issues with the VLAN, I want to be able to reach the management network without a fuss. I put two nics on that original vswitch, for the management network, and put the rest on a new port group on a second vswitch that I created for the VLAN. Put it in VLAN 4095 (will probably move it to the right VLAN at some point- 4095 means all VLANS). I then set the Netgear switch ports that came from the VMs in ESXI to the correct VLAN and it's working.
I don't know why it drops, maybe more information as to how it is set? One vswitch with everything in it (management/vlans all in the same vlan) or multiple vswitches..., plus I don't know how your physical switch handles the vlans, I never tried TP Link with vlans before.