It's fixed! 2 things fixed it (in my opinion).
Removing the modem/router box from my ISP in the middle. Today my ISP provided a media converter for glass fibre to RJ45 so I don't need their modem router anymore. Even though I had PPPoE Passthrough enabled on their modem, I have a feeling it was somehow blocking my own router getting through.
Setting trunking on the VM switch port/group. I would never have thought of that! Thank you so much
It's now working beautifully. 500Mb down and 750Mb upload with an MTU to google of 1500.
Thanks so much.
Matt
PS: if anyone is interested in my settings I can try to write them down, just reply here and let me know if there's interest in it. Running pfSense on Esxi VM directly via F3100 media converter to xs4all glass fibre