If you followed the guide (Tx checksuming of for the NICs, guest tools installed, etc), the VM is hardware virtualised (HVM), same CPU and RAM parameter as the one of ESXi and the XCP-NG host is reasonably fast then I can't see a reason why you have that limit.
I ran pfsense (plus 3 other VMs) with XCP-NG on a J3455 CPU with 16GB RAM on a 400/90 MBits PPPoE connection. That was pretty much the limit but your can be done with even quite slow hardware
What hardware to you use as a host. And what is the output of top -HaSP while you do the speedtest. I assume you run the speedtest from a client (client -> pfsense -> ISP -> speedtest server), not from pfsense itself.
And again, a network diagram would help. How are you connecting to the ISP?