iperf3 issue for IPsec VTI between two sites

[kevindd992002]

Site A WAN = has a public static IP and is acting as responder only; has 400Mbps down/400Mbps up connection
Site B WAN = has a private dynamic IP and is behind a CGNAT; it acts as the initiator; has 100Mbps down/100Mbps up connection

When I do an iperf3 test on any device from Site B to any device on Site A (sending data from B to A), I get the full 100Mbps bandwidth. When I do Site A to Site B, I get only an average of around 27Mbps!

Both use the same ISP and they get their full speed in speedtest.net. To further complicate things, when I download a file using SMB from either side (from Site A to Site B / from Site B to Site A) I get the full 100Mbps bandwidth! So there's something weird going on with iperf for sure. It seems that the issue is non-existent in the real world but I'm still curious why iperf says otherwise. Any ideas? Does it have to do something with Site A being set to "responder only"?

EDIT: If I run the same test with "-P 8", I do get the full expected bandwidth so there's that. It seems that running iperf3 with one connection from Site A to Site B is not fully utilizing the available bandwidth but, again, why? I have pretty much the same settings on these two boxes.

[kevindd992002]

@stephenw10 any ideas?