@kom OK, found the issue, it was basically this: https://forum.netgate.com/topic/82412/pia-openvpn-gateway-offline
the solution was to go into System / Routing / Gateways, and to set the Monitor IP in the VPN gateway to an IP that accepts pings (or to turn off gateway monitoring). Then the status of the gateway switches to online. Then my PC connects to the internet through the VPN.
I just don't understand why the same problem didn't occur on my private switch setup. Perhaps because it is an earlier version of pfsense (2.4.4-p2)