Hi jimp,
some good news here.
Your statement that "pkg can't reach the pfsense servers" pointed me to the right direction; I haven't understood it fully, but I found a way at least to unlock the pkg issue.
In my case, it was due to a double stack IPv4/IPv6 issue: to solve it, I had to temporaly disable the network interface linked to the GIF port; removing IPv6 name resolution plus removing the IPv6 default gateway and firewall rules to route IPv6 traffic didn't suffice.
I don't like to be so inaccurate in test results, but as IPv6 connectivity was actually working, defining this problem will require some more tests and I meant to find a quick workaround for everybody experiencing this kind of issue.
Let me know if this rings a bell.