As always is the case, I resolved this minutes after posting.
It's quite an obscure setting but I needed to also enable Security --> Forged Transmits in the vSwitch.
That is a layer 2 issue. Either that NIC is not passed through to pfSense correctly or the ONT is rejecting the MAC address. Rebooting would normally reset that bit not always.
Try some other device using the public IP directly. If that also fails and the Netgear is the only thing that works you will need to spoof the MAC address or call the ISP and have them reset it.