@binary_bandit I went with a solution roughly as explained by Mike here.. the advice came from elsewhere, but the comments were basically the same.
Have two sites both routable always, each with its own carp cluster (no pfsync across sites, not necessary for me), but only one is routed to at a time. I allow my upstream provider to route for me, but could do this myself later by enabling/disabling an IP at either site and have them route to that instead. Each site is completely independent and although they advertise 3 public ranges they both have their own native/local range of public ips too.
Really the concensus from everyone I've spoken to is to do this with switches and bgp not pfsense, which is a huge bottleneck - but it does work.