What you are wanting looks possible in apinger, which really just deals with monitor IPs. pfSense has "Gateway Groups" that consist of a set of Gateways with priorities (tiers). But really it seems to me that they are really "Monitor IP Groups". It would be possible to have multiple monitor IPs for each gateway. Each Monitor IP would potentially have its own advanced settings (loss, delay, down time…). The existing "Gateway Groups" could actually become a selection of the "Monitor IP entries", with a tier for each one. apinger reports which target (monitor) IP has a state change, pfSense can use this in a more refined way than now - passing that up to the various "service reload" commands that react to apinger alarms.
Then you can have different Gateway Groups that are prioritised on different sets of monitor IPs (although ultimately underneath the traffic is on the same gateways/interfaces). Particular OpenVPN instances, or policy-routing rules can then use a particular Gateway Group that responds as required to the failure of particular monitor IPs.
I can think of a use for this here in Nepal - sometimes links to the "rest of the world" internet go down, but our internal national ISP/s are working OK, so my VPN links between offices in the country will still work. In that case, I don't want to try to failover the VPN links to some slow backup link. On my main WAN gateway I could monitor an in-country ISP address, and use that in a Gateway Group for VPN failover. On my main WAN gateway I could also monitor an outside-Nepal IP, and use that in a gateway group for general policy-based routing of browser traffic. I could monitor my international mail server IP, and use that in a gateway group for policy-based routing of traffic to the mail server. The failover could detect (with a reasonable guess) what bit of the internet is unreachable on the main WAN, and just failover that bit to the backup link.
This is about having multiple networks/services/resources available over a single gateway, and detecting which particular network/service/resource is now unreachable, and allowing the firewall rules, VPN settings... to just failover the things that need to reach that network/service/resource.
How many people have a need for this?
And who has the time to code and test it?