For anyone finding this in future, I am glad to report that my router upgrades were successful (thank you pfSense team and contributors).
I did have to go from 2.4.4_3 to 2.4.5_1 first (on both the 7100 and the 3100). After the upgrade, I did not touch any of the plugins, changed the update branch to latest and rebooted again (just to be sure).
Then I was able to go from 2.4.5_1 straight to 21.05 and it even then upgraded all the plugins to the latest for me.
It did take me several hours because I had two routers to do, I had CARP failover and IPSec tunnels and I waited to make sure none of my alerting systems went crazy. But the actual upgrade process was suprisingly and refreshingly straight forward without any drama.