CARP & 3G failover
I'm thinking of the following setup for production use once 2.0 is stable:
1x incoming fibre
2x pfsense in CARP
2x 3g USB modem for failover
The functionality would be that if a pfsense box failed there would be autoamtic fail across to the other as per normal for CARP. But, what if the fibre failed? Would the primary PFsense use just its 3g modem or could it be load balanced on the other CARP member and its 3g modem? I presume if the fibre and primary PFsense failed then it would fail accross to the 3g on the second CARP member?
It can't be load balanced in that way across CARP members.
You also can't do proper CARP with two 3G modems in that way, since they can't share broadcast traffic or have a shared CARP IP on that interface.
If you want to do all of that, what you could do instead is make a small "3G Router" with both of your 3G cards in it, and use that for your second WAN. You could do CARP on that new segment (WAN2 on your CARP members, LAN on the "3G Router") and let the "3G Router" do load balancing between the 3G lines. That router wouldn't itself be redundant, but since it would only be acting in a secondary WAN capacity, it may be a risk you can live with.
You can non-statefully fail over that way, which is probably adequate. No way to retain states across 3G like that since there's no way two systems can share the same IP (aside from a separate 3G router device as Jim mentioned).