My first routed ipsec environment, tunnels keep failing

[jhorne]

in my vm lab envionment, i set up 5 pfsense 2.4.4s. 4 of them are connected with IPsec on their WANs (FW1-4), with the 5th connected to FW2 via a direct connected interface and static routes (this mimics a live setup i run in production with traditional phase 2s). the setup is non-mesh with site 4 and 1 connecting thru 2-3.

initially, everything works as expected, all sites can ping all sites. but after a couple hours, something happens (a rekey or whatever) and like clockwork all the tunnels drop and dont reconnect. im not sure what im missing here, can anyone give me some pointers?