I woke up this morning (afternoon actually) after beating my head against the wall last night and tunnels were working…
Turns out that raccoon crashed (there was a core dump in the root directory, which I didn't even think about and deleted), which most likely corrupted the IPSec state entries. Normally rebooting would have fixed this, however since I had pfSync on, the two boxes just passed the bad entries back and forth... :)
Had I thought to reset the state tables, it probably would have started working immediately. Luckily the IPSec timer was only 6 hours so after sleeping all was good.
Roy