Yeah like @michmoor is mentioning, I'd double check the config on both sides for Phase 1 and 2 and be sure they are identical.
If that still doesn't work then I'd dig deeper on the deleting SA issue mentioned by @Konstanti
Might also be worth checking to be sure the Fortigate is fully updated so there isn't a chance for some old bug.