Nov 6 14:04:54 kernel carp: 150@ix3: BACKUP -> MASTER (master timed out)
Nov 6 14:04:54 kernel carp: 155@ix3: BACKUP -> MASTER (master timed out)

but then it recovers immediately:

Nov 6 14:04:56 kernel carp: 155@ix3: MASTER -> BACKUP (more frequent advertisement received)
Nov 6 14:04:56 kernel carp: 150@ix3: MASTER -> BACKUP (more frequent advertisement received)

That is from the first set of entries but there were a few pairs like that.

Router1 logs this:
Nov 6 14:05:27 kernel carp: demoted by 0 to 0 (send ok on ix3)
Nov 6 14:05:27 kernel carp: demoted by 0 to 0 (send ok on ix3)
Nov 6 14:05:15 kernel carp: demoted by 0 to 0 (send error 55 on ix3)
Nov 6 14:05:15 kernel carp: demoted by 0 to 0 (send error 55 on ix3)
Nov 6 14:04:58 kernel carp: demoted by 0 to 0 (send ok on ix3)
Nov 6 14:04:58 kernel carp: demoted by 0 to 0 (send ok on ix3)
Nov 6 14:04:53 kernel carp: demoted by 0 to 0 (send error 55 on ix3)
Nov 6 14:04:53 kernel carp: demoted by 0 to 0 (send error 55 on ix3)
Nov 6 14:04:33 kernel carp: demoted by 0 to 0 (send ok on ix3)
Nov 6 14:04:32 kernel carp: demoted by 0 to 0 (send ok on ix3)
Nov 6 14:04:31 kernel carp: demoted by 0 to 0 (send error 55 on ix3)
Nov 6 14:04:31 kernel carp: demoted by 0 to 0 (send error 55 on ix3)

(note, either the times don't line up, or "14:04:31" didn't trigger the failover on router2)

Could this be a switch problem? Like the above post I looked at mbufs which per router1 have a max of about 24000 out of 1 million.

We upgraded these two routers to 24.03 six months ago so that's not new. I restarted both last week. This happens every few days, say somewhere between 1-10 days.

Anything else we can look at?