Alright last post from me. Leaving it here so someone can find it.
The documentation concerning carp is wrong: "A High Availability cluster using CARP needs three IP addresses in each subnet along with a separate unused subnet for the Sync interface."
The fact is, a High Availability cluster using CARP needs only one IP address. It only ever needed one IP address. This statement directly contradicts all the documentation available from carp(4) and the FreeBSD handbook.
The distinction that I initially missed, but now have reread and understand is that when using a single IP assigned to a VIP, so long as there isn't an existing network with another IP in that same network, then the network for that VIP should in fact be a whatever that network is. Otherwise it should be a /32.
Let's put it this way for a further understanding (sudo interface configuration):
Server 1 (Primary):
ifconfig em0 inet 192.168.0.10.1/24
ifconfig em0 inet 192.168.0.0.2/24 alias
ifconfig em0 vhid 1 advskew 100 pass mekmitasdigoat 192.168.0.1/24 alias
Server 2 (Backup):
ifconfig em0 inet 192.168.0.0.2/24
ifconfig em0 inet 192.168.0.0.3/24 alias
ifconfig em0 vhid 1 advskew 100 pass mekmitasdigoat 192.168.0.1/24 alias
In the above example, if the OS chooses the VIP of 192.168.0.1/24 for packets sourced from Backup, Backup will never see the response, as they'll go to Primary instead. Going to primary is the expected part. Source selection of 192.168.0.1 is the unexpected but, It's unexpected because the netgate documentation is just wrong as this VIP should have been a /32.
Documentation where the VIP isn't a /32 to which netgate is correct:
Server 1 (Primary):
ifconfig em0 inet 192.168.0.10.1/24
ifconfig em0 inet 192.168.0.1.1/24 alias
ifconfig em0.123 vhid 1 advskew 100 pass mekmitasdigoat 192.168.0.1/24 alias
Server 2 (Backup):
ifconfig em0 inet 192.168.0.0.2/24
ifconfig em0 inet 192.168.0.1.2/24 alias
ifconfig em0 vhid 1 advskew 100 pass mekmitasdigoat 192.168.0.1/24 alias
