After taking the screenshot, and recognizing the mismatch between the ports, I've updated the PHASE1 settings on both ends, specifying just the NAT-T port.
0dbb0d4a-70c8-496a-87dd-bee9fa740865-image.png
Now, the ports looks coherent.
SITE A
5387557b-d330-43ec-a494-e44119f1e484-image.png
SITE B
a0791832-6b78-4a3a-a053-f749822d43b5-image.png
Now ping works :)
996ddfcd-d96a-4b60-9bb5-f194f3ed1fa9-image.png
08a81d89-236c-44e0-8ecc-26dc19d27d4e-image.png
Still open the question on why this port mismatch happened.....I've lost like 40 hours on this