This appears to be a routing issue: I can do a packet capture on the IPSec interface of pfsense, and I can see incoming pings, and their destination:
12:52:18.793013 (authentic,confidential): SPI 0x083c9c1c: IP 10.1.53.1 > {LAN_IP}: ICMP echo request, id 1, seq 1871, length 40
12:52:19.826520 (authentic,confidential): SPI 0x083c9c1c: IP 10.1.53.1 > {LAN_IP}: ICMP echo request, id 1, seq 1872, length 40
12:52:21.329649 (authentic,confidential): SPI 0x083c9c1c: IP 10.1.53.1 > {LAN_IP}: ICMP echo request, id 1, seq 1873, length 40
12:52:23.829947 (authentic,confidential): SPI 0x083c9c1c: IP 10.1.53.1 > {LAN_IP2}: ICMP echo request, id 1, seq 1881, length 40
12:52:25.326576 (authentic,confidential): SPI 0x083c9c1c: IP 10.1.53.1 > {LAN_IP2}: ICMP echo request, id 1, seq 1882, length 40
After I disconnect, and have cleared the ipsec log, this appears after a moment or two:
Apr 28 12:49:50 racoon: DEBUG: pk_recv: retry[0] recv()
Apr 28 12:49:50 racoon: DEBUG: got pfkey ACQUIRE message
Apr 28 12:49:50 racoon: DEBUG: suitable outbound SP found: 0.0.0.0/0[0] 10.1.53.1/32[0] proto=any dir=out.
Apr 28 12:49:50 racoon: [Unknown Gateway/Dynamic]: DEBUG: sub:0xbfbfe728: 10.1.53.1/32[0] 0.0.0.0/0[0] proto=any dir=in
Apr 28 12:49:50 racoon: [Unknown Gateway/Dynamic]: DEBUG: db :0x28501288: {LAN_SUBNET}/24[0] {LAN_IP}/32[0] proto=any dir=in
Apr 28 12:49:50 racoon: [Unknown Gateway/Dynamic]: DEBUG: sub:0xbfbfe728: 10.1.53.1/32[0] 0.0.0.0/0[0] proto=any dir=in
Apr 28 12:49:50 racoon: [Unknown Gateway/Dynamic]: DEBUG: db :0x28501648: {LAN_IP}/32[0] {LAN_SUBNET}/24[0] proto=any dir=out
Apr 28 12:49:50 racoon: [Unknown Gateway/Dynamic]: DEBUG: sub:0xbfbfe728: 10.1.53.1/32[0] 0.0.0.0/0[0] proto=any dir=in
Apr 28 12:49:50 racoon: [Unknown Gateway/Dynamic]: DEBUG: db :0x285013c8: 10.1.53.1/32[0] 0.0.0.0/0[0] proto=any dir=in
Apr 28 12:49:50 racoon: [Unknown Gateway/Dynamic]: DEBUG: suitable inbound SP found: 10.1.53.1/32[0] 0.0.0.0/0[0] proto=any dir=in.
Im not sure if that is relevant or not.