The tunnel also didn't route IPv6 over itself, even though I had IPv4 & IPv6 P2s defined. Again, from the commandline I did this on one side:
ifconfig ipsec1000 inet6 2600:3c01:e000:31e::2 prefixlen 112
and this on the other:
ifconfig ipsec1000 inet6 2600:3c01:e000:31e::1 prefixlen 112
Giving me this:
ipsec1000: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1400
tunnel inet 96.126.96.153 --> 73.140.16.217
inet6 fe80::84b8:2eb3:a617:de8a%ipsec1000 prefixlen 64 scopeid 0x6
inet6 2600:3c01:e000:31e::2 prefixlen 112
inet 10.20.30.1 --> 10.20.30.2 netmask 0xfffffffc
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
reqid: 1000
groups: ipsec
And IPv6 worked.