Here is the log of racoon not working…
Sep 13 09:00:29 racoon: [VPN1]: [205.xx.xx.115] ERROR: can't start the quick mode, there is no ISAKMP-SA, a5ae34896d5cf232:2e28fa92fa0948f8:000086d1
Sep 13 09:00:16 racoon: ERROR: failed to start post getspi.
Sep 13 09:00:16 racoon: ERROR: encryption 7 failed.
Sep 13 09:00:16 racoon: ERROR: OpenSSL function failed
Sep 13 09:00:16 racoon: [VPN2]: INFO: initiate new phase 2 negotiation: 24.xx.xx.7[500]<=>24.xx.xx.69[500]
Sep 13 09:00:16 racoon: [VPN2]: INFO: ISAKMP-SA established 24.xx.xx.7[500]-24.xx.xx.69[500] spi:ab349cae70c29beb:47be8288014e0c1b
Sep 13 09:00:16 racoon: ERROR: encryption 7 failed.
Sep 13 09:00:16 racoon: ERROR: OpenSSL function failed
Sep 13 09:00:16 racoon: [VPN2]: [24.xx.xx.69] NOTIFY: couldn't find the proper pskey, try to get one by the peer's address.
Sep 13 09:00:15 racoon: INFO: begin Aggressive mode.
Sep 13 09:00:15 racoon: [VPN2]: INFO: initiate new phase 1 negotiation: 24.xx.xx.7[500]<=>24.xx.xx.69[500]
Sep 13 09:00:15 racoon: [VPN2]: INFO: IPsec-SA request for 24.xx.xx.69 queued due to no phase1 found.
Sep 13 09:00:12 racoon: ERROR: such policy already exists. anyway replace it: 10.69.7.0/24[0] 10.77.2.0/24[0] proto=any dir=in
Sep 13 09:00:12 racoon: ERROR: such policy already exists. anyway replace it: 10.77.2.0/24[0] 10.69.7.0/24[0] proto=any dir=out
Sep 13 09:00:12 racoon: ERROR: such policy already exists. anyway replace it: 10.9.143.0/24[0] 10.77.2.0/24[0] proto=any dir=in
Sep 13 09:00:12 racoon: ERROR: such policy already exists. anyway replace it: 10.77.2.0/24[0] 10.9.143.0/24[0] proto=any dir=out
Sep 13 09:00:12 racoon: ERROR: such policy already exists. anyway replace it: 10.0.0.0/24[0] 10.77.2.0/24[0] proto=any dir=in
Sep 13 09:00:12 racoon: ERROR: such policy already exists. anyway replace it: 10.77.2.0/24[0] 10.0.0.0/24[0] proto=any dir=out
Sep 13 09:00:12 racoon: ERROR: such policy already exists. anyway replace it: 2001:470:xx:dcb::/64[0] 2001:470:xx:dcb::1/128[0] proto=any dir=in
Sep 13 09:00:12 racoon: ERROR: such policy already exists. anyway replace it: 2001:470:xx:dcb::1/128[0] 2001:470:xx:dcb::/64[0] proto=any dir=out
Sep 13 09:00:12 racoon: ERROR: such policy already exists. anyway replace it: 10.77.2.0/24[0] 10.77.2.1/32[0] proto=any dir=in
Sep 13 09:00:12 racoon: ERROR: such policy already exists. anyway replace it: 10.77.2.1/32[0] 10.77.2.0/24[0] proto=any dir=out
Sep 13 09:00:12 racoon: INFO: unsupported PF_KEY message REGISTER
Sep 13 09:00:12 racoon: [Self]: INFO: 24.xx.xx.7[500] used as isakmp port (fd=15)
Sep 13 09:00:12 racoon: [Self]: INFO: 24.xx.xx.7[500] used for NAT-T
Sep 13 09:00:12 racoon: [Self]: INFO: 24.xx.xx.7[4500] used as isakmp port (fd=14)
Sep 13 09:00:12 racoon: [Self]: INFO: 24.xx.xx.7[4500] used for NAT-T
Sep 13 09:00:12 racoon: INFO: Reading configuration from "/var/etc/racoon.conf"
Sep 13 09:00:12 racoon: INFO: @(#)This product linked OpenSSL 0.9.8n 24 Mar 2010 (http://www.openssl.org/)
Sep 13 09:00:12 racoon: INFO: @(#)ipsec-tools 0.8.0 (http://ipsec-tools.sourceforge.net)
Sep 13 09:00:06 racoon: INFO: racoon process 25694 shutdown
Sep 13 09:00:06 racoon: ERROR: encryption 7 failed.
Sep 13 09:00:06 racoon: ERROR: OpenSSL function failed
Sep 13 09:00:06 racoon: INFO: caught signal 15
Sep 13 09:00:04 racoon: ERROR: failed to start post getspi.
Sep 13 09:00:04 racoon: ERROR: encryption 7 failed.
Sep 13 09:00:04 racoon: ERROR: OpenSSL function failed
Sep 13 09:00:04 racoon: [VPN2]: INFO: initiate new phase 2 negotiation: 24.xx.xx.7[500]<=>24.xx.xx.69[500]
Here is the result of ps
# ps -A|grep racoon
28441 ?? Ss 0:00.02 /usr/local/sbin/racoon -f /var/etc/racoon.conf
29993 0 R+ 0:00.00 grep racoon
When I use the exact same command from a ssh shell it works.