Racoon IPSEC Phase 2 AES issue when > 128Bit

wallacebw

All:

I have 2 PFsense hosts using an IPSEC tunnel

Host 1:
This host is an ALIX platform running the following 'embedded' version:
FreeBSD HOST.NAME.HERE 8.1-RELEASE-p4 FreeBSD 8.1-RELEASE-p4 #0: Sun Jun 12 12:24:53 EDT 2011 sullrich@FreeBSD_8.0_pfSense_2.0-snaps.pfsense.org:/usr/obj.pfSense/usr/pfSensesrc/src/syspfSense_wrap.8.i386 i386

Host 2:
This is a physical server running the following AMD64 version:
FreeBSD firewall.jaxlab.net 8.1-RELEASE-p4 FreeBSD 8.1-RELEASE-p4 #1: Wed Jun 15 21:13:49 EDT 2011 root@FreeBSD_8.0_pfSense_2.0-AMD64.snaps.pfsense.org:/usr/obj.pfSense/usr/pfSensesrc/src/sys/pfSense_SMP.8 amd64

Both hosts report the following Racoon version:

@(#)ipsec-tools 0.8.0 (http://ipsec-tools.sourceforge.net)

Compiled with:
- OpenSSL 0.9.8n 24 Mar 2010 (http://www.openssl.org/)
- IPv6 support
- Dead Peer Detection
- IKE fragmentation
- Hybrid authentication
- NAT Traversal
- Admin port
- Monotonic clock

I currently have a functioning tunnel between these hosts (AES-256 Phase1) and (BlowFish-256 Phase 2). Here is the odd thing. If I change the Phase 2 to the AES-128 cypher, everything works fine. The problem comes in when I attempt to use AES 192 or 256 for phase 2, When I attempt this, I see the following in the IPSEC logs (collected from the ALIX host)


Jun 17 09:26:31 	racoon: [Tunnel_Name]: INFO: initiate new phase 2 negotiation: ALIX_IP[500]<=>AMD64SERVER_IP[500]
Jun 17 09:26:31 	racoon: ERROR: pfkey UPDATE failed: Invalid argument
Jun 17 09:26:31 	racoon: ERROR: pfkey ADD failed: Invalid argument
Jun 17 09:27:01 	racoon: ERROR: AMD64SERVER_IP give up to get IPsec-SA due to time up to wait.

Does this make any sense?