Update
After doing some wireshark traces I concluded the traffic was not getting back to the phone. I was able to identify a routing issue that was causing the problem and resolve it.
I have now been able to connect the Avaya VPN handset through the IPSec tunnel to my phone system.
So just in case anyone else tries to set this up the the following settings in the Avaya handset work:
VPN VENDOR - OTHER
Gateway address - 0.0.0.0 (set by DHCP)
External Phone IP Address 0.0.0.0 (set by DHCP)
External Subnet - 0.0.0.0 (set by DHCP)
External DNS - 0.0.0.0 (set by DHCP)
Encapsulation - 4500-4500
Copy TOS - No
Auth Type - PSK with XAUTH
VPN User TYPE - any
VPN User -vpnuser
VPN PW - *
IKE ID (Group Name) - none
Pre-Shared Key (PSK) - *
IKE Phase 1
IKE ID Type - IPV4 ADDRESS
IKE Xchg Mode - Aggressive
IKE DH GROUP - 2
IKE Encryption Alg - AES-256
IKE Auth Alg - SHA-1
IKE Config Mode - Enabled.
IKE Phase 2
IPSEC PFS DH Group - No PFS
IPSEC Encryption Alg - AES-256
IPSec Auth Alg - SHA-1
Protected Network - 0.0.0.0/0