Windows 11 IPSec ESP no acceptable proposal found

lifeboy

We are trying to connect using a configuration that works perfectly with Windows 10 as a client, but with Windows 11 we get this:

Apr 22 17:22:26	charon	12318	01[IKE] <con-mobile|1602> no acceptable proposal found
Apr 22 17:22:26	charon	12318	01[CFG] <con-mobile|1602> configured proposals: ESP:AES_GCM_16_256/MODP_2048/NO_EXT_SEQ, ESP:AES_GCM_12_256/MODP_2048/NO_EXT_SEQ, ESP:AES_GCM_8_256/MODP_2048/NO_EXT_SEQ, ESP:AES_CBC_256/HMAC_SHA2_256_128/MODP_2048/NO_EXT_SEQ, ESP:AES_CBC_192/HMAC_SHA2_256_128/MODP_2048/NO_EXT_SEQ, ESP:AES_CBC_128/HMAC_SHA2_256_128/MODP_2048/NO_EXT_SEQ, ESP:AES_GCM_16_192/MODP_2048/NO_EXT_SEQ, ESP:AES_GCM_12_192/MODP_2048/NO_EXT_SEQ, ESP:AES_GCM_8_192/MODP_2048/NO_EXT_SEQ, ESP:AES_GCM_16_128/MODP_2048/NO_EXT_SEQ, ESP:AES_GCM_12_128/MODP_2048/NO_EXT_SEQ, ESP:AES_GCM_8_128/MODP_2048/NO_EXT_SEQ, ESP:AES_CBC_256/HMAC_SHA2_384_192/MODP_2048/NO_EXT_SEQ, ESP:AES_CBC_256/HMAC_SHA2_512_256/MODP_2048/NO_EXT_SEQ, ESP:AES_CBC_256/AES_XCBC_96/MODP_2048/NO_EXT_SEQ, ESP:AES_CBC_192/HMAC_SHA2_384_192/MODP_2048/NO_EXT_SEQ, ESP:AES_CBC_192/HMAC_SHA2_512_256/MODP_2048/NO_EXT_SEQ, ESP:AES_CBC_192/AES_XCBC_96/MODP_2048/NO_EXT_SEQ, ESP:AES_CBC_128/HMAC_SHA2_384_192/MODP_2048/NO_EXT_SEQ, ESP:AES_CBC_128/HMAC_SHA2_512_256/MODP_2048/NO_EXT_SEQ, ESP:AES_CBC_128/AES_XCBC_96/MODP_2048/NO_EXT_SEQ
Apr 22 17:22:26	charon	12318	01[CFG] <con-mobile|1602> received proposals: ESP:AES_CBC_256/AES_256_GMAC/NO_EXT_SEQ
Apr 22 17:22:26	charon	12318	01[CFG] <con-mobile|1602> no acceptable ENCRYPTION_ALGORITHM found
Apr 22 17:22:26	charon	12318	01[CFG] <con-mobile|1602> selecting proposal:

I have enabled all available phase 2 ESP options to make sure that I didn't just select the wrong one, but it's clear that Windows 11 requests something Windows 10 doesn't.

Windows 10 does this:

Apr 22 17:02:36	charon	12318	15[CFG] <con-mobile|1598> selected proposal: ESP:AES_GCM_16_256/NO_EXT_SEQ
Apr 22 17:02:36	charon	12318	15[CFG] <con-mobile|1598> configured proposals: ESP:AES_GCM_16_256/MODP_2048/NO_EXT_SEQ, ESP:AES_GCM_12_256/MODP_2048/NO_EXT_SEQ, ESP:AES_GCM_8_256/MODP_2048/NO_EXT_SEQ, ESP:AES_CBC_256/HMAC_SHA2_256_128/MODP_2048/NO_EXT_SEQ, ESP:AES_CBC_192/HMAC_SHA2_256_128/MODP_2048/NO_EXT_SEQ, ESP:AES_CBC_128/HMAC_SHA2_256_128/MODP_2048/NO_EXT_SEQ
Apr 22 17:02:36	charon	12318	15[CFG] <con-mobile|1598> received proposals: ESP:AES_GCM_16_256/NO_EXT_SEQ
Apr 22 17:02:36	charon	12318	15[CFG] <con-mobile|1598> proposal matches

So Windows 11 wants
ESP:AES_CBC_256/AES_256_GMAC/NO_EXT_SEQ
but with the same client config Windows 10 wants
ESP:AES_GCM_16_256/NO_EXT_SEQ

Any ideas on how to fix this?

jimp

Use The IPsec export function on Plus, or if you're on CE, use powershell on Windows to set the exact parameters you want on both to match.

https://learn.microsoft.com/en-us/powershell/module/vpnclient/?view=windowsserver2022-ps

We don't have any specific examples of that in the pfSense docs, but there are plenty of examples around the Internet (and even some in the TNSR docs)

lifeboy

@jimp I have indeed used powershell to create the tunnel in the first instance, since without it only an insecure tunnel is possible.

According to Microsoft's documentation Windows 10 offers:

Phase 1 encryption

Type:	EncryptionMethod
Accepted values: DES, DES3, AES128, AES192, AES256, GCMAES128, GCMAES256

Phase 2 encryption
Type:	CipherTransformConstants
Accepted values: DES, DES3, AES128, AES192, AES256, GCMAES128, GCMAES192, GCMAES256, None

Windows 11 offers:

Phase 1 encryption

Type:	EncryptionMethod
Accepted values: DES, DES3, AES128, AES192, AES256, GCMAES128, GCMAES256

Phase 2 encryption

Type:	CipherTransformConstants
Accepted values: DES, DES3, AES128, AES192, AES256, GCMAES128, GCMAES192, GCMAES256, None

So the only option one has to get both Win 10 and 11 to IPSec to work is to use a method that is supported by both and pfSense. Since GCM used hardware encryption GCMAES128 seems like a good choice or CGMAES256.

The Powershell command we used to create the tunnel on both Windows 10 and 11 is:

Set-VpnConnectionIPsecConfiguration `
-ConnectionName "IPSec-cloud" `
-AuthenticationTransformConstants GCMAES256 `
-CipherTransformConstants AES256 `
-EncryptionMethod GCMAES256 `
-IntegrityCheckMethod SHA256 `
-DHGroup Group14 `
-PfsGroup PFS2048 `
-PassThru

However, despite explicitly setting Windows 11 to use GCMAES256, it presents itself to pfSense as AES_CBC_256/AES_256_GMAC, neither of which are available for selection in pfSense.

So it's not simply a matter of picking the matching encryption. AES-CBC is not available in pfSense at all. What GMAC is I don't know.

I'm stuck now.... :-(

lifeboy

Has been any resolution of this? Surely I'm not only person in the world trying to connect a Windows 11 IPSec client to pfSense CE?

keyser

@lifeboy I have no issues connecting to pfSense using the built-in VPN in Win11.

I just use these settings:

And here is the powershell to create the VPN:

Add-VpnConnection -Name "XXXXX" -ServerAddress host.domain.dk -TunnelType "Ikev2"
Set-VpnConnectionIPsecConfiguration -ConnectionName "XXXXX" -AuthenticationTransformConstants "SHA256128" -CipherTransformConstants "AES256" -DHGroup "Group14" -EncryptionMethod "AES256" -IntegrityCheckMethod "SHA256" -PfsGroup "PFS2048" -Force

lifeboy

@keyser thanks for that! I'll have to check with a client that has a Windows 11 machine (I don't have one).

Which one of the these translates to SHA256128 in IPSec Encryption on pfSense?

keyser

@lifeboy When editing the Phase one and Phase 2 settings, only one encryption settings is enabled in both:
AES256 and using SHA256 with DH14: