IPSec connection goes stale when high throughput

reiven

Hi!

I'll try to explain the problem as best as I can. It's a weird one.
So I have two pfsense boxes, both running latest pfsense plus version. Both have IPSEC configured for remote access (using the cookbook guide)(IPSEC because this doesn't require any software installation on the client side). Both pfsenses are running Surricata and configured more or less in the same way.

"Map" would be like this
server - pfsense1 - ( The Internet ) - pfsense2 - mypc

So now about the problem.
I'm using Win11 on mypc, using Windows native client, connecting to pfsense1 via IPSec / IKEv2 and downloading stuff via SFTP from server. VPN connection establishes and stays good if I don't download or move heavy stuff. If I only browse the internet, it stays on for hours, but when I'm downloading something big (e.g a file ~10gb), it allows me to download approx 500mb at a time and then the tunnel becomes stale. Nothing moves. I cannot navigate to webpanels on server, the SFTP connection just hangs. Everything becomes to a halt. When I disconnect and reconnect, the cycle begins again.

What I've tried:

different clients - mac and windows - results are same
disabled suricata on pfsense1 and pfsense2, thought that it was some kinda connection drop or something
poked some IPSec configs, but without results

I hope that I could transfer my problem to text as English isn't my native.

PS. Suricata didn't like the low DH group 2, but Windows native client doesn't use higher, so I had to add that to suppress list. That is one thing. But it doesn't seem to be a Suricata issue

Any help would be appreciated

xylem

Hi!

I have the exact same error, with the same topology. For example, when I download larger files from the remote NAS via IPsec, the connection hangs.

I see the following entries under Diagnostics -> States -> States

When download starts (Throughput 100 MB/s until hanging):
Screenshot 2023-01-26 222245.png

When the connection hangs:
Screenshot 2023-01-26 222422.png
Screenshot 2023-01-26 222712.png

We have temporarily disabled Snort and pfBlocker.

Version 22-05-RELEASE (amd64), a Netgate 1537 at both sites, AES-NI CPU Crypto: Yes (active), QAT Crypto: No

VPN IPsec
Phase 1: AES128-GCM, AES-XCBC, DH14
Phase 2: ESP, AES128-GCM, PFS Key Group 14

Does anyone have a solution for this?
Thanks a lot!

xylem

@reiven Did you found a solution?

keyser

@xylem Try changing the cryptography cipher to AES-CBC (Just called AES in the UI).

I have had a lot of issues with AES-GCM between my boxes (stalled tunnels that does not reconnect). I thought it was related to one end being a ARM based box based on other posts in here. But perhaps it’s a more wirespread issue than that?

keyser

@xylem Also, you might want to consider enabling QAT to accelerate the VPn performance of your box.
Notice: I do not expect that to have any effect on the specific issue :-)

NOCling

The Windows 10 integrated ipsec Client runs fine with this Settings:

Mobike is activ and DPD.

You need only 2 lines of Powershell:

Add-VPNConnection -AllUserConnection -Name "Nameofvpnconnection" -ServerAddress "pfsensedyndnsFQDN" -TunnelType "Ikev2" -AuthenticationMethod Eap -RememberCredential

Set-VpnConnectionIPsecConfiguration -ConnectionName "Nameofvpnconnection" -AuthenticationTransformConstants GCMAES128 -CipherTransformConstants GCMAES128 -EncryptionMethod GCMAES128 -IntegrityCheckMethod SHA256 -DHGroup ECP256 -PfsGroup ECP256 -PassThru

Yes, there ist a Problem with SafeXcel on ARM and AES-GCM at the Moment.
But if you use a x86 VPN GW, GCM is the beste choice.

xylem

@keyser Thanks for the quick help! Too bad AES-GCM between two Netgate 1537 does not work!

The following encryptions work:
Throughput approx. 25 MB/s

Throughput approx. 16 MB/s

The following encryptions are very unstable:
Throughput approx. 75 MB/s

Can anyone confirm this? Or are there any other settings that can lead to these problems? We have just purchased two Netgate 1537 for the use of AES-GCM...

The throughput was not measured with iperf3. I can catch up on that when I get a chance.

reiven

This is my current configuration, which has the stalled connections issues. Includes AES256-GCM and AES

keyser

@xylem Whoaa, that was a huge difference in throughput between the ciphers. You obviously still have not enabled QAT on your boxes. If you do that throughput of AES-CBC will be much higher. I think enabling QAT requires a restart, but I may be mistaken.

keyser

@reiven said in IPSec connection goes stale when high throughput:

This is my current configuration, which has the stalled connections issues. Includes AES256-GCM and AES

When you offer so many different ciphers to the clients, you need to look in the logs to see which cipher was used when connections stalled. If you are in control of the clients, why not just offer one cipher and one transform (and no MoBike)? That way you know exactly what’s being used, and you can control whether a cipher thay may cause problems is available.

reiven

@keyser I did it because there wasn't a common denominator for Windows 10/11, Mac OS High Sierra / Ventura, Android 9-13, iOS 14-16 and Linux :) ... which is fine with everything.

With this configuration the stalling occurs on every platform. Windows - Android - Mac (haven't tested on iOS / Linux)

keyser

@reiven True if you are using the wizard driven creation of the VPN on clients :-)
If you are in control of the clients, you should consider just creating the tunnel on them with a profile tool/commandline statement. Then you can setup the ciphers and transforms as you desire (and restrict them to just one).

xylem

@keyser We do not have a CPIC-8955 Accelerator Card with Intel QuickAssist Technology installed in the Netgate 1537. I assume that QAT is therefore useless, right?

keyser

@xylem said in IPSec connection goes stale when high throughput:

@keyser We do not have a CPIC-8955 Accelerator Card with Intel QuickAssist Technology installed in the Netgate 1537. I assume that QAT is therefore useless, right?

Ahh, yeah, that’s right. The 1537/41 XEON based boxes does not have QAT built-in. Sorry about the confusion.

keyser

@reiven said in IPSec connection goes stale when high throughput:

With this configuration the stalling occurs on every platform. Windows - Android - Mac (haven't tested on iOS / Linux)

If the issue is the same as GCM on my ARM box, it’s actually the entire IPSec stack that stops when the issue occurs. So that would stop any other tunnels running - regardless of the ciphers they are using. But - that is on the ARM based boxes - that might be because the HW accelerator is involved in the issue.

NOCling

Look like if the rekey is a problem, check up your log files.
Do you have a look on it?

xylem

@nocling I have configured the tunnel again with the following settings:
Screenshot 2023-01-27 224329.png

Disconnected the tunnel manually and reconnected it.
After that I started downloading files from the remote NAS.
After the tunnel was established, no further log entries are visible.

Log Local Site

Jan 27 23:06:40	charon	99941	08[IKE] <con2|122> nothing to initiate
Jan 27 23:06:40	charon	99941	08[IKE] <con2|122> activating new tasks
Jan 27 23:06:40	charon	99941	08[CHD] <con2|122> CHILD_SA con2{316} state change: INSTALLING => INSTALLED
Jan 27 23:06:40	charon	99941	08[IKE] <con2|122> CHILD_SA con2{316} established with SPIs c18a90ef_i c133b9e9_o and TS 192.168.10.0/24|/0 === 192.168.20.0/24|/0
Jan 27 23:06:40	charon	99941	08[CHD] <con2|122> SPI 0xc133b9e9, src IP_LOCAL_SITE dst IP_REMOTE_SITE
Jan 27 23:06:40	charon	99941	08[CHD] <con2|122> adding outbound ESP SA
Jan 27 23:06:40	charon	99941	08[CHD] <con2|122> SPI 0xc18a90ef, src IP_REMOTE_SITE dst IP_LOCAL_SITE
Jan 27 23:06:40	charon	99941	08[CHD] <con2|122> adding inbound ESP SA
Jan 27 23:06:40	charon	99941	08[CHD] <con2|122> using AES_GCM_16 for encryption
Jan 27 23:06:40	charon	99941	08[CHD] <con2|122> CHILD_SA con2{316} state change: CREATED => INSTALLING
Jan 27 23:06:40	charon	99941	08[CFG] <con2|122> config: 192.168.20.0/24|/0, received: 192.168.20.0/24|/0 => match: 192.168.20.0/24|/0
Jan 27 23:06:40	charon	99941	08[CFG] <con2|122> selecting traffic selectors for other:
Jan 27 23:06:40	charon	99941	08[CFG] <con2|122> config: 192.168.10.0/24|/0, received: 192.168.10.0/24|/0 => match: 192.168.10.0/24|/0
Jan 27 23:06:40	charon	99941	08[CFG] <con2|122> selecting traffic selectors for us:
Jan 27 23:06:40	charon	99941	08[CFG] <con2|122> selected proposal: ESP:AES_GCM_16_256/NO_EXT_SEQ
Jan 27 23:06:40	charon	99941	08[CFG] <con2|122> configured proposals: ESP:AES_GCM_16_256/MODP_4096/NO_EXT_SEQ
Jan 27 23:06:40	charon	99941	08[CFG] <con2|122> received proposals: ESP:AES_GCM_16_256/NO_EXT_SEQ
Jan 27 23:06:40	charon	99941	08[CFG] <con2|122> proposal matches
Jan 27 23:06:40	charon	99941	08[CFG] <con2|122> selecting proposal:
Jan 27 23:06:40	charon	99941	08[IKE] <con2|122> received ESP_TFC_PADDING_NOT_SUPPORTED, not using ESPv3 TFC padding
Jan 27 23:06:40	charon	99941	08[IKE] <con2|122> maximum IKE_SA lifetime 27765s
Jan 27 23:06:40	charon	99941	08[IKE] <con2|122> scheduling rekeying in 24885s
Jan 27 23:06:40	charon	99941	08[IKE] <con2|122> IKE_SA con2[122] state change: CONNECTING => ESTABLISHED
Jan 27 23:06:40	charon	99941	08[IKE] <con2|122> IKE_SA con2[122] established between IP_LOCAL_SITE[IP_LOCAL_SITE]...IP_REMOTE_SITE[IP_REMOTE_SITE]
Jan 27 23:06:40	charon	99941	08[IKE] <con2|122> authentication of 'IP_REMOTE_SITE' with pre-shared key successful
Jan 27 23:06:40	charon	99941	08[IKE] <con2|122> received ESP_TFC_PADDING_NOT_SUPPORTED notify
Jan 27 23:06:40	charon	99941	08[ENC] <con2|122> parsed IKE_AUTH response 1 [ IDr AUTH N(ESP_TFC_PAD_N) SA TSi TSr ]
Jan 27 23:06:40	charon	99941	08[NET] <con2|122> received packet: from IP_REMOTE_SITE[500] to IP_LOCAL_SITE[500] (201 bytes)
Jan 27 23:06:40	charon	99941	08[NET] <con2|122> sending packet: from IP_LOCAL_SITE[500] to IP_REMOTE_SITE[500] (270 bytes)
Jan 27 23:06:40	charon	99941	08[ENC] <con2|122> generating IKE_AUTH request 1 [ IDi N(INIT_CONTACT) CERTREQ IDr AUTH N(ESP_TFC_PAD_N) SA TSi TSr N(MULT_AUTH) N(EAP_ONLY) N(MSG_ID_SYN_SUP) ]
Jan 27 23:06:40	charon	99941	08[IKE] <con2|122> establishing CHILD_SA con2{316} reqid 5
Jan 27 23:06:40	charon	99941	08[CFG] <con2|122> configured proposals: ESP:AES_GCM_16_256/NO_EXT_SEQ
Jan 27 23:06:40	charon	99941	08[CFG] <con2|122> 192.168.20.0/24|/0
Jan 27 23:06:40	charon	99941	08[CFG] <con2|122> proposing traffic selectors for other:
Jan 27 23:06:40	charon	99941	08[CFG] <con2|122> 192.168.10.0/24|/0
Jan 27 23:06:40	charon	99941	08[CFG] <con2|122> proposing traffic selectors for us:
Jan 27 23:06:40	charon	99941	08[IKE] <con2|122> successfully created shared key MAC
Jan 27 23:06:40	charon	99941	08[IKE] <con2|122> authentication of 'IP_LOCAL_SITE' (myself) with pre-shared key
Jan 27 23:06:40	charon	99941	08[IKE] <con2|122> sending cert request for "CN=mobile-ipsec-ca-bern"
Jan 27 23:06:40	charon	99941	08[IKE] <con2|122> IKE_AUTH task
Jan 27 23:06:40	charon	99941	08[IKE] <con2|122> IKE_CERT_PRE task
Jan 27 23:06:40	charon	99941	08[IKE] <con2|122> reinitiating already active tasks
Jan 27 23:06:40	charon	99941	08[IKE] <con2|122> received 1 cert requests for an unknown ca
Jan 27 23:06:40	charon	99941	08[IKE] <con2|122> received cert request for unknown ca with keyid 33:6f:1a:1a:f1:b6:24:fa:4f:cc:d7:1a:0c:1f:f3:71:87:ac:a6:6e
Jan 27 23:06:39	charon	99941	08[CFG] <con2|122> received supported signature hash algorithms: sha256 sha384 sha512 identity
Jan 27 23:06:39	charon	99941	08[CFG] <con2|122> selected proposal: IKE:AES_GCM_16_256/PRF_HMAC_SHA2_256/MODP_4096
Jan 27 23:06:39	charon	99941	08[CFG] <con2|122> configured proposals: IKE:AES_GCM_16_256/PRF_HMAC_SHA2_256/MODP_4096
Jan 27 23:06:39	charon	99941	08[CFG] <con2|122> received proposals: IKE:AES_GCM_16_256/PRF_HMAC_SHA2_256/MODP_4096
Jan 27 23:06:39	charon	99941	08[CFG] <con2|122> proposal matches
Jan 27 23:06:39	charon	99941	08[CFG] <con2|122> selecting proposal:
Jan 27 23:06:39	charon	99941	08[IKE] <con2|122> received CHILDLESS_IKEV2_SUPPORTED notify
Jan 27 23:06:39	charon	99941	08[IKE] <con2|122> received SIGNATURE_HASH_ALGORITHMS notify
Jan 27 23:06:39	charon	99941	08[IKE] <con2|122> received FRAGMENTATION_SUPPORTED notify
Jan 27 23:06:39	charon	99941	08[ENC] <con2|122> parsed IKE_SA_INIT response 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) CERTREQ N(FRAG_SUP) N(HASH_ALG) N(CHDLESS_SUP) N(MULT_AUTH) ]
Jan 27 23:06:39	charon	99941	08[NET] <con2|122> received packet: from IP_REMOTE_SITE[500] to IP_LOCAL_SITE[500] (745 bytes)
Jan 27 23:06:39	charon	99941	08[NET] <con2|122> sending packet: from IP_LOCAL_SITE[500] to IP_REMOTE_SITE[500] (712 bytes)
Jan 27 23:06:39	charon	99941	08[ENC] <con2|122> generating IKE_SA_INIT request 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) N(FRAG_SUP) N(HASH_ALG) N(REDIR_SUP) ]
Jan 27 23:06:39	charon	99941	08[CFG] <con2|122> sending supported signature hash algorithms: sha256 sha384 sha512 identity
Jan 27 23:06:39	charon	99941	08[CFG] <con2|122> configured proposals: IKE:AES_GCM_16_256/PRF_HMAC_SHA2_256/MODP_4096
Jan 27 23:06:39	charon	99941	08[IKE] <con2|122> IKE_SA con2[122] state change: CREATED => CONNECTING
Jan 27 23:06:39	charon	99941	08[IKE] <con2|122> initiating IKE_SA con2[122] to IP_REMOTE_SITE
Jan 27 23:06:39	charon	99941	08[IKE] <con2|122> activating IKE_AUTH_LIFETIME task
Jan 27 23:06:39	charon	99941	08[IKE] <con2|122> activating CHILD_CREATE task
Jan 27 23:06:39	charon	99941	08[IKE] <con2|122> activating IKE_CONFIG task
Jan 27 23:06:39	charon	99941	08[IKE] <con2|122> activating IKE_CERT_POST task
Jan 27 23:06:39	charon	99941	08[IKE] <con2|122> activating IKE_AUTH task
Jan 27 23:06:39	charon	99941	08[IKE] <con2|122> activating IKE_CERT_PRE task
Jan 27 23:06:39	charon	99941	08[IKE] <con2|122> activating IKE_NATD task
Jan 27 23:06:39	charon	99941	08[IKE] <con2|122> activating IKE_INIT task
Jan 27 23:06:39	charon	99941	08[IKE] <con2|122> activating IKE_VENDOR task
Jan 27 23:06:39	charon	99941	08[IKE] <con2|122> activating new tasks
Jan 27 23:06:39	charon	99941	08[IKE] <con2|122> queueing CHILD_CREATE task
Jan 27 23:06:39	charon	99941	08[IKE] <con2|122> queueing IKE_AUTH_LIFETIME task
Jan 27 23:06:39	charon	99941	08[IKE] <con2|122> queueing IKE_CONFIG task
Jan 27 23:06:39	charon	99941	08[IKE] <con2|122> queueing IKE_CERT_POST task
Jan 27 23:06:39	charon	99941	08[IKE] <con2|122> queueing IKE_AUTH task
Jan 27 23:06:39	charon	99941	08[IKE] <con2|122> queueing IKE_CERT_PRE task
Jan 27 23:06:39	charon	99941	08[IKE] <con2|122> queueing IKE_NATD task
Jan 27 23:06:39	charon	99941	08[IKE] <con2|122> queueing IKE_INIT task
Jan 27 23:06:39	charon	99941	08[IKE] <con2|122> queueing IKE_VENDOR task
Jan 27 23:06:39	charon	99941	10[CHD] <con2|121> CHILD_SA con2{314} state change: INSTALLED => DESTROYING
Jan 27 23:06:39	charon	99941	10[IKE] <con2|121> IKE_SA con2[121] state change: DELETING => DESTROYING
Jan 27 23:06:39	charon	99941	10[NET] <con2|121> sending packet: from IP_LOCAL_SITE[500] to IP_REMOTE_SITE[500] (65 bytes)
Jan 27 23:06:39	charon	99941	10[ENC] <con2|121> generating INFORMATIONAL request 2 [ D ]
Jan 27 23:06:39	charon	99941	10[IKE] <con2|121> sending DELETE for IKE_SA con2[121]
Jan 27 23:06:39	charon	99941	10[IKE] <con2|121> IKE_SA con2[121] state change: ESTABLISHED => DELETING
Jan 27 23:06:39	charon	99941	10[IKE] <con2|121> deleting IKE_SA con2[121] between IP_LOCAL_SITE[IP_LOCAL_SITE]...IP_REMOTE_SITE[IP_REMOTE_SITE]
Jan 27 23:06:39	charon	99941	10[IKE] <con2|121> activating IKE_DELETE task
Jan 27 23:06:39	charon	99941	10[IKE] <con2|121> activating new tasks
Jan 27 23:06:39	charon	99941	10[IKE] <con2|121> queueing IKE_DELETE task
Jan 27 23:06:26	charon	99941	08[CHD] CHILD_SA con2{315} state change: CREATED => ROUTED
Jan 27 23:06:26	charon	99941	08[CFG] installing 'con2'
Jan 27 23:06:26	charon	99941	08[CHD] CHILD_SA con2{305} state change: ROUTED => DESTROYING
Jan 27 23:06:26	charon	99941	08[CFG] uninstalling 'con2'
Jan 27 23:06:26	charon	99941	08[CFG] updated vici connection: con2
Jan 27 23:06:26	charon	99941	08[CFG] child con2:
Jan 27 23:06:26	charon	99941	08[CFG] conn con2:

Log Remote Site

Jan 27 23:06:40	charon	20256	14[NET] <con1|12> sending packet: from IP_REMOTE_SITE[500] to IP_LOCAL_SITE[500] (201 bytes)
Jan 27 23:06:40	charon	20256	14[ENC] <con1|12> generating IKE_AUTH response 1 [ IDr AUTH N(ESP_TFC_PAD_N) SA TSi TSr ]
Jan 27 23:06:40	charon	20256	14[CHD] <con1|12> CHILD_SA con1{26} state change: INSTALLING => INSTALLED
Jan 27 23:06:40	charon	20256	14[IKE] <con1|12> CHILD_SA con1{26} established with SPIs c133b9e9_i c18a90ef_o and TS 192.168.20.0/24|/0 === 192.168.10.0/24|/0
Jan 27 23:06:40	charon	20256	14[CHD] <con1|12> SPI 0xc18a90ef, src IP_REMOTE_SITE dst IP_LOCAL_SITE
Jan 27 23:06:40	charon	20256	14[CHD] <con1|12> adding outbound ESP SA
Jan 27 23:06:40	charon	20256	14[CHD] <con1|12> SPI 0xc133b9e9, src IP_LOCAL_SITE dst IP_REMOTE_SITE
Jan 27 23:06:40	charon	20256	14[CHD] <con1|12> adding inbound ESP SA
Jan 27 23:06:40	charon	20256	14[CHD] <con1|12> using AES_GCM_16 for encryption
Jan 27 23:06:40	charon	20256	14[CHD] <con1|12> CHILD_SA con1{26} state change: CREATED => INSTALLING
Jan 27 23:06:40	charon	20256	14[CFG] <con1|12> config: 10.10.10.0/24|/0, received: 192.168.10.0/24|/0 => no match
Jan 27 23:06:40	charon	20256	14[CFG] <con1|12> config: 192.168.10.0/24|/0, received: 192.168.10.0/24|/0 => match: 192.168.10.0/24|/0
Jan 27 23:06:40	charon	20256	14[CFG] <con1|12> selecting traffic selectors for other:
Jan 27 23:06:40	charon	20256	14[CFG] <con1|12> config: 192.168.20.0/24|/0, received: 192.168.20.0/24|/0 => match: 192.168.20.0/24|/0
Jan 27 23:06:40	charon	20256	14[CFG] <con1|12> config: 192.168.20.0/24|/0, received: 192.168.20.0/24|/0 => match: 192.168.20.0/24|/0
Jan 27 23:06:40	charon	20256	14[CFG] <con1|12> selecting traffic selectors for us:
Jan 27 23:06:40	charon	20256	14[CFG] <con1|12> selected proposal: ESP:AES_GCM_16_256/NO_EXT_SEQ
Jan 27 23:06:40	charon	20256	14[CFG] <con1|12> configured proposals: ESP:AES_GCM_16_256/MODP_4096/NO_EXT_SEQ, ESP:AES_GCM_16_256/MODP_2048/NO_EXT_SEQ, ESP:AES_GCM_16_128/MODP_2048/NO_EXT_SEQ, ESP:AES_CBC_256/HMAC_SHA2_256_128/MODP_2048/NO_EXT_SEQ, ESP:AES_CBC_256/HMAC_SHA2_384_192/MODP_2048/NO_EXT_SEQ, ESP:AES_CBC_256/HMAC_SHA2_512_256/MODP_2048/NO_EXT_SEQ, ESP:AES_CBC_192/HMAC_SHA2_256_128/MODP_2048/NO_EXT_SEQ, ESP:AES_CBC_192/HMAC_SHA2_384_192/MODP_2048/NO_EXT_SEQ, ESP:AES_CBC_192/HMAC_SHA2_512_256/MODP_2048/NO_EXT_SEQ, ESP:AES_CBC_128/HMAC_SHA2_256_128/MODP_2048/NO_EXT_SEQ, ESP:AES_CBC_128/HMAC_SHA2_384_192/MODP_2048/NO_EXT_SEQ, ESP:AES_CBC_128/HMAC_SHA2_512_256/MODP_2048/NO_EXT_SEQ
Jan 27 23:06:40	charon	20256	14[CFG] <con1|12> received proposals: ESP:AES_GCM_16_256/NO_EXT_SEQ
Jan 27 23:06:40	charon	20256	14[CFG] <con1|12> proposal matches
Jan 27 23:06:40	charon	20256	14[CFG] <con1|12> selecting proposal:
Jan 27 23:06:40	charon	20256	14[CFG] <con1|12> found matching child config "con1" with prio 10
Jan 27 23:06:40	charon	20256	14[CFG] <con1|12> candidate "con1" with prio 5+5
Jan 27 23:06:40	charon	20256	14[CFG] <con1|12> 10.10.10.0/24|/0
Jan 27 23:06:40	charon	20256	14[CFG] <con1|12> 192.168.10.0/24|/0
Jan 27 23:06:40	charon	20256	14[CFG] <con1|12> proposing traffic selectors for other:
Jan 27 23:06:40	charon	20256	14[CFG] <con1|12> 192.168.20.0/24|/0
Jan 27 23:06:40	charon	20256	14[CFG] <con1|12> 192.168.20.0/24|/0
Jan 27 23:06:40	charon	20256	14[CFG] <con1|12> proposing traffic selectors for us:
Jan 27 23:06:40	charon	20256	14[CFG] <con1|12> looking for a child config for 192.168.20.0/24|/0 === 192.168.10.0/24|/0
Jan 27 23:06:40	charon	20256	14[IKE] <con1|12> maximum IKE_SA lifetime 30568s
Jan 27 23:06:40	charon	20256	14[IKE] <con1|12> scheduling rekeying in 27400s
Jan 27 23:06:40	charon	20256	14[IKE] <con1|12> IKE_SA con1[12] state change: CONNECTING => ESTABLISHED
Jan 27 23:06:40	charon	20256	14[IKE] <con1|12> IKE_SA con1[12] established between IP_REMOTE_SITE[IP_REMOTE_SITE]...IP_LOCAL_SITE[IP_LOCAL_SITE]
Jan 27 23:06:40	charon	20256	14[IKE] <con1|12> successfully created shared key MAC
Jan 27 23:06:40	charon	20256	14[IKE] <con1|12> authentication of 'IP_REMOTE_SITE' (myself) with pre-shared key
Jan 27 23:06:40	charon	20256	14[IKE] <con1|12> received ESP_TFC_PADDING_NOT_SUPPORTED, not using ESPv3 TFC padding
Jan 27 23:06:40	charon	20256	14[IKE] <con1|12> authentication of 'IP_LOCAL_SITE' with pre-shared key successful
Jan 27 23:06:40	charon	20256	14[CFG] <con1|12> selected peer config 'con1'
Jan 27 23:06:40	charon	20256	14[CFG] <12> candidate "con1", match: 20/20/3100 (me/other/ike)
Jan 27 23:06:39	charon	20256	14[CHD] <con1|11> CHILD_SA con1{25} state change: INSTALLED => DESTROYING
Jan 27 23:06:39	charon	20256	14[IKE] <con1|11> IKE_SA con1[11] state change: DELETING => DESTROYING
Jan 27 23:06:39	charon	20256	14[NET] <con1|11> sending packet: from IP_REMOTE_SITE[500] to IP_LOCAL_SITE[500] (57 bytes)
Jan 27 23:06:39	charon	20256	14[ENC] <con1|11> generating INFORMATIONAL response 2 [ ]
Jan 27 23:06:39	charon	20256	14[IKE] <con1|11> IKE_SA deleted
Jan 27 23:06:39	charon	20256	14[IKE] <con1|11> IKE_SA con1[11] state change: ESTABLISHED => DELETING
Jan 27 23:06:39	charon	20256	14[IKE] <con1|11> deleting IKE_SA con1[11] between IP_REMOTE_SITE[IP_REMOTE_SITE]...IP_LOCAL_SITE[IP_LOCAL_SITE]
Jan 27 23:06:39	charon	20256	14[IKE] <con1|11> received DELETE for IKE_SA con1[11]
Jan 27 23:06:39	charon	20256	14[ENC] <con1|11> parsed INFORMATIONAL request 2 [ D ]
Jan 27 23:06:39	charon	20256	14[NET] <con1|11> received packet: from IP_LOCAL_SITE[500] to IP_REMOTE_SITE[500] (65 bytes)

I don't know if this is helpful....

NOCling

Looks good, do you see a working rekey?

Jan 28 09:04:23	charon	88416	05[KNL] creating rekey job for CHILD_SA ESP/0xc69ac11a/x.x.x.x

But i have to use AES_CBC_256 to my 2100.

keyser

@xylem It looks right. Rellay interesting if this issue is the CGM cipher problem on AMD64 hardware. To be fair your symptoms are different, so it’s unlikely it’s the same issue. Looks more like a rekey problem like @NOCling suggested.

xylem

@nocling Unfortunately, I do not receive any log entries after the tunnel is established. But strangely enough I can access the remote network. Iperf3 to the remote pfSense also works. File download from the remote NAS starts without problems with 100 MB/s and suddenly stops after a few GB. Without log entries it is quite difficult. How can I get to the bottom of this problem?