@thewaterbug Excellent tests and performance comparison :-) Last time i checked, both AES-128 and AES256 (both in CBC and GCM mode) were considered safe since they need HEAVY supercomputertime to be decrypted. 128bit is “possible” to decrypt with modern supercomputers, but not in a anywhere close to usable timeframe. 256bit is not practically decryptable (we are looking at many many years of superc time).

@keyser Fortunately I don't think we are getting any packet loss here, more testing and I'm seeing absolutely zero retransmits so that's good. Unfortunately though the specific device that needs the high bandwidth is from a vendor and uses their own OS, it's technically Ubuntu but point is I can't login to make any adjustments to it. Might find out if the vendor can do any tuning on it for this remote setup. It's an appliance that typically is on a local subnet rather than remote so it's definitely not setup for this, so considering it only operates about half as fast as on site, I'm pretty happy lol, just wanted to double check things. Thanks again for all the insight here, greatly appreciate it!

@mclaborn Ok from there Id check the Routes table, I'd check all your firewall rules, and I'd run a tracert to see if if it going somewhere funky. Also check your Outbound NAT rules to see if there's a redirect there, too, or maybe you have a 1:1 that is translating the IP to something else.

@jimp must be something else… Site-to-Site has remote address set. I've checked P2 as well vor both VPNs. They have the correct values for their P1 set. [image: 1669213686037-ipsec-tunnel-config.png]

@nick-loenders Anyway, if you have sequenced subnets like these you can embrace them in the p2 using an appropriate mask. But with a local LAN of 10.0.1.0/24 you run into risk of overlapping. So if the LAN here is 10.0.1.0/24 you could only merge tunnel 2 and 3 by stating 10.0.2.0/23 as the remote network. If you have control over all involved site you should consider this when designing the networks.

Yep VTI work without issues with AWS/BGP

@upper-deck that is awesome. I’ll play with mine later but this was a cool learning moment to me.

@thewaterbug I’d personally recommend you look into VTI ipsec which will create a gateway that you can then use in a policy based firewall rule for the specific host you want to give access to.

maybe you need to check the firewall rule of ipsec.

BTW, this is not for me: [image: 1668647646584-03a3bfdb-92a9-437a-88f5-90c7db833c2b-image.png] I get best ipsec performance by these: [image: 1668647884865-3f5503cf-ce81-4ef8-adff-d4a82a611547-image.png]

@viragomann Hi 10.0.0.0 is connected over ipsec to 10.0.8.0 And our local destination 192.168.2.0 is connected over ip sec to 10.0.0.0 The location of the customer with 192.168.2.0 ist connected to 10.0.8.0 When the customer starts vpn, then wen cant connect from 192.168.2.0 over 10.0.0.0

@thewaterbug Interesting. I have this working from macOS (10.14.6), but not from iOS 16. Here's the IPSec log after my Mac connects successfully: Nov 8 15:21:23 charon 1186 06[NET] <11> received packet: from my.mobile.ip.address[500] to my.public.gateway.address[500] (604 bytes) Nov 8 15:21:23 charon 1186 06[ENC] <11> parsed IKE_SA_INIT request 0 [ SA KE No N(REDIR_SUP) N(NATD_S_IP) N(NATD_D_IP) N(FRAG_SUP) ] Nov 8 15:21:23 charon 1186 06[IKE] <11> my.mobile.ip.address is initiating an IKE_SA Nov 8 15:21:23 charon 1186 06[CFG] <11> selected proposal: IKE:AES_CBC_256/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_2048 Nov 8 15:21:24 charon 1186 06[IKE] <11> remote host is behind NAT Nov 8 15:21:24 charon 1186 06[IKE] <11> sending cert request for "CN=ACMERocketCarsCA, C=US, ST=California, L=San Francisco, O=ACME RocketCars" Nov 8 15:21:24 charon 1186 06[ENC] <11> generating IKE_SA_INIT response 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) CERTREQ N(FRAG_SUP) N(CHDLESS_SUP) N(MULT_AUTH) ] Nov 8 15:21:24 charon 1186 06[NET] <11> sending packet: from my.public.gateway.address[500] to my.mobile.ip.address[500] (481 bytes) Nov 8 15:21:24 charon 1186 06[NET] <11> received packet: from my.mobile.ip.address[51296] to my.public.gateway.address[4500] (528 bytes) Nov 8 15:21:24 charon 1186 06[ENC] <11> unknown attribute type INTERNAL_DNS_DOMAIN Nov 8 15:21:24 charon 1186 06[ENC] <11> parsed IKE_AUTH request 1 [ IDi N(INIT_CONTACT) N(MOBIKE_SUP) IDr CPRQ(ADDR DHCP DNS MASK ADDR6 DHCP6 DNS6 DOMAIN) N(ESP_TFC_PAD_N) N(NON_FIRST_FRAG) SA TSi TSr ] Nov 8 15:21:24 charon 1186 06[CFG] <11> looking for peer configs matching my.public.gateway.address[ACMERocketCars.dyndns.org]...my.mobile.ip.address[192.168.0.213] Nov 8 15:21:24 charon 1186 06[CFG] <con-mobile|11> selected peer config 'con-mobile' Nov 8 15:21:24 charon 1186 06[IKE] <con-mobile|11> initiating EAP_IDENTITY method (id 0x00) Nov 8 15:21:24 charon 1186 06[IKE] <con-mobile|11> received ESP_TFC_PADDING_NOT_SUPPORTED, not using ESPv3 TFC padding Nov 8 15:21:24 charon 1186 06[IKE] <con-mobile|11> peer supports MOBIKE Nov 8 15:21:24 charon 1186 06[IKE] <con-mobile|11> authentication of 'ACMERocketCars.dyndns.org' (myself) with RSA signature successful Nov 8 15:21:24 charon 1186 06[IKE] <con-mobile|11> sending end entity cert "CN=ACMERocketCars.dyndns.org, C=US, ST=California, L=San Francisco, O=ACME RocketCars" Nov 8 15:21:24 charon 1186 06[ENC] <con-mobile|11> generating IKE_AUTH response 1 [ IDr CERT AUTH EAP/REQ/ID ] Nov 8 15:21:24 charon 1186 06[ENC] <con-mobile|11> splitting IKE message (1664 bytes) into 2 fragments Nov 8 15:21:24 charon 1186 06[ENC] <con-mobile|11> generating IKE_AUTH response 1 [ EF(1/2) ] Nov 8 15:21:24 charon 1186 06[ENC] <con-mobile|11> generating IKE_AUTH response 1 [ EF(2/2) ] Nov 8 15:21:24 charon 1186 06[NET] <con-mobile|11> sending packet: from my.public.gateway.address[4500] to my.mobile.ip.address[51296] (1236 bytes) Nov 8 15:21:24 charon 1186 06[NET] <con-mobile|11> sending packet: from my.public.gateway.address[4500] to my.mobile.ip.address[51296] (500 bytes) Nov 8 15:21:24 charon 1186 06[NET] <con-mobile|11> received packet: from my.mobile.ip.address[51296] to my.public.gateway.address[4500] (112 bytes) Nov 8 15:21:24 charon 1186 06[ENC] <con-mobile|11> parsed IKE_AUTH request 2 [ EAP/RES/ID ] Nov 8 15:21:24 charon 1186 06[IKE] <con-mobile|11> received EAP identity 'user@domain.com' Nov 8 15:21:24 charon 1186 06[IKE] <con-mobile|11> initiating EAP_MSCHAPV2 method (id 0xE8) Nov 8 15:21:24 charon 1186 06[ENC] <con-mobile|11> generating IKE_AUTH response 2 [ EAP/REQ/MSCHAPV2 ] Nov 8 15:21:24 charon 1186 06[NET] <con-mobile|11> sending packet: from my.public.gateway.address[4500] to my.mobile.ip.address[51296] (112 bytes) Nov 8 15:21:24 charon 1186 06[NET] <con-mobile|11> received packet: from my.mobile.ip.address[51296] to my.public.gateway.address[4500] (160 bytes) Nov 8 15:21:24 charon 1186 06[ENC] <con-mobile|11> parsed IKE_AUTH request 3 [ EAP/RES/MSCHAPV2 ] Nov 8 15:21:24 charon 1186 06[ENC] <con-mobile|11> generating IKE_AUTH response 3 [ EAP/REQ/MSCHAPV2 ] Nov 8 15:21:24 charon 1186 06[NET] <con-mobile|11> sending packet: from my.public.gateway.address[4500] to my.mobile.ip.address[51296] (144 bytes) Nov 8 15:21:24 charon 1186 06[NET] <con-mobile|11> received packet: from my.mobile.ip.address[51296] to my.public.gateway.address[4500] (80 bytes) Nov 8 15:21:24 charon 1186 06[ENC] <con-mobile|11> parsed IKE_AUTH request 4 [ EAP/RES/MSCHAPV2 ] Nov 8 15:21:24 charon 1186 06[IKE] <con-mobile|11> EAP method EAP_MSCHAPV2 succeeded, MSK established Nov 8 15:21:24 charon 1186 06[ENC] <con-mobile|11> generating IKE_AUTH response 4 [ EAP/SUCC ] Nov 8 15:21:24 charon 1186 06[NET] <con-mobile|11> sending packet: from my.public.gateway.address[4500] to my.mobile.ip.address[51296] (80 bytes) Nov 8 15:21:24 charon 1186 06[NET] <con-mobile|11> received packet: from my.mobile.ip.address[51296] to my.public.gateway.address[4500] (112 bytes) Nov 8 15:21:24 charon 1186 06[ENC] <con-mobile|11> parsed IKE_AUTH request 5 [ AUTH ] Nov 8 15:21:24 charon 1186 06[IKE] <con-mobile|11> authentication of '192.168.0.213' with EAP successful Nov 8 15:21:24 charon 1186 06[IKE] <con-mobile|11> authentication of 'ACMERocketCars.dyndns.org' (myself) with EAP Nov 8 15:21:24 charon 1186 06[IKE] <con-mobile|11> IKE_SA con-mobile[11] established between my.public.gateway.address[ACMERocketCars.dyndns.org]...my.mobile.ip.address[192.168.0.213] Nov 8 15:21:24 charon 1186 06[IKE] <con-mobile|11> scheduling rekeying in 23424s Nov 8 15:21:24 charon 1186 06[IKE] <con-mobile|11> maximum IKE_SA lifetime 26304s Nov 8 15:21:24 charon 1186 06[IKE] <con-mobile|11> peer requested virtual IP %any Nov 8 15:21:24 charon 1186 06[CFG] <con-mobile|11> reassigning offline lease to 'user@domain.com' Nov 8 15:21:24 charon 1186 06[IKE] <con-mobile|11> assigning virtual IP 192.168.202.1 to peer 'user@domain.com' Nov 8 15:21:24 charon 1186 06[IKE] <con-mobile|11> peer requested virtual IP %any6 Nov 8 15:21:24 charon 1186 06[IKE] <con-mobile|11> no virtual IP found for %any6 requested by 'user@domain.com' Nov 8 15:21:24 charon 1186 06[CFG] <con-mobile|11> selected proposal: ESP:AES_CBC_256/HMAC_SHA2_256_128/NO_EXT_SEQ Nov 8 15:21:24 charon 1186 06[IKE] <con-mobile|11> CHILD_SA con-mobile{2} established with SPIs cdbc8d8f_i 0e359038_o and TS 192.168.200.0/24|/0 === 192.168.202.1/32|/0 Nov 8 15:21:24 charon 1186 06[ENC] <con-mobile|11> generating IKE_AUTH response 5 [ AUTH CPRP(ADDR DNS DNS SUBNET (27674) (27675)) N(ESP_TFC_PAD_N) SA TSi TSr N(MOBIKE_SUP) N(ADD_4_ADDR) N(ADD_4_ADDR) ] Nov 8 15:21:24 charon 1186 06[NET] <con-mobile|11> sending packet: from my.public.gateway.address[4500] to my.mobile.ip.address[51296] (352 bytes)

@viragomann You're a gentleman and a scholar! Thanks so much

@fazevedo How would that be any different than the actual IP address? You don't have to tell anyone what the domain name you chose is. There's literally no security concern any different than having a public IP.