IPsec Authentication Fails - "Constraint Required Public Key"

[Balthxzar]

Hi,

I'm trying to setup a site to site VPN between a pfsense router and a mikrotik router, all seems to go well and the connection is started however quickly closes with the message "constraint requires public key authentication, but pre-shared key was used"

I've tried to look this error message up but I get no relevant results, on the mikrotik side I simply get "got fatal error: AUTHENTICATION_FAILED"

Jan 17 18:05:09	charon		12[NET] <540> received packet: from "REMOTE IP"[4500] to "LOCAL IP"[4500] (448 bytes)
Jan 17 18:05:09	charon		12[ENC] <540> parsed IKE_SA_INIT request 0 [ N(NATD_D_IP) N(NATD_S_IP) No KE SA ]
Jan 17 18:05:09	charon		12[CFG] <540> looking for an IKEv2 config for "LOCAL IP"..."REMOTE IP"
Jan 17 18:05:09	charon		12[CFG] <540> candidate: %any...%any, prio 24
Jan 17 18:05:09	charon		12[CFG] <540> candidate: "LOCAL IP"..."REMOTE IP", prio 3100
Jan 17 18:05:09	charon		12[CFG] <540> found matching ike config: "LOCAL IP"..."REMOTE IP" with prio 3100
Jan 17 18:05:09	charon		12[IKE] <540> "REMOTE IP" is initiating an IKE_SA
Jan 17 18:05:09	charon		12[IKE] <540> IKE_SA (unnamed)[540] state change: CREATED => CONNECTING
Jan 17 18:05:09	charon		12[CFG] <540> selecting proposal:
Jan 17 18:05:09	charon		12[CFG] <540> proposal matches
Jan 17 18:05:09	charon		12[CFG] <540> received proposals: IKE:AES_CBC_256/AES_CBC_192/AES_CBC_128/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_2048
Jan 17 18:05:09	charon		12[CFG] <540> configured proposals: IKE:AES_CBC_256/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_2048
Jan 17 18:05:09	charon		12[CFG] <540> selected proposal: IKE:AES_CBC_256/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_2048
Jan 17 18:05:09	charon		12[IKE] <540> remote host is behind NAT
Jan 17 18:05:09	charon		12[ENC] <540> generating IKE_SA_INIT response 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) N(MULT_AUTH) ]
Jan 17 18:05:09	charon		12[NET] <540> sending packet: from "LOCAL IP"[4500] to "REMOTE IP"[4500] (440 bytes)
Jan 17 18:05:10	charon		12[NET] <540> received packet: from "REMOTE IP"[4500] to "LOCAL IP"[4500] (432 bytes)
Jan 17 18:05:10	charon		12[ENC] <540> parsed IKE_AUTH request 1 [ IDi AUTH N(INIT_CONTACT) SA TSi TSr ]
Jan 17 18:05:10	charon		12[CFG] <540> looking for peer configs matching "LOCAL IP"[%any]..."REMOTE IP"[sep-net-IPsec-mkt]
Jan 17 18:05:10	charon		12[CFG] <540> candidate "bypasslan", match: 1/1/24 (me/other/ike)
Jan 17 18:05:10	charon		12[CFG] <bypasslan|540> selected peer config 'bypasslan'
Jan 17 18:05:10	charon		12[IKE] <bypasslan|540> authentication of '"site 2 id"' with pre-shared key successful
Jan 17 18:05:10	charon		12[CFG] <bypasslan|540> constraint requires public key authentication, but pre-shared key was used
Jan 17 18:05:10	charon		12[CFG] <bypasslan|540> selected peer config 'bypasslan' unacceptable: non-matching authentication done
Jan 17 18:05:10	charon		12[CFG] <bypasslan|540> no alternative config found
Jan 17 18:05:10	charon		12[ENC] <bypasslan|540> generating IKE_AUTH response 1 [ N(AUTH_FAILED) ]
Jan 17 18:05:10	charon		12[NET] <bypasslan|540> sending packet: from "LOCAL IP"[4500] to "REMOTE IP"[4500] (80 bytes)
Jan 17 18:05:10	charon		12[IKE] <bypasslan|540> IKE_SA bypasslan[540] state change: CONNECTING => DESTROYING

I've posted the log entry for a single connection event, this repeats constantly with no change, I have scrubbed my IP addresses and site ids, however everything else is left as-is,

thanks in advance for the help,

Balthxzar

[Balthxzar]

I was unable to get this working, despite being sure my config was correct, I switched to using RSA and it is now working perfectly, so I am beginning to wonder if the fault just arose from trying to use PSK with different vendors hardware.

[lst_hoe]

@balthxzar We also had this problem and it turns out that the "bypasslan" peer config is used when we have no remote/own ID matching in phase 1. The "bypasslan" config is only used if in the advanced settings the following is active:

Auto-exclude LAN address
Enable bypass for LAN interface IP Exclude traffic from LAN subnet to LAN IP address from IPsec.

As soon as this was disabled our peer config selection failed. With fixing our IDs we got the correct "peer config selection" and PSK worked as expected.