2.2.1 multiple SAs and SPIs



  • After upgrading to 2.2.1 I am finding that IPSEC is having problems very similar to those in the 2.2 beta.    All works well until a connection disruption event occurs.  The original SA is not cleared, and a new one is created with a new SPI.  Once that happens, VPN traffic stops flowing through the connection.  If I stop the VPN service for a few minutes and restart it the problem goes away until the next event.

    I do not find any specific error messages in the log, however there are hundreds of the following sequence of messages:

    charon: 16[ENC] parsed INFORMATIONAL_V1 request 2221050085 [ HASH N(DPD_ACK) ]
    Mar 21 14:01:59 charon: 16[NET] received packet: from 111.1111.111.111[500] to 222.222.222.222500] (108 bytes)
    Mar 21 14:01:59 charon: 05[NET] sending packet: from 222.222.222.222[500] to 111.111.111.111[500] (108 bytes)
    Mar 21 14:01:59 charon: 05[ENC] generating INFORMATIONAL_V1 request 1804281962 [ HASH N(DPD) ]
    Mar 21 14:01:59 charon: 05[IKE] sending DPD request
    Mar 21 14:01:59 charon: 05[IKE] <con1000|53>sending DPD request</con1000|53>



  • Similar experience here.
    We updated 20 something routers to 2.2.1.  Glad we did not do all of them.
    The tunnels stop passing traffic.  Both ends show up with green under the VPN status but no traffic will pass.  It appears that this happens after the key life time expires.  They are all set to 28,00 now.

    The only way to get it going again is to bump the remote end.  Stop that connection then restart it.
    I had to do it to 10 of them this morning .  Good thing the network monitor tells us which tunnels will not ping.  Otherwise we would be flying blind.
    Unfortunately, I do not have he log files.  I only had time to bump the connections this morning.  This has been going on since we upgraded to 2.2.1.  Not going to do anything with the other 60 routers until we get his resolved.  Not enough time in the day to keep bumping tunnels that show as connected but do not pass traffic.
    If anybody has any suggestions for a resolution we are willing to give it a try. 
    Thanks



  • I've noticed this as well, and the only thing that seems to work is manually killing the extra SA's.  I've been mucking with settings for a few hours now without much luck finding a self-correcting solution.



  • From the other end of the connection I see the following that results in the numerous SAs/SPIs:

    Mar 23 18:49:11 racoon: INFO: purged ISAKMP-SA spi=ba3f2feec326295d:6b4b6d4ddae781e5.
    Mar 23 18:49:11 racoon: INFO: purged IPsec-SA spi=43140126.
    Mar 23 18:49:11 racoon: INFO: Unknown IPsec-SA spi=43140126, hmmmm?
    Mar 23 18:49:11 racoon: INFO: purged IPsec-SA spi=5593773.
    Mar 23 18:49:11 racoon: INFO: Unknown IPsec-SA spi=5593773, hmmmm?
    Mar 23 18:49:11 racoon: INFO: purged IPsec-SA spi=45729866.
    Mar 23 18:49:11 racoon: INFO: Unknown IPsec-SA spi=45729866, hmmmm?
    Mar 23 18:49:11 racoon: INFO: purged IPsec-SA spi=162775757.
    Mar 23 18:49:11 racoon: INFO: Unknown IPsec-SA spi=162775757, hmmmm?
    Mar 23 18:49:11 racoon: INFO: purged IPsec-SA spi=3237879351.
    Mar 23 18:49:11 racoon: INFO: Unknown IPsec-SA spi=3237879351, hmmmm?
    Mar 23 18:49:11 racoon: INFO: purging ISAKMP-SA spi=ba3f2feec326295d:6b4b6d4ddae781e5.
    Mar 23 18:49:11 racoon: [ssss Yyyy]: [69.69.69.69] INFO: DPD: remote (ISAKMP-SA spi=ba3f2feec326295d:6b4b6d4ddae781e5) seems to be dead.
    Mar 23 18:48:54 racoon: [ssss Yyyy]: INFO: IPsec-SA established: ESP 142.142.142.142[500]->69.69.69.69[500] spi=3485970490(0xcfc7b03a)
    Mar 23 18:48:54 racoon: [ssss Yyyy]: INFO: IPsec-SA established: ESP 142.142.142.142[500]->69.69.69.69[500] spi=140179270(0x85af746)
    Mar 23 18:48:54 racoon: ERROR: not matched
    Mar 23 18:48:54 racoon: [ssss Yyyy]: INFO: respond new phase 2 negotiation: 142.142.142.142[500]<=>69.69.69.69[500]



  • So we had this problem and changed to IKE v 2 and that has solved the multiple SA's.



  • I suspect that if PFSense is at v 2.2.1 on both ends the problem might not exist, but, at least in my case, I'm working with a bunch of end points that are still on 2.1.5 and racoon does not support IKEv2.



  • I have discovered that if the connection is interrupted for long enough the tunnels will be rebuilt.    I have IPSEC connections to 15 end points.  From a fresh restart the connections are all established and traffic flows with no problems.    However, if the connection is briefly interrupted or the rekey period is reached, additional child SA entries are created and while the status of the connections is "green lighted" traffice no long can pass through the connection.  Additional SA entries continue to be created (I have seen as many as 16 duplicate child entries).  At first, I would restart PFSense and the connections would be restored.  Then I discovered that merely stopping and restarting the IPSEC task would restore the connections.  But, it also appears that if the connection drops for a longer period of time, charon restarts the ipsec tunnels and traffic flows again.

    Charon floods the ipsec log with messages when the connection problem occurs,  but in the general log I see the following events in sequence.

    check_reload_status: Syncing firewall    (the problem begins)
    kernel: key_get: no SA found.  (message is repeated 50 - 100 times)    (traffic stops flowing)
    kernel: key_delete: no SA found
    kernel: key_get: no SA found.  (message is repeated 50 - 100 times)
      (sequence repeats itself)

    php-fpm[4818]: /rc.newipsecdns: MONITOR: GW has packet loss, omitting from routing group GWGroup1    (connectivity is lost)
    check_reload_status: Restarting ipsec tunnels    (connection is re-established, tunnels begin passing traffic)



  • @kitdavis:

    Charon floods the ipsec log with messages when the connection problem occurs,  but in the general log I see the following events in sequence.

    The log noise there is just because I left debug logging for IPsec cranked up on your system. That's under System>Advanced, Tunables, net.inet.ipsec.debug. You can delete that if you want to get rid of the excessive noise.

    I suspect your root problem, and anyone else's who's seeing rekeying issues is:
    https://forum.pfsense.org/index.php?topic=91627.0



  • Thanks - I figured it was a "left behind setting" and I searched for some time to find it to no avail.  I made the OLDSA entry and expect to be reporting shortly that it has resolved the IPSEC issue.  Thanks again.



  • @cmb:

    I suspect your root problem, and anyone else's who's seeing rekeying issues is:
    https://forum.pfsense.org/index.php?topic=91627.0

    This did indeed resolve all my IPSec tunnel issues.  Thanks for the heads-up. 8)



  • @cmb:

    I suspect your root problem, and anyone else's who's seeing rekeying issues is:
    https://forum.pfsense.org/index.php?topic=91627.0

    I'm still having connection issues after rekeying (incl. multiple SAs) with 2.2.5 at both ends. I understood that the workaround above shouldn't be required anymore. Is it sill?

    Thanks



  • @brevilo:

    I'm still having connection issue after rekeying (incl. multiple SAs) with 2.2.5 at both ends. I understood that the workaround above shouldn't be required anymore. Is it sill?

    No it's not. There are no longer any general issues along those lines (though any number of config issues could potentially result in symptoms like that). Start a new thread describing what you're seeing, and what your logs show.


Log in to reply