IPSEC Cisco ASA Issues
-
Hi,
We have a number of IPSEC VPNs going to a variety of endpoints, with different vendors at the remote ends, which all work perfectly, apart from two VPNs which go to Cisco ASA firewalls.The issue we're seeing with these devices is that during the phase 2 re-key, our pfsense firewall seems to do a re-key of the tunnel successfully, then fail to use the new key, and it also fails to delete the old key, leaving our firewall in limbo until we restart the IPSEC service, which obviously kills all the VPN connections. We have tried just restarting the two Cisco VPNs in question, but this doesn't work. Only a full service restart seems to quickly resolve the issue (Until the next phase 2 re-key), or we can leave the service to eventually time out (Which can take a considerable amount of time, and so is a very impractical solution).
We have tried these VPNs as IKEv1 and IKEv2 tunnels, short and long re-key times, as well as a variety of different encryption protocols and levels on both phases, as well as enabling and disabling the Unity support on our pfsense, but nothing seems to change their behaviour. I have also tried every revision of pfsense from 2.3 onwards in an attempt to see if a revision regression would solve the issue, but with no success. Currently, we are on 2.4.4-p1 (I've just noticed 2.4.4-p2 is now available) and we're still seeing the issue.
We've been in discussions with the remote side as to how we can proceed with this, but they have firmly refused to escalate the matter with Cisco, claiming that the fault lies entirely with pfsense, which I'm finding most unhelpful. They're even refusing to tell me the version of the ASA software that's running to see if I can cross reference anything.
I've looked at this support site, as well as the StrongSWAN forums, but I can't see a recent issue with the software, in fact only finding one similar fault reported on the StrongSWAN bug tracker from several years back, but it being dismissed as a Cisco fault by their developers.Currently I'm having to run a cron job that restarts the IPSEC service every six hours as that is the only way we can maintain connectivity to the remote sites. Whilst obviously not the correct solution, it's the only one I've currently got at the moment.
For reference, a service critical application hangs off these VPNs, and it's not something we can source from anywhere else, so we can't just move endpoints to a new provider. I'm stuck with these VPNs, and the frankly unhelpful support from the remote side.
So, has anyone seen anything like this happen with IPSEC tunnels to Cisco ASA hardware, and if you've found a better solution, or even the answer as to why this is happening?
Thanks very much in advance.
Cheers,
Monty -
Not really. Works fine with ASAs.
They're even refusing to tell me the version of the ASA software that's running
That's pretty arrogant.
The beginning of finding the answer probably lies in the IPsec logs.
Any specific information they provided for what parameters/algorithms/etc they expect might also help, along with shots of your P1 and P2 config pages.
If they allow IKEv2 and the tunnel comes up I would stick with that. If you have more than one P2 just be sure to enable Split Connections in the Phase 1 to work around limitations in the ASA.
-
Hi Derelict,
Yes, I agree, it's pretty arrogant of them. I could go on at length about how lazy and incompetent their network admin is, but that's not the issue here, but it doesn't help resolving the issue I have when I can't get any logs out of them, which has left me trying to solve a puzzle with only half the clues, and why I ended up posting here.
Anyway, thank you so much for your help on this. Your comments have given me a new avenue to investigate.
The split connections were not enabled on either tunnel as I didn't think it applied in this scenario (Two IPSEC VPNs going to different endpoints with a phase 1 config, and a single phase 2 config), but after you mentioning this, I started to look into Cisco ASA tunnels with split connections and to be honest, that's thrown up a whole lot more info around this that looks very, very, similar to the issues I'm seeing, which was the old phase 2 tunnel would re-key, a new key would be generated for the replacement phase 2 tunnel, then neither would be used with the firewall just sitting there unable to delete the old key, and the logs telling me the remote end wasn't accepting the new key either.
I've enabled the split connections tick box on both VPNs and commented out the IPSEC restart commands in the crontab to see what transpires. At best, it works, and that would be great. At worst, at least I have a recent set of IPSEC logs with the debug output showing what's happening when things break.
Once again, thanks for the pointers in the right direction with this.
Cheers,
Monty -
Thought I'd put a final update into this as the working solution.
After getting a couple of Cisco ASA 5506 units on site and creating exact copies of the IPSEC VPNs that I was having issues with, and running these test VPNs over other IP addresses on our twin internet links back to the firewall, I couldn't get the damn things to fail like the original links. They worked for a good couple of weeks without a dropped packet, even although I'd put the phase 1&2 settings to rekey every hour and half hour respectively in the hopes of generating enough debug traffic on both sides to see where the issue lay.
Went back to the outside agency in question, presented them with my findings, to be told point blank again, that the fault was on our side. After a pretty gritted teeth conversation with their network admin, he let slip that their configuration had a data transfer limit on both VPNs, where it would rekey every 4Gb of traffic. This was the first I'd heard about it, the agreed VPN documentation didn't have this noted, and PfSense IPSEC configs didn't have this in there (I don't think the StrongSWAN version currently in use on PfSense has this as an option anyway). After insisting that this be removed, both VPNs haven't failed since.
Cheers,
Monty
