IPSec unstable since upgrade to 2.2
-
I am also facing major issues with IPSec since upgrading from 2.1.5 to 2.2
I have a tunnel between our office and our cloud provider with 5 phase 2 entries. It is configured with PSK, 3DES, SHA1 and DH Group 2.
It is completely random which of the 5 will work and even sometimes all 5 work together but very seldom. Stopping and starting IPSec sometimes has an effect on which links are up and which are down. HAving the link up does not guarantee that it will work though. Links show up but no traffic goes over them. I have tried to remove everything and reconfigure it again but still no luck.
Just to give some additional information, it is running on a Supermicro D525 on a USB since the new OS will not install to the discs. Tried every option in the Bios but after formatting and setting volumes it fails with Error Code 19. Only way to get it back was to run it off USB. So all in all a very bad experience upgrading to 2.2
Let me know what information I need to send for the IPSec debugging. Major issue for me right now.
Attached is a screen shot of the config and a screen shot of the status. You can see 3 up but only 1 able to send and receive traffic.
I have 2 other boxes (different HW) all running fine in other offices around the world, but unfortunately this is our main office and 24 hours of broken internet so far!
 -
Upon further hacking, IT seems like there cannot be more than one Phase 2 entry working at the same time. It shows more than one as being up but the sent bytes never increases on all but one of the Phase 2 entries. Force closing 1 sometimes activates one of the others. Sometimes it requires a force close of all of them.
-
Switch to IKEv2 and try again
-
Hi Doc
Do both sides have to be set to V2? I tried V2 but it was not able to connect. I can log a request with our provider to change the other end to V2 if needed. Do you know if this is a solution or if it is just something I should try?
-
Yes. Both sides need IKEv2 obviously. And make sure to stop and start IPsec on both sides, or better reboot.
-
It's a solution if the other side can change to IKEv2. That particular circumstance with multiple P2s and IKEv1 is exactly the one noted in the upgrade guide for which you shouldn't upgrade. IKEv2 has no such issues though.
-
Thanks Doc and cmb. Have put in a request to change to IKEv2. Will update the thread once it is done.
-
Can you also try this change.
On /var/etc/ipsec/strongswan.conf
change this value
threads = 16to
threads = 1and see if that stabilizes it.
-
Changed to 1 thread and restarted ipsec. Will monitor. Version still IKEv1. Waiting for our provider to change to confirm if they support v2 and if they can change.
-
Changed to 1 thread and restarted ipsec. Will monitor. Version still IKEv1. Waiting for our provider to change to confirm if they support v2 and if they can change.
Consider that the config file is generated automatically when the service is restarted through the GUI, make sure that the change you made is really applied
-
Hi Georgeman
So the config reverted back to 16. How do I restart it without the config being rebuilt?
-
So the config reverted back to 16. How do I restart it without the config being rebuilt?
You can edit /etc/inc/vpn.inc to change the source that generates that config file.
I'm not sure that'll do much if anything for you, that's something I'll test out in a known-problematic circumstance and see if it helps. If you can get the other end to switch to IKEv2, that's preferable in general (even if v1 wasn't problematic), so that's still the best option if it can be done.
-
Changing to IKEv2 does not help. I am still able to only have 1 Phase 2 route working at the same time. Which one depends on which gets traffic first.
I think the only solution is to fall back to 2.1.5 for now. IPSec seems quite broken for now.
-
Changing to IKEv2 does not help. I am still able to only have 1 Phase 2 route working at the same time. Which one depends on which gets traffic first.
One caveat that may be causing you issues in this circumstance, if you hit the + to the right of a P2 in 2.2 to create a new one based on an existing, it wrongly uses a duplicate ID, which might create problems along those lines. If you still have it up, delete all your P2s, and add them back one by one without duplicating any of them.
Outside of that circumstance, what you're describing isn't possible with IKEv2 as it doesn't have multiple child SAs, if one is up they're all up.
-
Hi cmb
I did use the + originally so I removed everything and redefined it again. The same problem persists. Attached are screen shots of the config and status. IKEv2 is configured on both ends. 2 Phase 2 entries and only 1 will work at a time. Which one depends on which send traffic first.
 -
Can you show your /var/etc/ipsec/ipsec.conf?
-
Sent you a PM with the details ermal.
-
All firewall rules are correct?
If yes than can you please execute from diagnostic->command sysctl net.inet.ipsec.debug=0xffffffEstablish the vpn send traffic on the not working path and see if anything comes out on dmesg -a command?
-
Hi @All
i have a similar problem after upgrade to 2.1.5 to 2.2 in one of my pfs boxes.
Previous ipsec tunnel between these 2 boxes (with P1 both aggressive mode) works perfectly
Please note that both PFSs are BEHIND a firewallAfter upgrade, old confs do not work anymore. No connection, P1 never established. Only this error on PFS 2.1.5 -> "ERROR: phase1 negotiation failed due to time up"
After a lot of attemps, I finally found a working configuration.
My actual confs, BOLD main changes after upgrade in OfficeB's PFS
Office A - pfs 2.1.5
P1
Mutual PSK
Main Mode (was aggressive)
My Identifier -> My IP Address
Peer Identifier -> Peer IP Address
Enc Alg AES 256Bit
Hash Alg SHA1
DH KeyGroup 2 (1024bit)
Lifetime 28800Nat-T DISABLE <- leave to ENABLE do not work, P1 established, P2 maybe (sorry, I don't remember)
DPD Enabled (10 Secs,5 retries)(two P2 phases)
P2 Protocol ESP
P2 Transforms 3DES
P2 Auth Methods MD5,SHA1
PFS key group OFF
Lifetime 3600Office B - pfs 2.2
P1
Key Exchange Version -> V1
Mutual PSK
Main Mode (was aggressive)
My Identifier -> My IP Address
Peer Identifier -> IP Address -> 192.168.46.10 <– this is WAN IP address of OfficeA PFS
Enc Alg AES 256Bit
Hash Alg SHA1
DH KeyGroup 2 (1024bit)
Lifetime 28800Nat-T AUTO
DPD Enabled (10 Secs,5 retries)(two P2 phases)
P2 Protocol ESP
P2 Transforms 3DES
P2 Auth Methods MD5,SHA1
PFS key group OFF
Lifetime 3600This was a message on PFS 2.2 box that give me an hint:
charon: 15[IKE] IDir '192.168.46.10' does not match to 'XX.XXX.XX.XXX'
XX.XXX.XX.XXX is public IP address of officeA ( Remote Gateway in OfficeB conf)
I hope this can be helpful for someone else
Simone
-
Or on pfSense 2.2 you have to set the proper identifier by choosing ip address and setting the private ip address of pfSense on the remote end.
your example is a clear id mismatch problem on configuration.
-
Adding this from another thread since it's been the root cause of a few issues and seemingly the ones in this thread as well.
One relevant change in behavior we've seen is racoon didn't seem to care whether the ID it was sent matched if it had a match for the IP where the traffic was actually being originated. So for instance if you have the remote end behind NAT set to use "My IP Address" for its identifier, it puts the private WAN IP in there as its identifier. racoon would still find it by matching the public IP where the request was initiated (probably technically incorrect). But strongswan has to find a match to the ID being sent by the other side, it won't fall back to "eh, you're coming from X IP and I have a match there, so that's good enough."
-
All firewall rules are correct?
I am fairly sure they are since either Phase 2 will work without me changing anything just depending on which one I initiate traffic over first. Send a ping via Phase 2 No1 and it is active and working and No2 is not. Only shows Bytes In and No Bytes Out. Send a ping via Phase 2 No2 first and it works with No1 failing and showing Bytes In and no Bytes Out.
sysctl net.inet.ipsec.debug=0xffffff and dmesg -a
This produced no difference in the output before or after trying to send traffic on the failed route or after restarting ipsec.
-
Ok the last thing to verify is if you see the packets with tcpdump/packet capture on enc0 interface on both cases be it receiving a reply or not.
If you can see both ping requests through enc when sending traffic but not receive reply would mean some issues with phase2 to be looked at. -
Thanks everyone for all the help but in the end I am going to have to go back to 2.1.5. 2.2 is just not ready. I have 3 show stoppers that I see no resolution for:
Unable to boot with Error Code = 19 although I am able to format and install it just fails to boot. (This is a new SuperMicro D525 so should work)
More than 1 Phase 2 in a IPSec Site to Site will not work on either IKEv1 or IKEv2.
IPSec traffic regularly stops and the only solution is to bounce the connection. -
I am getting to the same conclusion, I think. Hoping for a 2.2.1 soon which hopefully fixes some of those issues :(
-
If you would be able to test this last tweak.
Just disable reauthentication on the phase1 advanced settings so only a normal rekey is performed rather than a full reauthentication.
-
This change has made no difference. If anything I think it is worse. There are now Phase 2 routes in the status that are not in the configuration. IPSec (Strongswan) is truly broken. Really sad as pfSense was a great product before 2.2.
I have looked for the 2.1.5 download but I cannot find it. Anyone know where I can find it before I have to switch to a different product?
-
That's what backup are for :)
-
I have a config backup but not the install since I installed a version some revisions back. I could put that one up and spend time going from one version to the next or just install IPCop with the latest stable release. Since 2.2 is not a stable release there should be at least 1 version back available for download.
-
there should be at least 1 version back available for download.
http://files.nyi.pfsense.org/mirror/downloads/
-
I have also trouble with IPSec since upgrading from 2.1.5 to 2.2
My problem is that most of the tunnels are going down after the phase 1 lifetime expires. Restart of IPSec service does not help. I have to restart the whole firewall. After reboot all tunnels coming up automatically until again phase 1 lifetime expires…
Have anybody some idea?? THANKS!!
-
http://files.nyi.pfsense.org/mirror/downloads/
Thanks doc.
twaldorf, I found this happening too. Most of the time the phase 2 would come up after disconnecting and reconnecting the Phase 1. Note that if you have more than 1 Phase 2, only one of them will actually work even though the other show as up. Have you tried stopping and starting IPSec itself? Not just the Phase 1. I also use that a few times each day to get the link working again. I had to merge a few of my ip ranges at the various offices around the world to use a single range so that I could delete all the Phase 2s except 1.
-
I restarted the whole IPSec service. That didn't help. Also all (!) my tunnels have more than 1 phase 2, but only some of them are going down after phase 1 lifetime expires. For me it is no solution to reduce the number phase 2 because we have also an OpenVPN server running and want to allow all OpenVPN users to use the IPSec connections which is only possible if you tunnel the OpenVPN range to a second phase 2 in IPSec.
-
I just noticed something! The tunnels which are going down after phase 1 lifetime expires have ENABLED dead peer detection. The ones which are still alive after phase 1 lifetime expires have it DISABLED !
So it could be that there is an issue with DPD in pfSense 2.2 ???
-
DPD is already disabled on all my tunnels
-
DPD is already disabled on all my tunnels
The problem here is… that people with broken IPsec yet again recycled a thread for dumping all kinds of different issues here. There's is no generic "IPSec unstable" issue with a single cause.
Unless you are 300% sure you have exactly identical issue with the OP, kindly start your own thread, providing logs and relevant IPsec configuration bits.
-
There's is no generic "IPSec unstable" issue with a single cause.
For sure there IS SOME KIND of issue with IPSec after upgrading from 2.1.5 to 2.2
If have MANY different tunnels with different settings to all kinds of other firewalls (incl. pfSense, Cisco, WatchGuard, etc., etc., etc.). After upgrading 12 of 19 tunnels now have issues.
-
I have already started my own thread, but it's stalling.
Can't hurt to see what others do to find a solution for this -
For sure there IS SOME KIND of issue with IPSec after upgrading from 2.1.5 to 2.2
Please, read again. You won't get any issue fixed by posting generic rants and hijacking other people's threads with problems caused by something different (or caused by something unknown since people actually don't post any relevant info usable for debugging either.)
-
I have posted logs requested here: https://forum.pfsense.org/index.php?topic=87943.0