pfSense 2.5 to pfSense 2.5 IPsec tunnel fails to connect
-
Hi folks-
I'm encountering a pfSense 2.5 (10.0.2.8) to pfSense 2.5 (10.0.2.15) IPsec tunnel connection failure. These are freshly deployed VMs in VirtualBox using the NAT Network for both WANs to simulate internet. The two pfSense 2.5 VMs can ping each other fine, but I haven't had luck with an IPsec tunnel using mostly default settings. When I press connect, it flashes for a sec then remains disconnected. Do my logs yield any clues?
Feb 28 06:03:42 charon 76827 16[CFG] vici client 33 disconnected Feb 28 06:03:42 charon 76827 06[CFG] vici client 33 requests: list-sas Feb 28 06:03:42 charon 76827 14[CFG] vici client 33 registered for: list-sa Feb 28 06:03:42 charon 76827 16[CFG] vici client 33 connected Feb 28 06:03:42 charon 76827 14[CFG] vici client 32 disconnected Feb 28 06:03:42 charon 76827 16[CFG] vici client 31 disconnected Feb 28 06:03:42 charon 76827 16[CFG] vici initiate CHILD_SA 'con1000' Feb 28 06:03:42 charon 76827 16[CFG] vici client 32 requests: initiate Feb 28 06:03:42 charon 76827 09[CFG] vici terminate IKE_SA 'con1000' Feb 28 06:03:42 charon 76827 09[CFG] vici client 31 requests: terminate Feb 28 06:03:42 charon 76827 09[CFG] vici client 32 registered for: control-log Feb 28 06:03:42 charon 76827 09[CFG] vici client 32 connected Feb 28 06:03:42 charon 76827 09[CFG] vici client 31 registered for: control-log Feb 28 06:03:42 charon 76827 09[CFG] vici client 31 connected Feb 28 06:03:40 charon 76827 14[CFG] vici client 30 disconnected Feb 28 06:03:40 charon 76827 07[CFG] vici client 30 requests: list-sas Feb 28 06:03:40 charon 76827 14[CFG] vici client 30 registered for: list-sa Feb 28 06:03:40 charon 76827 14[CFG] vici client 30 connected Feb 28 06:03:38 charon 76827 07[CFG] vici client 29 disconnected Feb 28 06:03:38 charon 76827 14[CFG] updated vici connection: con100000 Feb 28 06:03:38 charon 76827 14[CFG] id = 10.0.2.15 Feb 28 06:03:38 charon 76827 14[CFG] class = pre-shared key Feb 28 06:03:38 charon 76827 14[CFG] remote: Feb 28 06:03:38 charon 76827 14[CFG] id = 10.0.2.8 Feb 28 06:03:38 charon 76827 14[CFG] class = pre-shared key Feb 28 06:03:38 charon 76827 14[CFG] local: Feb 28 06:03:38 charon 76827 14[CFG] if_id_out = 0 Feb 28 06:03:38 charon 76827 14[CFG] if_id_in = 0 Feb 28 06:03:38 charon 76827 14[CFG] proposals = IKE:AES_CBC_256/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_2048 Feb 28 06:03:38 charon 76827 14[CFG] rand_time = 2880 Feb 28 06:03:38 charon 76827 14[CFG] over_time = 2880 Feb 28 06:03:38 charon 76827 14[CFG] rekey_time = 25920 Feb 28 06:03:38 charon 76827 14[CFG] reauth_time = 0 Feb 28 06:03:38 charon 76827 14[CFG] keyingtries = 1 Feb 28 06:03:38 charon 76827 14[CFG] unique = UNIQUE_REPLACE Feb 28 06:03:38 charon 76827 14[CFG] childless = 0 Feb 28 06:03:38 charon 76827 14[CFG] fragmentation = 2 Feb 28 06:03:38 charon 76827 14[CFG] dpd_timeout = 60 Feb 28 06:03:38 charon 76827 14[CFG] dpd_delay = 10 Feb 28 06:03:38 charon 76827 14[CFG] encap = 0 Feb 28 06:03:38 charon 76827 14[CFG] dscp = 0x00 Feb 28 06:03:38 charon 76827 14[CFG] aggressive = 0 Feb 28 06:03:38 charon 76827 14[CFG] mobike = 0 Feb 28 06:03:38 charon 76827 14[CFG] ppk_required = 0 Feb 28 06:03:38 charon 76827 14[CFG] ppk_id = (null) Feb 28 06:03:38 charon 76827 14[CFG] send_cert = CERT_SEND_IF_ASKED Feb 28 06:03:38 charon 76827 14[CFG] send_certreq = 1 Feb 28 06:03:38 charon 76827 14[CFG] remote_port = 500 Feb 28 06:03:38 charon 76827 14[CFG] local_port = 500 Feb 28 06:03:38 charon 76827 14[CFG] remote_addrs = 10.0.2.15
Thanks!
-
To ensure you have all of the current known and fixed IPsec issues corrected, You can install the System Patches package and then create entries for the following commit IDs to apply the fixes:
ead6515637a34ce6e170e2d2b0802e4fa1e63a00
#1143557beb9ad8ca11703778fc483c7cba0f6770657ac
#1143510eb04259fd139c62e08df8de877b71fdd0eedc8
#11442ded7970ba57a99767e08243103e55d8a58edfc35
#11486afffe759c4fd19fe6b8311196f4b6d5e288ea4fb
#114872fe5cc52bd881ed26723a81e0eed848fd505fba6
#11488f731957f945af90d6a75f0e33f91a440a6a55736
#11526
-
Hi,
I also need to install this one:- https://github.com/pfsense/pfsense/commit/4e5857b656c7bfd59efadbb9a124876a5516c7df.patch
and have to setup the Dead Peer Detection (DPD) to Delay 60 and Max failures 5.
Cheers
Marco -
@marco42 said in pfSense 2.5 to pfSense 2.5 IPsec tunnel fails to connect:
Hi,
I also need to install this one:- https://github.com/pfsense/pfsense/commit/4e5857b656c7bfd59efadbb9a124876a5516c7df.patch
That is the same as
ead6515637a34ce6e170e2d2b0802e4fa1e63a00
which is in the list above already, but the commit you linked is to master andead6515637a34ce6e170e2d2b0802e4fa1e63a00
is to RELENG_2_5_0 -- they both apply cleanly but it's better to use the one from the appropriate branch. -
Hi,
Just wanted to give some feedback that after the upgrade of one side of a VTI-connection between two pfSenses, IPsec failed with "trap not found, unable to acquire reqid".
Reconfiguration did not help, applying all the patches above did. I will revert to a VM snapshot anyway and wait for a maintenance release.
-
@morlock said in pfSense 2.5 to pfSense 2.5 IPsec tunnel fails to connect:
"trap not found, unable to acquire reqid"
That isn't a fatal error, it's normal with VTI.
-
@jimp Ok, then it didn't connect for some other reason that is not logged with my settings. I hadn't really time to look into it more detailed. I had applied the first six patches first, but only 731957f945af90d6a75f0e33f91a440a6a55736 eventually made a difference.
-
@jimp Thank you for compiling this convenient list of patches! Can I apply them to pfSense Plus 21.02 devices, or are they only for pfSense CE 2.5? I'm trying to make a basic IPsec tunnel between an SG-1100 and a homebuilt box, and cannot get it to work with those respective software versions. Forgive the question, as I have never tried using patches before. Also, do I need to revert the patches when the next upgrade becomes available, or is it safe to apply the upgrade on top of them? Thank you again!
-
Just a question, why is this not in an official fix or release yet? I have 3 firewalls spread out between my parents, my friends house and my own running the community edition. I don't understand why this hasn't been fixed yet. My ipsec tunnels are down even after a fresh install. I'm just confused why it is taking so long for this to become a "fix".
-
Making a new release takes time, effort, and testing. There are still numerous things we're actively investigating.
-
After upgraded to version 21.02-RELEASE-p1, my Netgate XG-7100 kept getting disconnected to a remote Cisco RV042 (even though it showed as connected). The IPsec tunnel had been working smoothly for a year without any hiccup now decided to act up. There is no pattern when it would stop the connection, most of the time just within couple minutes. I applied the 7 patches as advised about but still no luck. My tunnel is using IKEv1, AES_CBC (128), HMAC_SHA1_96, PRF_HMAC_SHA1, MODP_768. I am under the pressure to get it back on track. Please help! Thank you.
-
I finally got my IPsec tunnel to work without interruption on my Netgate XG-7100 by turning of the Hardware Crypto. I believe it is a work around solution since the Hardware Crypto has to be off.
-
@beejee said in pfSense 2.5 to pfSense 2.5 IPsec tunnel fails to connect:
I finally got my IPsec tunnel to work without interruption on my Netgate XG-7100 by turning of the Hardware Crypto. I believe it is a work around solution since the Hardware Crypto has to be off.
Which hardware crypto option did you have enabled on there? If it was AES-NI, that sounds similar to an issue we're already tracking. If it's not, it could be a different problem. On the XG-7100 you can switch from AES-NI to QAT which should be equal to or faster in performance and potentially more stable as the issues we're aware of only affect AES-NI at the moment.
-
@jimp said in pfSense 2.5 to pfSense 2.5 IPsec tunnel fails to connect:
Which hardware crypto option did you have enabled on there?
You were right. It was "AES-NI and BSD Cryto Device (aesni,cryptodev)" option. I believe it would be the best option for my appliance overall.
I will try the "QAT" as your suggested sometime tonight. Thank you!
-
@jimp said in pfSense 2.5 to pfSense 2.5 IPsec tunnel fails to connect:
On the XG-7100 you can switch from AES-NI to QAT which should be equal to or faster in performance and potentially more stable
Yes my XG-7100 IPsec tunnel is working smoothly with "Intel QuickAssist (QAT)" on. My CPU Type is "Intel(R) Atom(TM) CPU C3558 @ 2.20GHz". I actually don't know where to check if my IPsec tunnel is really using the QAT since I didn't see it is listed anywhere in the Dashboard. The Dashboard mentioned that "AES-NI CPU Crypto: Yes (inactive)" instead.
-
The next update will show QAT on the dashboard properly, but for now you can check:
- That the module is loaded:
: kldstat | grep qat 5 1 0xffffffff84322000 146e0 qat.ko 6 1 0xffffffff84337000 9f521 qat_c3xxxfw.ko
- That the device is consuming interrupts (will increase as traffic is encrypted/decrypted through IPsec):
: vmstat -i | egrep 'total|qat' interrupt total rate irq300: qat0 489041 0
- That the module is loaded:
-
@jimp Awesome! I think my IPsec is really utilizing the QAT. Thank you so much!
I downloaded a large file through the tunnel and checked the stat::kldstat | grep qat 3 1 0xffffffff83f22000 146e0 qat.ko 4 1 0xffffffff83f37000 9f521 qat_c3xxxfw.ko :vmstat -i | egrep 'total|qat' interrupt total rate irq293: qat0 43576 1 irq294: qat0 27168 1 irq295: qat0 7909 0 irq296: qat0 18777 0 :vmstat -i | egrep 'total|qat' interrupt total rate irq293: qat0 396393 9 irq294: qat0 39664 1 irq295: qat0 19005 0 irq296: qat0 47598 1
-
@jimp Thanks I had problem with tunnel to a Cisco router where I could ping but then sending any traffic through would kill the tunnel. Switching my 5100 to QAT seems to have fixed this issue.
I have another tunnel between the SG-5100 and an older SG-2220.. enabling QAT on the 2200 is slower than using AES-NI - iperf3 test are 220Mbps with AES-NI and only around 100Mbps on QAT when I enable it on the SG-2220.
-
@jimp Guessing the Dashboard display for QAT or other crypto modules didn't make it to 21.02.2 - at least my XG-7100's still show 'AES-NI CPU Crypto: Yes (inactive) - no mention of QAT anywhere that I can find on a Dashboard widget etc.
-
@danjeman said in pfSense 2.5 to pfSense 2.5 IPsec tunnel fails to connect:
@jimp Guessing the Dashboard display for QAT or other crypto modules didn't make it to 21.02.2 - at least my XG-7100's still show 'AES-NI CPU Crypto: Yes (inactive) - no mention of QAT anywhere that I can find on a Dashboard widget etc.
It's there on 21.05 snapshots, didn't make it into 21.02.2.