IPSec tunnel problems after pfSense 2.2.3 upgrade
-
Hi,
I have upgraded my pfSense full-installation on my SG-2440 unit from version 2.2.2 to version 2.2.3.
The upgrade was successful, but now my two IPSec LAN-to-LAN tunnels are not working anymore.When I log with "tcpdump -i enc0" I can see the traffic going into the tunnel, but no traffic is returning anymore.
The System/IPSec logs are without errors or warnings and the tunnels are up and running.
Both sides do not show IPSec errors, so it looks like the firewall is blocking the incomming traffic thru the tunnel.
I have tried allow-any-any rules in the IPSec rule segment but that doesn't solve anything either.I have also deleted the tunnels and recreated them, but without any success.
Does anyone have a suggestion how to solve this?
pfinfo shows:
enc0
Cleared: Thu Jun 25 12:07:39 2015
References: 6
In4/Pass: [ Packets: 0 Bytes: 0 ]
In4/Block: [ Packets: 0 Bytes: 0 ]
Out4/Pass: [ Packets: 833 Bytes: 69592 ]
Out4/Block: [ Packets: 0 Bytes: 0 ]
In6/Pass: [ Packets: 0 Bytes: 0 ]
In6/Block: [ Packets: 0 Bytes: 0 ]
Out6/Pass: [ Packets: 0 Bytes: 0 ]
Out6/Block: [ Packets: 0 Bytes: 0 ]ipsec.conf:
config setup
uniqueids = yes
charondebug=""conn con2000
fragmentation = yes
keyexchange = ikev1
reauth = yes
forceencaps = no
mobike = no
rekey = yes
installpolicy = yes
type = tunnel
dpdaction = restart
dpddelay = 30s
dpdtimeout = 180s
auto = route
left = XXX.XXX.XXX.XXX
right = XXX.XXX.XXX.XXX
leftid = XXX.XXX.XXX.XXX
ikelifetime = 21600s
lifetime = 3600s
ike = aes128-sha1-modp1024!
esp = aes256-sha1-modp1024,aes192-sha1-modp1024,aes128-sha1-modp1024!
leftauth = psk
rightauth = psk
rightid = XXX.XXX.XXX.XXX
aggressive = no
rightsubnet = 192.168.0.0/24
leftsubnet = 192.168.1.0/24conn con1000
fragmentation = yes
keyexchange = ikev1
reauth = yes
forceencaps = no
mobike = no
rekey = yes
installpolicy = yes
type = tunnel
dpdaction = clear
dpddelay = 10s
dpdtimeout = 60s
auto = add
left = XXX.XXX.XXX.XXX
right = XXX.XXX.XXX.XXX
leftid = XXX.XXX.XXX.XXX
ikelifetime = 21600s
lifetime = 3600s
ike = aes128-sha1-modp1024!
esp = aes256-sha1-modp1024,aes192-sha1-modp1024,aes128-sha1-modp1024,aes256-sha1-modp1024,aes192-sha1-modp1024,aes128-sha1-modp1024!
leftauth = psk
rightauth = psk
rightid = XXX.XXX.XXX.XXX
aggressive = no
rightsubnet = 192.168.101.0/24
leftsubnet = 192.168.1.0/24conn con1001
fragmentation = yes
keyexchange = ikev1
reauth = yes
forceencaps = no
mobike = no
rekey = yes
installpolicy = yes
type = tunnel
dpdaction = clear
dpddelay = 10s
dpdtimeout = 60s
auto = add
left = XXX.XXX.XXX.XXX
right = XXX.XXX.XXX.XXX
leftid = XXX.XXX.XXX.XXX
ikelifetime = 21600s
lifetime = 3600s
ike = aes128-sha1-modp1024!
esp = aes256-sha1-modp1024,aes192-sha1-modp1024,aes128-sha1-modp1024,aes256-sha1-modp1024,aes192-sha1-modp1024,aes128-sha1-modp1024!
leftauth = psk
rightauth = psk
rightid = XXX.XXX.XXX.XXX
aggressive = no
rightsubnet = 192.168.101.0/24
leftsubnet = 192.168.150.0/24 -
I was using certs on 2.2.2 but the tunnels wont come up on 2.2.3 .
I will try PSK instead to see if I get the same problem as you before reinstalling 2.2.2 . -
Are you seeing any errors in the IPSEC log?
EDIT I should have asked if you have debugging enabled and if you are seeing errors in the logs with the debugging on. /EDITIPSEC can be a temperamental beast. What does the far end look like? Does it still have security associations stuck from when the IPSEC tunnel was up prior to the upgrade? Can you try to send traffic from the far end to a specific IP on the side you upgraded? Often times IPSEC needs 2-way traffic to properly reinitialize the tunnels, specifically when one side is reset and the other side is not.
-
I don't see any IPSec errors logged, not on the pfSense side and not on the two other sides (Stormshield and Draytek).
I control all these products and have restarted everything, without any luck.
The tunnels are connected without problems, phase 1 and phase 2, but no traffic is passing thru the tunnel.
With pfSense 2.2.2 this was working fine.
Debug logging is not enabled yet…. -
You posted stats that show traffic is being sent out of the enc0 interface, but not being received back in. Can your other firewalls produce similar statistics?
If you are sending traffic out over the enc0 interface, the other sides should be receiving traffic. It would be interesting to see if those IPSEC tunnels are sending traffic. Since your PFSense doesn't show any blocked or passed traffic IN, then at least it would appear it isn't receiving any traffic at all.
-
I have checked with ssh on both sides and can see the trafic going into the tunnel on the pfSense but not coming out on the Stormshield and vice versa. So it seems to get lost in a big black hole :-)
It seems that I am not alone;
https://forum.pfsense.org/index.php?topic=95659.0There are a lot of IPSec related problems mentioned on this forum after upgrading to v2.2.3.
-
Yeah, seems you might be getting smacked with a bug since others are experiencing the same thing.
You stated you rebuilt your tunnels… did you try rebuilding the rules? Maybe the rules somehow became dis-associated with the IPSEC tunnel.
-
Hi,
Having the same issue too. VPN up (to Amazon), but no traffic flowing.
-=david=-
-
I have created some new allow-any-any rules above all existing rules to check if it was rule related, but that didn't help. So I think it is not rule related….
OpenVPN is still running fine, it is purely IPSec related. -
Hmm, gremlins perhaps…
I have 2 PFSense units I just upgraded today to 2.2.3 (i386). I have a tunnel between them. I did have to "manually" start the tunnel, but these are units I use specifically for testing, so they don't really send traffic over the tunnel unless I create it.
Anyway, I just manually started the tunnel and it came up and I am able to ping across the tunnel. I am using fairly standard settings and seems pretty close to your build.
config setup uniqueids = yes charondebug="" conn bypasslan leftsubnet = 192.168.1.11/32 rightsubnet = 192.168.1.0/24 authby = never type = passthrough auto = route conn con1000 fragmentation = yes keyexchange = ikev1 reauth = yes forceencaps = no mobike = no rekey = yes installpolicy = yes type = tunnel dpdaction = none auto = route left = x.x.x.x right = y.y.y.y leftid = x.x.x.x ikelifetime = 28800s lifetime = 28800s ike = aes256-sha256-modp1024! esp = aes256-sha256-modp1024! leftauth = psk rightauth = psk rightid = y.y.y.y aggressive = no rightsubnet = 192.168.230.0/24 leftsubnet = 172.16.100.1|192.168.1.0/24Granted, this is between 2 PFS hosts. I can't speak to how IPSEC is working to other firewalls under 2.2.3.
-
Silly question, but do you see any "blocked" packets the WAN of your firewall for packets coming from the far end firewalls (and vice versa)? I don't mean IPSEC in particular, I am just curious since the enc0 interface isn't reporting any received packets, are any of those packets actually reaching the firewall and getting blocked.
-
Just want to chime in and say I am also experiencing this issue after a successful upgrade to 2.2.3 - IPSec working fine before the upgrade.
Chris
-
Another "me too". I've got a pair so I updated only the secondary. The primary, still on 2.2.2, works fine for all tunnels. The secondary on 2.2.3 has dead tunnels. ESP traffic is going both directions, DPD packets exchanged, but nothing makes it to the remote LAN at either end.
The remote endpoints are a Cisco (no access for me) and an OpenBSD host (full access). I have taken a lot of notes, verbose outputs of "ipsec statusall" and "ipsecctl -sa -v" from each end, and traffic captures to try to compare the difference between the working 2.2.2.<->OpenBSD and the broken 2.2.3<->OpenBSD, but nothing jumps out to my relatively untrained eye.
The Cisco tunnel is also dead when on 2.2.3, I just can't troubleshoot from the Cisco end. It works great on 2.2.2 though.
It's not well organized or anything but if it helps someone else help us all figure out what's broken, it's ZeroBin'ed at: http://sebsauvage.net/paste/?1659557931393c28#Rt+LFQvj5kP8L0Bhv5D/8N7ZNypDZ9Wz7y9fMPXnuZU=
-
Glad I am not the only one with this exactly same problem. We use WatchGuard at the office and been working great right up to PfSense 2.2.2. Soon as I upgraded it to 2.2.3 IPSec stopped working. Far as I can tell Phase 1 and Phase 2 are working. Just it's not passing traffic over the tunnel like it should. On my PfSense I do have the pings to hit a sever at work to keep the tunnel alive. That is not even working.
Then I thought it's a firewall rules / routing issue since I actually don't have IPSec rules setup and didn't need to since I only want to access the servers from my house and not the other way around. I created the pass rules anyway. No luck.
I don't have any packages installed that would account for this. I may have to roll back to 2.2.2 until it's been fixed. Bummer. Love the added features in IPSec but at this point can't use it.
-
- 1
After upgrading from PFSense 2.2.2 to 2.2.3 (X64), my IPsec VPNs between PFSense and 2 Cisco RV042 (version 2 hardware) are now down.
The IPsec status show up as:
ADI_VPN xxx.isp.com 188.27.x.x
Port: 500 Any identifier 5.12.y.y
Port: 500 IKEv1
initiator 3DES_CBC:0
HMAC_SHA1_96:0
PRF_HMAC_SHA1
MODP_768connecting
Connect
ADI_VPN xxx.isp.com 188.27.x.x
Port: 500 5.12.y.y 5.12.y.y
Port: 500 IKEv1
responder 47 minutes 3DES_CBC:0
HMAC_SHA1_96:0
PRF_HMAC_SHA1
MODP_768established
5 seconds agoLocal LAN class: 172.77.17.0/24
Remote LAN class: 172.24.96.0/24Thanks for any suggestion.
I will be coming later with screenshots if required. - 1
-
Same here with local ipsec tunnel between vmware horizon view connection and security server. pfsense somehow blocks ipsec traffic although the udb ports 500/4500 are open in nat.
until this is fixed, i will use 2.2.2.
:(
-
It appears as though this maybe an error decrypting traffic from the remote host. Though we have not replicated it here yet, we use IPSec extensively and have been running 2.2.3 for some time internally.
Has anyone tried an encryption type other than AES?
Where exactly is this traffic being lost and are you all seeing the same thing?
Run a ping from behind pfSense 2.2.3 to to some host on the other side of the tunnel.
Run a packet capture on the ipsec interface in pfSense. I expect to see only the leaving echo requests there if the ping is failing.
If you can, run a packet capture at the other end. Do you see the ping requests and replies?
Run a packet capture on the pfSense WAN interface. Do you see the ESP (or UDP 4500 if you're using NAT-T) traffic leaving and returning from the remote tunnel end IP?From the evidence we have gathered so far into this it seems as though the encrypted traffic is returning but not being decrypted correctly so never leaving the ipsec interface internally.
If any of you can confirm or refute that it would be very helpful.Steve
-
I am using 3DES with Broadcom hardware crypto cards. On 2.2.2 everything is fine.
Updating to 2.2.3 the phase 1 cannot be established as mutual RSA certificate authentication fails. After restoring 2.2.2, all of my tunnels authenticate again. -
I just looked on the pfSense bug tracker and there are no issues yet for 2.2.3
https://redmine.pfsense.org/projects/pfsense/issues?query_id=49
Does this one qualify? -
It definitely will when/if we have it narrowed down to something. Or eliminated it in some other way.
Steve
