PFS <> ASA IPSec tunnel help

sipple31

Hey all,

I'm having issues with an IPSec tunnel between PFSense 2.1 and ASA 8.4(7). It seems to be going to sleep… one way. I have been struggling through tons of settings and cannot keep the tunnel up the way I'd like. The tunnel shows up as I would expect. Pinging from PFS side works 100% of the time. Once I do that, the ASA side wakes up and works for a period of time.

One the flow breaks... I can watch the ASA stats and see the ping packets leaving, but nothing coming back. As soon as I ping from the PFS side, the beast is awakened, and everything flows fine.

I just don't get why the tunnel is showing up on both sides... but PFSense doesn't respond unless it initiates the traffic flow?

I have DPD and NAT-T disabled on both ends. The "Automatically ping host" feature doesn't seem to be working as I don't see any packets flow.

Can anyone weigh in?

thanks

Matthias

Have you tried enabling DPD and NAT-T on both ends? It's my understanding that Cisco devices do not initiate a tunnel unless traffic flagged by the acl defined in the crypto map is detected. After a certain period of time if no traffic is detected the tunnel will automatically be brought down. This is regardless of the tunnel lifetime seconds. There is a function called isakmp keepalive that is supposed to keep the tunnel open. I'm not familiar with this command so you will need to research it yourself.

sipple31

@Matthias:

Have you tried enabling DPD and NAT-T on both ends? It's my understanding that Cisco devices do not initiate a tunnel unless traffic flagged by the acl defined in the crypto map is detected. After a certain period of time if no traffic is detected the tunnel will automatically be brought down. This is regardless of the tunnel lifetime seconds. There is a function called isakmp keepalive that is supposed to keep the tunnel open. I'm not familiar with this command so you will need to research it yourself.

Thanks for your reply. What you're saying makes sense if the tunnel is being torn down due to inactivity… but it is not. The tunnel shows active on both sides. That's what's weird about this to me... :-\

It's almost like there is another layer going inactive beyond phase2... I've double-checked that the route sticks on the ASA.

Matthias

That's odd. Is there a specific time this happens?

sipple31

@Matthias:

That's odd. Is there a specific time this happens?

Yeah… it's a very short period of time. Around a minute. Super weird!

Matthias

So the tunnel only lasts for about a minute then all traffic ceases to pass?

Have you put racoon into debug mode and monitored the logs while this happens? A syslog server helps.

sipple31

@Matthias:

So the tunnel only lasts for about a minute then all traffic ceases to pass?

Have you put racoon into debug mode and monitored the logs while this happens? A syslog server helps.

So the story continues… I have tried debug mode which shows nothing. I resorted to pcaps. When the ASA is supposedly sending the pings from my workstation into the tunnel, they are not showing up on the PFS side. The packet capture on IPSEC is blank until the traffic starts flowing. So maybe the issue is the ASA side... may have to open a ticket with them.

Matthias

Would you mind posting your ASA config and pfSense IPSec config sans passwords and public IPs? I've just set up my own test network using a 2600 (unfortunately I don't have an ASA to test with) and pfSense. Maybe I can replicate the issue you are having.

sipple31

@Matthias:

Would you mind posting your ASA config and pfSense IPSec config sans passwords and public IPs? I've just set up my own test network using a 2600 (unfortunately I don't have an ASA to test with) and pfSense. Maybe I can replicate the issue you are having.

Took me forever to scrub the ASA one… need to be a little careful there. :-)

Knock yourself out. I'll report back with Cisco's findings.

ASA.txt
pfSense.txt

Matthias

Just a few things I noticed while looking through your ASA config. You have several crypto maps defined but only appear to be using crypto map mymap 6. In that crypto map you are using two transform sets ESP-AES-192-SHA and ESP-3DES-SHA. On pfSense side it's only configured to use AES 192 and SHA. The crypto map is set to match traffic from ACL outside_cryptomap_5 that has the remote network 192.170.1.0 /29. Just curious but why are you using a public IP range for your internal subnet? You also have two IPSec policies defined with different settings.

Maybe if you limited the transform set to only the ESP-AES-192-SHA profile and removed the other IPSec policy that may have an effect.

sipple31

@Matthias:

Just a few things I noticed while looking through your ASA config. You have several crypto maps defined but only appear to be using crypto map mymap 6. In that crypto map you are using two transform sets ESP-AES-192-SHA and ESP-3DES-SHA. On pfSense side it's only configured to use AES 192 and SHA. The crypto map is set to match traffic from ACL outside_cryptomap_5 that has the remote network 192.170.1.0 /29. Just curious but why are you using a public IP range for your internal subnet? You also have two IPSec policies defined with different settings.

Maybe if you limited the transform set to only the ESP-AES-192-SHA profile and removed the other IPSec policy that may have an effect.

Yeah a few of these settings are me just getting desperate and trying different things. ;D

I limited the transform set to just the one being used with no effect. I am using both of those IPSec policies… so I don't think I can remove one at this time. Good point on the public address space, though, I should probably work on that...

Matthias

Hmm ok. The only other thing I could suggest is posting the outputs of the following commands when the tunnel is working and when it stops working.


show crypto ipsec sa
show crypto ikev1 sa
show crypto map

sipple31

@Matthias:

Hmm ok. The only other thing I could suggest is posting the outputs of the following commands when the tunnel is working and when it stops working.
show crypto ipsec sa
show crypto ikev1 sa
show crypto map

So I went ahead and switched the whole network around to 192.168.31.x to get off the public range. Got the tunnel back up. Same exact behavior. Here are some show commands off the ASA if you are interested. They do not change whether the tunnel is "dormant" or not… I'm so confuzzled.

ASA_Show_Cmds.txt

Matthias

Well I'm not sure on this one. Your results with NAT-T enabled were the same I'm assuming? My guess is there's something going on with the security associations. They might be expiring on the pfSense side. You said the indicator stays green on the IPSec status page right? Maybe check the SAD tab. There should be two SAs there, one for inbound and one for outbound and the data column should be increasing when you refresh the page (there may be more than two but only two will be active with data increasing). Their SPI's should match the SPI's listed from the sh crypto ipsec sa output under current inbound/outbound SPIs.

I'm not sure where else to look for. Kinda at a loss for this one. Seems like there's something deeper happening.

sipple31

@Matthias:

Well I'm not sure on this one. Your results with NAT-T enabled were the same I'm assuming? My guess is there's something going on with the security associations. They might be expiring on the pfSense side. You said the indicator stays green on the IPSec status page right? Maybe check the SAD tab. There should be two SAs there, one for inbound and one for outbound and the data column should be increasing when you refresh the page (there may be more than two but only two will be active with data increasing). Their SPI's should match the SPI's listed from the sh crypto ipsec sa output under current inbound/outbound SPIs.

I'm not sure where else to look for. Kinda at a loss for this one. Seems like there's something deeper happening.

lol… I know...

when the tunnel is dormant the traffic is NOT coming across the tunnel from ASA to PFS. The traffic does not increment. If I send one ping from PFS to ASA, the tunnel awakens and traffic flows freely both ways. If the tunnel is dormant and I start a continuous ping from ASA to PFS... it might wake up after 4 failed pings or after 40. It's totally random. I definitely think this issue lies in the ASA.

This does not happen with our other branch networks running the little ASA 5505's. I need to get this working flawlessly to convince the boss-man to start putting more PFS into branch offices versus the overpriced 5505's. :D

sipple31

Well Cisco washed their hands of this. I got on a support call with a guy and he was able to prove that the request packets were being sent to the pfSense box. Packet captures don't lie.

This is either a hardware issue with the appliance running my pfSense or an ISP issue. Sucks to have a little nagging issue like this. It bugs me. :-\

cmb

You have "Prefer old SAs" enabled under System>Advanced? Probably shouldn't. Description matches what might happen if that were set in this circumstance.

sipple31

Yes - thanks for pointing that out. I have tried with that setting on and off… same behavior.

sipple31

Well. Tried new interfaces on my device to prove out any sort of IRQ issue. Messed around with settings a bunch more - always the same behavior. I also tried setting NTP to peer with a server over the IPsec. The slight addition of traffic keeps the tunnel "alive" MORE often… but still pretty easy to catch it sleeping. PCAPS show the ESP packets coming in and pfs not responding. So weird.

I got a desktop PC ready with the same pfs version, restored my config, and am going to throw it in place of my current hardware. At this point I'm curious if i'm running into a hardware issue or a pfs bug. Will try it out the next time I'm home...

EDIT: It's funny... this thread from way back in 2007-2008 explains almost the exact same behavior I'm seeing: https://forum.pfsense.org/index.php?topic=5920.0

cmb

Is that inbound ESP being blocked? Should show in the firewall log if it is, unless you disabled logging on the default deny rule. There are two reasons you see ESP coming in and nothing actually decrypting that traffic - it's getting blocked, or it isn't for an active SA. The rules to allow that ESP would be automatically added unless you have that disabled under System>Advanced.