Racoon makes me sad, this tunnel will not stay up!



  • Alright,

    Let me first explain the fact that I have followed the wiki, exactly. Months ago I was able to create my first tunnel using two pfsense boxes running the latest 1.2.3-RC1 release. When i had first begun working on connecting two remote sites using pfsense, the IPsec tunnel was seemingly impossible to establish. It would simply NOT connect. When it finally did i was very surprised, and to tell the truth, it seemed to happen just because i restarted the systems at different times. Anyway, I have not changed one thing in my config under the Ipsec tab and the power went out Monday of this week. The battery in our UPS ran lower and lower until the shutdown command was issued. I am always afraid of power outages because the tunnel is sooo difficult to get back up. I have tried everything from deleting the configs for ipsec and restarting therafter just so i could start fresh you know? But nooo, racoon just yells this jibberish at me. And it's the same jibberish it always presents me whenever pfsense needs to be restarted, and then magically after possibly the 20th restart, phase 2 will authenticate and the connection will re-establish. I am not too keen with IPsec or furthermore, racoon. If somebody would be willing to help me out, it would be greatly appreciated. We use the connection for a "production environment" where the backup server is off site.

    The Jibberish: First timestamps at bottom.

    Oct 3 00:06:29 racoon: [Connection From Madison University]: INFO: respond new phase 1 negotiation: 192.168.0.1[500]<=>22.222.222.22[500]
    Oct 3 00:04:50 racoon: ERROR: phase1 negotiation failed due to time up. a956681c6b085e2f:199d5bcccf112430
    Oct 3 00:04:00 racoon: INFO: Hashing 192.168.0.1[500] with algo #2
    Oct 3 00:04:00 racoon: [Connection From Madison University]: INFO: Hashing 22.222.222.22[500] with algo #2
    Oct 3 00:04:00 racoon: INFO: Adding remote and local NAT-D payloads.
    Oct 3 00:04:00 racoon: NOTIFY: couldn't find the proper pskey, try to get one by the peer's address.
    Oct 3 00:04:00 racoon: INFO: Selected NAT-T version: RFC 3947
    Oct 3 00:04:00 racoon: INFO: received Vendor ID: DPD
    Oct 3 00:04:00 racoon: INFO: received Vendor ID: draft-ietf-ipsec-nat-t-ike-00
    Oct 3 00:04:00 racoon: INFO: received Vendor ID: draft-ietf-ipsec-nat-t-ike-02
    Oct 3 00:04:00 racoon: INFO: received Vendor ID: draft-ietf-ipsec-nat-t-ike-02
    Oct 3 00:04:00 racoon: INFO: received Vendor ID: RFC 3947
    Oct 3 00:04:00 racoon: INFO: received broken Microsoft ID: FRAGMENTATION
    Oct 3 00:04:00 racoon: INFO: begin Aggressive mode.
    Oct 3 00:04:00 racoon: [Connection From Madison University]: INFO: respond new phase 1 negotiation: 192.168.0.1[500]<=>22.222.222.22[500]
    Oct 3 00:01:22 racoon: [Connection From Madison University]: INFO: IPsec-SA established: ESP 192.168.0.1[4500]->22.222.222.22[4500] spi=90608655(0x566940f)
    Oct 3 00:01:22 racoon: [Connection From Madison University]: INFO: IPsec-SA established: ESP 22.222.222.22[0]->192.168.0.1[0] spi=97209630(0x5cb4d1e)
    Oct 3 00:01:22 racoon: INFO: Adjusting peer's encmode UDP-Tunnel(3)->Tunnel(1)
    Oct 3 00:01:22 racoon: INFO: Adjusting my encmode UDP-Tunnel->Tunnel
    Oct 3 00:01:22 racoon: INFO: NAT detected -> UDP encapsulation (ENC_MODE 1->3).
    Oct 3 00:01:22 racoon: [Connection From Madison University]: INFO: initiate new phase 2 negotiation: 192.168.0.1[4500]<=>22.222.222.22[4500]
    Oct 3 00:01:21 racoon: [Connection From Madison University]: INFO: ISAKMP-SA established 192.168.0.1[4500]-22.222.222.22[4500] spi:eb6c6fe96d0a9d94:c2c6c848bac9244e
    Oct 3 00:01:21 racoon: INFO: Hashing 192.168.0.1[4500] with algo #2
    Oct 3 00:01:21 racoon: [Connection From Madison University]: INFO: Hashing 22.222.222.22[4500] with algo #2
    Oct 3 00:01:21 racoon: INFO: Adding remote and local NAT-D payloads.
    Oct 3 00:01:21 racoon: NOTIFY: couldn't find the proper pskey, try to get one by the peer's address.
    Oct 3 00:01:21 racoon: [Connection From Madison University]: INFO: KA list add: 192.168.0.1[4500]->22.222.222.22[4500]
    Oct 3 00:01:21 racoon: INFO: NAT detected: ME PEER
    Oct 3 00:01:21 racoon: INFO: NAT-D payload #0 doesn't match
    Oct 3 00:01:21 racoon: [Connection From Madison University]: INFO: Hashing 22.222.222.22[500] with algo #2
    Oct 3 00:01:21 racoon: INFO: NAT-D payload #-1 doesn't match
    Oct 3 00:01:21 racoon: INFO: Hashing 192.168.0.1[500] with algo #2
    Oct 3 00:01:21 racoon: INFO: Selected NAT-T version: RFC 3947
    Oct 3 00:01:21 racoon: INFO: received Vendor ID: DPD
    Oct 3 00:01:21 racoon: INFO: received broken Microsoft ID: FRAGMENTATION
    Oct 3 00:01:21 racoon: INFO: received Vendor ID: RFC 3947
    Oct 3 00:01:21 racoon: INFO: begin Aggressive mode.
    Oct 3 00:01:21 racoon: [Connection From Madison University]: INFO: initiate new phase 1 negotiation: 192.168.0.1[500]<=>22.222.222.22[500]
    Oct 3 00:01:21 racoon: [Connection From Madison University]: INFO: IPsec-SA request for 22.222.222.22 queued due to no phase1 found.
    Oct 3 00:01:18 racoon: INFO: unsupported PF_KEY message REGISTER
    Oct 3 00:01:18 racoon: INFO: 11.111.111.11[4500] used for NAT-T
    Oct 3 00:01:18 racoon: [Self]: INFO: 11.111.111.11[4500] used as isakmp port (fd=19)
    Oct 3 00:01:18 racoon: INFO: 11.111.111.11[500] used for NAT-T
    Oct 3 00:01:18 racoon: [Self]: INFO: 11.111.111.11[500] used as isakmp port (fd=18)
    Oct 3 00:01:18 racoon: INFO: 192.168.0.1[4500] used for NAT-T
    Oct 3 00:01:18 racoon: [Self]: INFO: 192.168.0.1[4500] used as isakmp port (fd=17)
    Oct 3 00:01:18 racoon: INFO: 192.168.0.1[500] used for NAT-T
    Oct 3 00:01:18 racoon: [Self]: INFO: 192.168.0.1[500] used as isakmp port (fd=16)
    Oct 3 00:01:18 racoon: INFO: 127.0.0.1[4500] used for NAT-T
    Oct 3 00:01:18 racoon: [Self]: INFO: 127.0.0.1[4500] used as isakmp port (fd=15)
    Oct 3 00:01:18 racoon: INFO: 127.0.0.1[500] used for NAT-T
    Oct 3 00:01:18 racoon: [Self]: INFO: 127.0.0.1[500] used as isakmp port (fd=14)
    Oct 3 00:01:18 racoon: NOTIFY: NAT-T is enabled, autoconfiguring ports
    Oct 3 00:01:18 racoon: INFO: Resize address pool from 0 to 255
    Oct 3 00:01:18 racoon: INFO: Reading configuration from "/var/etc/racoon.conf"
    Oct 3 00:01:18 racoon: INFO: @(#)This product linked OpenSSL 0.9.8e 23 Feb 2007 (http://www.openssl.org/)
    Oct 3 00:01:18 racoon: INFO: @(#)ipsec-tools 0.7.1 (http://ipsec-tools.sourceforge.net)

    What can i do to fix my problem, there's not anything in the GUI configuration could possibly be wrong; and i know this because the settings that i use have worked fantastically for the past few months! Backed up gigs of dataz.
    My best guess here is that there is a timing issue, and i don't know where i could edit timing settings.
    Please assist me in my ever long quest to understand why racoon doesn't like to play nice and just establish the connection. Thanks in advance.



  • If 13 people can look at this post then I would think that possibly half could give a piece of advice or a best guess for me. I'm really desperate and trying to get this back up by Monday. If I could post any more information to help troubleshoot, please let me know what that pertinent information would be.  :-\



  • well make it 14.

    I'm reading your note to get some tips in order find a solution to MY problem of not getting my IPSec tunnel to establish - at all!
    I can get them running on two test pc's side by side with a switch between them - but not across the 'net. I think NAT-T is a snag in my case.
    I have no idea on your problem. Will be watching though.
    Good luck



  • Thank you pmb1010.

    Anyone else?  :'(



  • Support pleasssee?  ??? ??? ???  :'(



  • I'm close to just reinstalling pfsense on both machines just to get the tunnel back up :-/ i wish someone could help even just a little bit :(


  • Rebel Alliance Developer Netgate

    Try again with an RC3 snapshot. NAT-T was pulled due to various instabilities it caused.



  • Thanks Jimp, Will try flashing both pfsense machines right now with:
    http://snapshots.pfsense.org/FreeBSD_RELENG_7_2/pfSense_RELENG_1_2/updates/pfSense-Full-Update-1.2.3-20091005-1447.tgz
    off of the snapshots page from today. I will post back soon. Hopefully it works.  :-\



  • Phase two established on both ends as reported to me in the logs. However, i went to the status page, and it seems like the SPIs do not match, (im not sure if they're supposed to or not) are they supposed to match?

    After the flash update, I enabled IPsec again and set up the tunnel between site a and site b.
    I was very surprised that the tunnel went up. Same configuration that has never given me any problems before. Oh, I restarted racoon on both servers because no traffic would pass. In the log it got as far as establishing phase two, then this:

    Oct 5 15:52:59 last message repeated 4 times
    Oct 5 15:52:39 racoon: ERROR: unknown Informational exchange received.

    Im not too sure what's going on here. Do you? Thanks


  • Rebel Alliance Developer Netgate

    You may have something wrong in your tunnel config. The SPIs should match up between both sides, though in the reverse direction. For example the SPI should look like:
    A -> B  00000001
    B -> A  00000002

    on one side (this is an example, it's really like 0b23cb26) and

    B -> A 00000001
    A -> B 00000002

    on the other side of the tunnel.

    That log error seems to suggest that it didn't fully establish the tunnel due to some kind of configuration mismatch. To help much more we'd need the logs from both ends of the tunnel, as well as full screenshots of the tunnel config screens (you can black out the public IPs and the PSK)



  • Well here's what you asked for. I hope it helps out :/

    Site A (home base or HQ if you will)::

    Site A (allow mobile connections page)::

    Site B (backup site)::

    The logs can be viewed at the following two links. Site A hasn't been as consistent as site B in keeping a log of whats going on. The data I uploaded to my server is fresh and these logs are current as you can see by the time stamps.

    SITE A W/ Raw Filter Logging Enabled
    http://www.inferno-wan.com/pfs/sitealog.html

    SITE B W/ Raw Filter Logging Enabled
    http://www.inferno-wan.com/pfs/siteblog.html

    Just to remind who ever is helping me… I haven't touched my firewall settings in regards to IPsec allowance, mostly because the tunnel worked great last sunday and for multiple months before that (i even had to restart a couple times over those few months). I did look over the rules and make sure all IPsec applicable schema were entered properly, and they were. If for some reason it's possible that entries can become somehow corrupt, should i re-create the rules?

    You know how you can be working on some type of annoyance for weeks and then you find the problem to be a stupid check box setting? If that's the case here, I'm gonna have a good laugh. But IMHO here, i don't think that's the case.

    If i can post any more information please let me know.



  • I'm game for trying a different configuration if mine looks questionable.


  • Rebel Alliance Developer Netgate

    Why are you using mobile tunnels? Are site A and site B dynamic IP?

    If they are dynamic IP, they will need a different identifier, such as "sitea@example.com" and "siteb@example.com" and PSKs to match.

    You'd probably be better off with static tunnels built between each site and using dyndns hostnames for the peer addresses if you have dynamic IPs.



  • I read in the documentation that if one IP is dynamic then Mobile clients had to be enabled. I will try disabling it.

    Both sites use Time Warner as an ISP. There is only one hop in between the sites and ping times over the tunnel (when working) are an average of 18 ms.

    Site A has a static IP address, and Site B has a dynamic address but it only changes when the modem sees a different mac being used. It hasn't changed in months. I doubt it will change any time soon either, once before i kept the same ip address under a dynamic account from them for over a year! lol. But my question has always been when configuring PFsense is does it matter that site b has a dynamic ip address even though it doesn't change?



  • ehhh :( I tried disabling it to no avail. Same problem. The logs look the same.  :-\



  • Is one of the ends actually a private IP on WAN?

    The logs you pasted show a phase 1 timeout, which either means a setting mismatch, which isn't the case based on your screenshots, or traffic not actually getting to the remote end. I suspect some other firewall is blocking that traffic before it gets to you.



  • nope no private ip on wan. I did a find all/replace all in macromedia before i posted. site a is static on wan, and site b is dynamic on wan, but that ip never changes. There are no other firewalls before the pfsense boxes, so if traffic is being blocked before racoon can handle it pfsense must be the culprit? Time warner doesn't block any ports or protocols specific to my application scenarios. Could i post any other information to aid in resolution?



  • Sooo, would reinstalling pfsense and starting from scratch be my best bet?



  • I tried reinstalling pfsense and it was to no avail! The tunnel would NOT STAY UP! I used 1.2.3 - RC1 iso, same install that i used for months and had a stable IPSEC site to site VPN. Can somebody please answer me this, is it possible that the hardware combination of both pfsense boxes could be the culprit? I'm using two old power spec boxes (micro center pcs) for pfsense. All four nics are different, and each machine contains a different processor. So is it possible that this could be a reason why the tunnel wont stay up? I'm out of ideas… should i assemble two brand new pfsense boxes with the same exact hardware configurations? I really need some help on this issue. I am more then willing to donate to FreeBSD if i can get my issue corrected. I am not in the position to spend XXX dollars on support because of lack of budget, so any help on this issue would be greatly appreciated. Site to site IPSEC VPN tunnels are not very user friendly from what i can see....  ???



  • I had similar issues with tunnels between 2 pfsense boxes not staying up.  I'm running 1.2.2, so not sure how much of this is applicable.  It seemed like to get the tunnels up I would just randomly restart and disable/reenable tunnels until they worked.  Finally though I found something that seem to work every time.  1) Disabled both ends of the tunnel 2) setkey -FP on both ends of the tunnel 3) restarted both racoons 4) Reenabled the tunnels.  Not sure if this will help or not.  The way I finally fixed my tunnel stability issue was to change the lifetime to about 10 days for both phase 1 and phase 2, making sure not to set them to the same thing.  I have no idea why this fixed the problem, but it did.  Not sure if any of this will help you or not.



  • My IPSEC tunnels have always connected, but sometimes wouldn't reconnect.  I switched to RC3 and a lot of this was fixed.  The only tunnel I have problems with is one over a wireless connection.

    If your tunnels are establishing, but no data passing, be sure to double-check your firewall to make sure there are IPSEC rules to allow it.  I forgot to do this after replacing a pfsense router, and it caused me grief.


Log in to reply