PFSense 2.0 IPSec Configuration Instructions?

  • I need help with IPSec on PFSense 2.0, I have spent over 180 hours since last Tuesday trying to get it to work with no luck.  I have read every guide possible, though most are for PFSense 1.2.3 which hasn't really been much help.

    I am an IT College Student so I am not a networking professional and this is not for a business.  I am familiar with Mac, Linux, and Windows, and do web development, but my networking knowledge is limited to subnetting and basic routing, I am not familiar with VPN configuration of any sort.

    My goal is to have a VPN to connect to my home network while I am traveling, such as on college campus.  It would give me access to shared files and my web server at home.  I wanted to do this with IPSec because to my understanding it is compatible with both Windows 7 & OS X built-in clients, so I am not interested in alternative solutions like OpenVPN.

    I have setup a home server, but I don't run tons of computers, to simplify the number of electronics, I made the configuration itself moderately complex, but the topology is simple and strait forward.

    Cable Modem -> PFSense (Host Computer OnBoard NIC) -> PCI NIC -> Asus Switch -> Wireless AP.

    PFSense gets the WAN IP directly from the cable modem.  The wireless has ethernet ports, but they are 10/100 and it only has three, so I chose to use a Switch because I have numerous wired devices.  The wireless AP is setup in bridge mode, and the network is 100% function including the web server and local file sharing.

    The complex part is that PFSense, the Web Server and File Server all run on the single host computer, which is actually running Windows Server 2008 R2.  I setup PFSense and Debian Linux in VMWare Server, so they all start at system boot before any users login.  So far, it works excellent and only took a few days of tinkering to figure it all out.

    My ISP provides me with a dynamic IP so I setup a free DynDNS account.  However I have a paid registered domain connected to the DynDNS domain.  Some guides have mentioned the need to configure DynDNS in PFSense, but have to imagine that isn't necessary, so far I have no trouble using SSH or Remote Desktop, and the VPN is only for short durations, it is not a sustained connection (I won't be doing it while my ISP changes the IP).

    As mentioned I just want to be able to connect on the go, so if I understand the naming scheme I want a Mobile/Road Warrior configuration.

    The LAN is subnet from, the first available being the PFSense IP, second the host, and third the web server, and a few more are using reserved/static IPs.  DHCP is setup for the remaining seven.

    IPSec is turned on, and a firewall rule to pass all traffic on IPSec is enabled.

    Mobile clients tab "Enable IPsec Mobile Client Support" has been checked.  Some guides show "Virtual Address Pool" being set to a value, I have tried with it set and with it unset, but for now I have set it to [] (I still don't know if this is required, or whether it has to be on a separate network even).  Some guides show setting DHCP servers, I tried, but again no changes in the results, so I turned that off.

    I have a single preshared key and identifier.

    My Phase 1 is configured as follows:

    Interface:  WAN
    Authentication:  Mutual PSK
    Negotiation Mode:  Main

    My Identifier:  My IP Address
    Encryption algorithm:  3DES
    Hash Algorithm:  SHA1
    DH Key Group:  2
    Lifetime:  86400

    NAT-T:  Force

    My Phase 2 is configured as follows:

    Mode:  Tunnel
    Local Network:  MyLAN
    Protocol:  ESP
    Encryption Algorithm:  AES (Auto), Blowfish (Auto), 3DES, CAST128
    Hash Algorithms:  SHA1, MD5
    PFS key group:  off
    Lifetime:  3600

    I have tried connecting from my Mac using the build-in OS X Client in network preferences.  There are error logs indicating that trafffic is definitely reaching its destination, but the connection itself is no good.

    I have tried:

    -  Just the domain and shared key
    -  Adding the "Account Name" & password for pfsense

    Neither seems to make a difference with regards to the error logs, but no guides clearly explained whether additional accounts are needed in PFSense for this.

    I have also tried:

    -  I have tried changing the Negotiation Mode to Aggressive & Main
    -  I tried turning NAT-T to off & Enabled

    Aggressive mode fails outright, leaving me with the following log errors:

    racoon: [xx.xx.xx.xx] ERROR: exchange Identity Protection not allowed in any applicable rmconf.
    racoon: [xx.xx.xx.xx] ERROR: exchange Identity Protection not allowed in any applicable rmconf.
    racoon: [xx.xx.xx.xx] ERROR: exchange Identity Protection not allowed in any applicable rmconf.
    racoon: [xx.xx.xx.xx] ERROR: exchange Identity Protection not allowed in any applicable rmconf.
    (Note:  xx.xx.xx.xx is the ip I was accessing from.)

    I researched the errors and found suggestions to change it to "Main", which gives me this lovely series of log messages instead:

    racoon: [xx.xx.xx.xx] ERROR: phase1 negotiation failed.
    racoon: [xx.xx.xx.xx] ERROR: failed to pre-process ph1 packet [Check Phase 1 settings, lifetime, algorithm] (side: 1, status 1).
    racoon: [xx.xx.xx.xx] ERROR: failed to get valid proposal.
    racoon: ERROR: no suitable proposal found.
    racoon: [xx.xx.xx.xx] INFO: Selected NAT-T version: RFC 3947
    racoon: INFO: received Vendor ID: DPD
    racoon: INFO: received Vendor ID: draft-ietf-ipsec-nat-t-ike-02
    racoon: INFO: received Vendor ID: draft-ietf-ipsec-nat-t-ike-02
    racoon: INFO: received Vendor ID: draft-ietf-ipsec-nat-t-ike-03
    racoon: INFO: received Vendor ID: draft-ietf-ipsec-nat-t-ike-04
    racoon: INFO: received Vendor ID: draft-ietf-ipsec-nat-t-ike-05
    racoon: INFO: received Vendor ID: draft-ietf-ipsec-nat-t-ike-06
    racoon: INFO: received Vendor ID: draft-ietf-ipsec-nat-t-ike-07
    racoon: INFO: received Vendor ID: draft-ietf-ipsec-nat-t-ike-08
    racoon: INFO: received Vendor ID: RFC 3947
    racoon: INFO: begin Identity Protection mode.
    racoon: [Self]: INFO: respond new phase 1 negotiation:[500]<=>xx.xx.xx.xx[500]
    (Note: is my WAN IP)

    The key error I think might be "ERROR: no suitable proposal found.", but I have tried AES and Blowfish for the Phase 1 encryption algorithm, nothing changes.  I can't seem to find any detailed information on the error that is in any way something I understand or am familiar with, so not really too helpful.  I also noticed "NAT-T Version: RFC 3947", so I tried changing it to off, and that just eliminated that message, but the rest remained identical.  The fact is Phase 1 negotiation is failing.  Given its title I believe Phase 1 comes before Phase 2, but as mentioned before I don't really know how VPNs work, just what they does.

    When I go to the IPSec Status page, I see my WAN IP as my Local IP, Local Network is "LAN" (Not "MyLAN", the name I gave it in PFSense), Remote Network says "Mobile Client", and I have a yellow icon with an X under Status.  No records exist in SAD or SPD.

    I have to imagine I missed one or more important steps in configuring IPSec, but the lack of detailed guides available (especially for PFSense 2.0) has me stumped.

    If anyone could provide me with some help that would be fantastic.

  • In some places pptp, l2tp and ipsec is blocked via firewall rules, openvpn is quite hard to block, unless you block https also. Only my 2 cents

Log in to reply