BOUNTY: IPSec VPN TUNNEL REDUNDANT 1K$ USD



  • Noted that package support will not be available on embedded platform. Then I think I need the IPsec failover with multi WAN at one pfsense. I tried to establish ipsec tunnel at OPT1 interface with Wrap but I failed. I think I have done everything including static route. But the tunnel is not made at OPT. There is no problem to establish SPD but SAD is not made. The only difference between Martinc_77's environment and mine is that he used static IP ISP's for ISP1 and ISP2 but I used DHCP ISP's for ISP1 and ISP2. So I added two routers for making static IP's as below.

    pfsense1
       WAN                      OPT1
    192.168.5.2          192.168.4.2
          l                             l
    192.168.5.1           192.168.4.1
      router                     router
         l                             l
      DHCP                      DHCP
      ISP1                       ISP2
        l                             l
                INTERNET
                      l
                   ISP3
                 static IP
                pfsense2 (Waiting movile client)

    And I really need IPsec failover immediatley. So I decide to donate USD1,000 if your team will achive IPSec failover immediately. Belows are what I want.

    1. IPsec should be available at both of OPT1 and WAN interfaces.
    2. If one of ISP fail on OTP1 or WAN, another one should operat automatically and immediately without fail.
    3. It should be available in DHCP ISP (both of OPT1 and WAN). I don't want to add routers to change DHCP ip to static ip. It make me spend more money.
    4. I use IPSEC to send all tracfics from pfsense1 to pfsense2. So I use 1.0.0.0/0 for "Remote subnet" in ipsec configuration.
    5. I need it with PC Engines Wrap (embedded platform)
    6. Hope it will be available with 1.0 version or before 1.0 version. ASAP.

    I don't know how much it will be difficult for your team. So I don't know whether USD1,000 is enough for this request. Pls tell me if it will be possible or not. I hope it is not so difficult and your team can make it easily. As I know, it is available in Linux but I want to use pfsense which is the best software I have ever met.

    Thank you.



  • For immediate fail over this means two active VPN lines rather than initiate a second when the first fails.  This can be setup now with using static routes and metrics, however what you are really asking for is just IPSEC support on OPT1, which with m0n0 already is supported, so apparently there is some bug you are encountering?

    An easier method might be to implement BGP instead, I see the related posts on this topic:

    (edit) Actually an easier method might be to use OLSR instead of BGP, but you still need IPSEC working, it would probably be better to use OpenVPN in that case.  I don't know how well pfSense/OpenVPN on embedded hardware is (i'm waiting for v1.0 to test again) as m0n0 on WRAP consistently crashes whenever I send Kerberos data over the connection :(

    Sorry, not really helping, but is IPSEC an absolute requirement?

    Another alternative is adding another pfSense unit and using OSLR on that, this should work now:

    
             |
             |
          pfSense 1
             |
             |
      {^^^^^^^^^^^^^}
      {  Internets  }
      {             }
      {~~~~~~~~~~~~~}
       |           |
       |           |
     pfSense 2  pfSense 3
       \           /
        \         /
         \       /
         pfSense 4
             |
             |
    
    

    With pfSense 1, 2, 3 running IPSEC, and 1 & 4 running OSLR.



  • This works now.  Set VPN -> IPSEC -> Failover IP.

    I tested this last night while using a voip phone across a IPSEC tunnel.  I lost 1 second of audio and everything just worked.



  • I think he means across redundant WANs, not redundant boxes.

    –Bill



  • I'd like to use OPT1 and WAN for IPSec. And ISP1 is connected WAN1 and ISP2 is connected to OPT1. I want use only one pfsense box for redundant.
    Please see below diagram.

    ____ pfsense1_____
          l                            l
      WAN                      OPT1
      DHCP                      DHCP
      ISP1                      ISP2
        l                            l
                INTERNET
                      l
                  ISP3
                static IP
                pfsense2 (Waiting movile client)

    Now, IPSec at OPT1 is not available if OPT1 and WAN are connected to internet with DHCP. I think it should be solved first.

    Thank you.



  • I took a look at this issue but I think there are some FBSD limitations.

    First, I was able to get ipsec working on OPT1 by modifying vpn.inc and filter.inc.  Right now it looks like the firewall rules for ipsec are hardcoded to the WAN interface.  I changed it to add rules based on the interfaces for tunnels but it's a little kludgy.

    The real problem, however, is with setkey and the SPDs.  You said you use DHCP on the dual WANs.  I assume your address are dynamic.  I think that poses a problem because you have to specify the end-points of your tunnel with fixed IP addresses.  You'll need a mechanism to detect when your WAN's IP changes; then it will have to create new SPDs.

    The next problem is that you can't have simultaneous redundant tunnels for the same range of networks as far as I can tell…

    You want to have two tunnels come up for the same range of traffic, but setkey won't allow it.

    Here's an example spd.conf from my experiments:

    spdadd 192.168.20.0/24 192.168.20.1/32 any -P in none;
    spdadd 192.168.20.1/32 192.168.20.0/24 any -P out none;
    spdadd 192.168.20.0/24 192.168.50.0/24 any -P out ipsec esp/tunnel/192.168.0.2-192.168.0.5/unique;
    spdadd 192.168.50.0/24 192.168.20.0/24 any -P in ipsec esp/tunnel/192.168.0.5-192.168.0.2/unique;
    spdadd 192.168.20.0/24 192.168.50.0/24 any -P out ipsec esp/tunnel/192.168.0.4-192.168.0.5/unique;
    spdadd 192.168.50.0/24 192.168.20.0/24 any -P in ipsec esp/tunnel/192.168.0.5-192.168.0.4/unique;

    the last two entries will be rejected because a tunnel is already specified for communication between the two networks.  Setkey won't let you add additional tunnels...

    I noticed this has been discussed to death in previous posts...

    Even with ipsec working on OPT1 it's not going to work.
    I can only think of two ways to make this happen:  write a daemon to manually bring up a new tunnel during failover (not instant, not seamless), or modify setkey/freebsd's ipsec implementation to be aware of interfaces instead of just IP addresses.

    The latter solution would be nice but it's not something that I think could be quickly or easily implemented.



  • ifstated might be an option here - I don't see a good way to implement this in pfSense so it's generic enough for broad use though.  I can see how to make this work, obviously you have to be checking link (and tunnel) status and tear down the SPDs for the WAN tunnel on failover and recreate those SPDs to the OPT1 interface.  That takes care of one end…the other end is trickier, how do you allow this new random IP coming in....obviously you can't lock the tunnel down to IPs (and thus ends the research I've done :))

    --Bill



  • The tunnel is set up for mobile clients, so it doesn't matter that the other end's IP address changes.

    But the problem remains that a tunnel has to be torn down and a new one brought up.  It's possible to have a daemon detect failover and respond accordingly, but the process is by no means seamless or instantaneous.

    Also a route can go down without an interface going down.  Some routing protocol should be used to detect when the route is down rather than when the interface is down.

    I see that ifstated can use pings to detect if a route is down, so this could work.

    It's not instantaneous, however, because of the lag while waiting for a new tunnel to be established.



  • will this work with ipsec for fall over?

    server1                                                                              server2
    lan 10.141.251.1  -> wan 80.1.1.1            vpn1  wan 60.90.12.23 -> 10.142.251.1 lan
    vip on lan 10.141.251.2 ->  wan 80.1.1.1 vpn2  opt1 60.91.125.243  -> 10.142.251.2 vip on lan

    routes on server1:
    route add 10.142.251.0/24 gateway 10.142.251.1 metric 0  (vpn1 as default vpn)
    route add 10.142.251.0/24 gateway 10.142.251.2 metric 50 (vpn2 as fallover vpn)

    routes on server 2:
    route add 10.141.251.0/24 gateway 10.141.251.1 metric 0  (vpn1 as default vpn)
    route add 10.141.251.0/24 gateway 10.141.251.2 metric 50  (vpn2 as fallover vpn)



  • It doesn't work because you can't bring two simultaneous tunnels up for the same traffic…

    There are all sorts of issues with racoon crashing also when trying to do failover tunnels on the same box.  I wasted a full day writing scripts to manage the failover and testing this out.  Basically it is possible in theory, but the FBSD implementation of IPsec doesn't allow it work in practice.

    The best way to do it right now is with three machines.  Two to set up the tunnels and one to manage the failover/load balacing (if desired).
    A one-box solution does not work reliably or seamlessly on FBSD, let alone PFsense!



  • I'd like to keep this bounty but I already supplied my client with another solution for IPSec failover. So now I don't need this function actually. But I believe I will need this function for future. So I will reduce the mony to $100 and I will keep this bounty. I think somebody can add some more bounty for this function.

    Thanks to pfsense developers and especially thanks to Ollopa.



  • What was the other solution that you found?

    Robert



  • @valnar:

    What was the other solution that you found?

    Robert

    I want to know it to….



  • IPSEC failover works TODAY on 1.0 using CARP + IPSEC Failover feature.

    I have been using well before 1.0 was released at my work.  2 firewalls.  When the first firewall goes down, all IPSEC traffic still works.    I have gone over the setup many times in the forum.  No scripts needed, nada, it just works.



  • Sorry for late reply.

    I've used other dual wan router in front of pfsense. The dual wan router has a fail over fuction. And I made a script for re-establishing ipsec on pfsense when the failing-over occurs on dual wan router.

    What I want is not box redundant. What I want is fail over internet line with ipsec.
    Pls read my earlier post.

    I'm continuing test for this funtion with pfsense snapshot. But until now it does not work. The main prblem that should be solved first is that ipsec does not work at second wan(opt1).

    Thanks.



  • I think it should work if you add a static route through the OPT-WAN to the remote endpoint. Maybe you can integrate this in your script and test. If this works we maybe could integrate it into the linkdown detection code (no promise though).



  • I have managed to get IPSEC over OPTx working correctly.  I've been testing it on the 3-27-07 updates and .iso's for the full version install.  Here is what I've done to make it work:

    1.  I created the IPSEC definitions on the IPSEC page and chose the OPTx interface that I want to use.
    2.  I created manual rules for the desired OPTx interface to allow IPSEC traffic to connect to the IP of the pfsense box on that Interface.  Specifically, I created rules to allow UDP 500, ESP, AH, and GRE to connect to the Public IP on the OPTx interface.

    Note, I am using static IP's on all of my WAN type interfaces.  All of the routers in front of me are operating in bridged mode and do not do any port forward filtering or NAT type stuff, so if you have private IP's on your OPTx interfaces and the routers in front of them are doing any sort of port forwarding or NAT, you may run into tunnel stability problems.

    Vaughn



  • It should be creating the rules on the OPT interfaces behind the scenes but there is a bug preventing it.  I am aware of the bug but this week has been nonstop madness at my day job and other engagements that have prevented me from fixing it.



  • @sullrich:

    It should be creating the rules on the OPT interfaces behind the scenes but there is a bug preventing it.  I am aware of the bug but this week has been nonstop madness at my day job and other engagements that have prevented me from fixing it.

    There is any solutions now ?


Log in to reply