• Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login
Netgate Discussion Forum
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login

IPSEC VTI Tunnels

Scheduled Pinned Locked Moved IPsec
51 Posts 16 Posters 21.6k Views
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • J
    jimp Rebel Alliance Developer Netgate
    last edited by jimp Oct 5, 2018, 1:30 PM Oct 5, 2018, 1:30 PM

    Palo Alto seems unhappy as well. I have a new patch to test but it does need testing. It comes up and works for me but I don't have access to any of these other devices (ubnt, sonicwall, PA, etc). Also need to be sure it doesn't interfere with other non-IPsec traffic and other non-VTI IPsec tunnels.

    From my other post:

    Try the attached patch and see if it helps. I could not get the VTI to come up and pass traffic with only 0.0.0.0/0 in the rightsubnet and leftsubnet, but it did seem to connect and work with the attached patch that has both the VTI endpoints and all zeroes. I haven't testing to see if it interferes with anything else yet, though, just VTI itself (BGP connects and exchanges routes, traffic passes)

    0_1538745996158_ipsec-vti-0.0.0.0.diff

    Use the System Patches package to apply the diff, or make the changes by hand. After applying the patch, stop IPsec, then edit/save/apply the IPsec VTI P1 or P2 and it should restart with the new policy in place.

    Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

    Need help fast? Netgate Global Support!

    Do not Chat/PM for help!

    F 1 Reply Last reply Oct 8, 2018, 8:47 AM Reply Quote 0
    • T
      turbulence
      last edited by Oct 5, 2018, 2:22 PM

      Hi Jim,

      Long time pfSense user here.

      Thought I would sign up to the forum to contribute to this. I have just installed patch 0_1538745996158_ipsec-vti-0.0.0.0.diff and setup a VTI between the pfSense and an EdgeRouter 4 (running the latest firmware) and I can report that the VPN is now working correctly. I'll let you know if I come across any subsequent strange behaviors, but everything is looking good so far.

      B 1 Reply Last reply Oct 12, 2018, 7:14 AM Reply Quote 1
      • F
        fsamareanu @jimp
        last edited by fsamareanu Oct 8, 2018, 9:06 AM Oct 8, 2018, 8:47 AM

        @jimp I've encountered a similar issue (I could ping the tunnel IP addresses but nothing else) by doing a pfsense-debian buster ipsec connection. I can prepare a testcase on a vultr VM pair if required and ship you the credentials.

        EDIT: I still see these in the log file(s) when I go to status-ipsec:

        Oct 8 13:04:37 192.168.100.1 charon: 04[KNL] <con3000|3> querying policy 0.0.0.0/0|/0 === 0.0.0.0/0|/0 in failed, not found
        Oct 8 13:04:37 192.168.100.1 charon: 04[KNL] <con3000|3> querying policy 0.0.0.0/0|/0 === 0.0.0.0/0|/0 out failed, not found
        Oct 8 13:04:37 192.168.100.1 charon: 04[KNL] <con2000|2> querying policy 0.0.0.0/0|/0 === 0.0.0.0/0|/0 in failed, not found
        Oct 8 13:04:37 192.168.100.1 charon: 04[KNL] <con2000|2> querying policy 0.0.0.0/0|/0 === 0.0.0.0/0|/0 out failed, not found
        Oct 8 13:04:37 192.168.100.1 charon: 04[KNL] <con1000|1> querying policy 0.0.0.0/0|/0 === 0.0.0.0/0|/0 in failed, not found
        Oct 8 13:04:37 192.168.100.1 charon: 04[KNL] <con1000|1> querying policy 0.0.0.0/0|/0 === 0.0.0.0/0|/0 out failed, not found

        But the tunnels are up and passing traffic.

        J 1 Reply Last reply Oct 8, 2018, 3:37 PM Reply Quote 0
        • J
          jimp Rebel Alliance Developer Netgate @fsamareanu
          last edited by Oct 8, 2018, 3:37 PM

          @fsamareanu said in IPSEC VTI Tunnels:

          @jimp I've encountered a similar issue (I could ping the tunnel IP addresses but nothing else) by doing a pfsense-debian buster ipsec connection. I can prepare a testcase on a vultr VM pair if required and ship you the credentials.

          But the tunnels are up and passing traffic.

          Is this with the new patch applied? If not, apply that patch.

          EDIT: I still see these in the log file(s) when I go to status-ipsec:

          Oct 8 13:04:37 192.168.100.1 charon: 04[KNL] <con3000|3> querying policy 0.0.0.0/0|/0 === 0.0.0.0/0|/0 in failed, not found

          I'm not terribly surprised there, since VTI doesn't actually install the policy in the kernel since it isn't needed. That may be prohibitively difficult to suppress that warning but if I do end up committing this patch we can look into it after.

          Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

          Need help fast? Netgate Global Support!

          Do not Chat/PM for help!

          F 1 Reply Last reply Oct 8, 2018, 4:10 PM Reply Quote 0
          • F
            fsamareanu @jimp
            last edited by fsamareanu Oct 8, 2018, 4:12 PM Oct 8, 2018, 4:10 PM

            @jimp the warning is with the patch applied. The error was there before as well, just showing the /30 subnet and the corresponding remote tunnel ip.

            I have not tested the pfsense-Linux ipsec tunnel after the pfsense patch. Will get to it tomorrow and update here.

            1 Reply Last reply Reply Quote 0
            • B
              BluRay @turbulence
              last edited by BluRay Oct 12, 2018, 7:15 AM Oct 12, 2018, 7:14 AM

              @turbulence said in IPSEC VTI Tunnels:

              Hi Jim,

              Long time pfSense user here.

              Thought I would sign up to the forum to contribute to this. I have just installed patch 0_1538745996158_ipsec-vti-0.0.0.0.diff and setup a VTI between the pfSense and an EdgeRouter 4 (running the latest firmware) and I can report that the VPN is now working correctly. I'll let you know if I come across any subsequent strange behaviors, but everything is looking good so far.

              Mind sharing your configuration?
              I am trying to get IPSec VTI running between PfSense and EdgeRouter X but i'm not able to get it working. (I already applied latest patch)
              alt text
              alt text
              alt text

              Logging:

              Oct 12 09:11:40	charon		12[KNL] creating acquire job for policy X.X.X.X/32|/0 === X.X.X.X/32|/0 with reqid {0}
              Oct 12 09:11:40	charon		12[KNL] received an SADB_ACQUIRE with policy id 8936 but no matching policy found
              Oct 12 09:11:38	charon		06[CFG] vici client 148 disconnected
              Oct 12 09:11:38	charon		16[CFG] vici client 148 requests: list-sas
              Oct 12 09:11:38	charon		16[CFG] vici client 148 registered for: list-sa
              Oct 12 09:11:38	charon		08[CFG] vici client 148 connected
              
              1 Reply Last reply Reply Quote 0
              • T
                turbulence
                last edited by turbulence Oct 12, 2018, 7:36 AM Oct 12, 2018, 7:30 AM

                Sure thing.

                Here's the ER4 config to start with. BTW, you need to be using IKEV2.

                ==PEER CONFIG==
                show vpn ipsec site-to-site peer X.X.X.X
                authentication {
                mode pre-shared-secret
                pre-shared-secret SECRETGOESHERE
                }
                connection-type initiate
                description TUNNEL-NAME-HERE
                ike-group FOO4
                ikev2-reauth yes
                local-address Y.Y.Y.Y
                vti {
                bind vti4
                esp-group FOO4
                }

                ==ESP CONFIG==
                show vpn ipsec esp-group FOO4
                compression disable
                lifetime 28800
                mode tunnel
                pfs dh-group14
                proposal 1 {
                encryption aes256
                hash sha256
                }

                ==IKE CONFIG==
                show vpn ipsec ike-group FOO4
                ikev2-reauth yes
                key-exchange ikev2
                lifetime 28800
                proposal 1 {
                dh-group 14
                encryption aes256
                hash sha256
                }

                ==VTI CONFIG==
                show interfaces vti vti4
                address 10.10.202.2/30
                mtu 1436

                ==ROUTE CONFIG==
                show protocols static interface-route 172.24.16.0/24
                next-hop-interface vti4 {
                description TUNNEL-NAME-HERE
                }

                1 Reply Last reply Reply Quote 0
                • T
                  turbulence
                  last edited by Oct 12, 2018, 7:34 AM

                  And here's the PFSense configuration.

                  Let me know if you need any further assistance!

                  2_1539329677025_Routes.PNG 1_1539329677024_Phase2.png 0_1539329677024_Phase1.png

                  B 1 Reply Last reply Oct 12, 2018, 7:37 AM Reply Quote 0
                  • B
                    BluRay @turbulence
                    last edited by Oct 12, 2018, 7:37 AM

                    @turbulence said in IPSEC VTI Tunnels:

                    And here's the PFSense configuration.

                    Let me know if you need any further assistance!

                    Thank you for the information. Will try it out later today and report back.

                    B 1 Reply Last reply Oct 13, 2018, 12:57 AM Reply Quote 1
                    • B
                      BluRay @BluRay
                      last edited by BluRay Oct 13, 2018, 12:59 AM Oct 13, 2018, 12:57 AM

                      @turbulence said in IPSEC VTI Tunnels:

                      And here's the PFSense configuration.

                      Let me know if you need any further assistance!

                      Ok, I tried to do it with IKEv2 and instead of Type 'Network' on Remote Network in PfSense Phase 2 setting, I used Type 'Address'.

                      With those settings the tunnel will come online. But I'm still not able to pass traffic.
                      What I found out is the following:
                      When I start a packetdump on PfSense I see ICMP traffic. (192.168.111.1 --> 192.168.111.2)
                      On EdgeRouter X side I see the ICMP messages arrive and the router also responds, but the response packets never reach the IPSEC1000 interface on the PfSense.

                      PfSense
                      alt text

                      EdgeRouter
                      alt text

                      1 Reply Last reply Reply Quote 0
                      • V
                        volga629
                        last edited by Oct 13, 2018, 12:55 PM

                        Hello Everyone,
                        I got vti setup working between pfsense and edgerouter pro with ebgp in place . Patch for 0.0.0.0/0 is required.

                        0_1539434435515_vti_pfsense.png

                        key point was Address in phase 2 and on edge router firewall allow BGP in VPN zone. VPN configuration on edge router is standard vti setup.

                        Phase 2

                        0_1539435267226_vti_phase2.png

                        The only one think is that tunnel goes down time to time. In logs I see

                        Oct 13 08:48:58 php-fpm 35862 /rc.newipsecdns: Gateway, none 'available' for inet6, use the first one configured. ''
                        Oct 13 08:48:58 php-fpm 35862 /rc.newipsecdns: The command '/sbin/ifconfig 'ipsec3000' create reqid '3000'' returned exit code '1', the output was 'ifconfig: create: bad value'
                        Oct 13 08:48:57 check_reload_status Reloading filter
                        Oct 13 08:48:57 php-fpm 35862 /rc.newipsecdns: IPSEC: One or more IPsec tunnel endpoints has changed its IP. Refreshing.
                        Oct 13 08:48:42 php-fpm 61417 /rc.openvpn: Gateway, none 'available' for inet6, use the first one configured. ''
                        Oct 13 08:48:41 check_reload_status Reloading filter

                        1 Reply Last reply Reply Quote 0
                        • B
                          BluRay
                          last edited by BluRay Oct 14, 2018, 10:21 AM Oct 13, 2018, 3:05 PM

                          My problem is solved. On the other side (EdgeRouter side) I use Dual-WAN with LB. I'm migrating this tunnel from VyOS to PfSense. The PfSense public IP was being loadbalanced. This caused the IPSec traffic to go over the wrong outside interface. After adding the public IP to the LB Exclude list, things started working.

                          Now I am running into another (NAT) problem. Maybe one of you can test if you get the same result. If I add an NAT rule to masquerade traffic with IP-address of VTI interface I am not able to reach anything on the EdgeRouter side.

                          What I am doing to test:
                          Ping from PfSense CLI to 192.168.111.2 (EdgeRouter). This works without the NAT rule. When I enable the NAT rule it stops working.

                          So far my observations are:
                          Traffic is leaving IPsec1000 interface on PfSense with correct NAT address (192.168.111.1). Traffic arrives on interface EdgeRouter and EdgeRouter sends traffic back which also arrives back at the IPsec1000 interface. But PfSense never gets reply on ICMP-echo request.

                          NAT rule looks like this. (Interface is VTI interface)
                          alt text

                          @turbulence

                          J 1 Reply Last reply Oct 15, 2018, 12:36 PM Reply Quote 0
                          • V
                            volga629
                            last edited by Oct 14, 2018, 12:51 AM

                            I am watching tunnel and it not 100% stable. Sometimes pfsense stop reply to traffic from vpn tunnel. I see traffic arrive on vti interface, but never send reply.

                            1 Reply Last reply Reply Quote 0
                            • J
                              jimp Rebel Alliance Developer Netgate @BluRay
                              last edited by Oct 15, 2018, 12:36 PM

                              @bluray said in IPSEC VTI Tunnels:

                              Now I am running into another (NAT) problem. Maybe one of you can test if you get the same result. If I add an NAT rule to masquerade traffic with IP-address of VTI interface I am not able to reach anything on the EdgeRouter side.

                              That's a known issue, NAT doesn't work with VTI currently. There is some weirdness between if_ipsec/VTI and pf where the traffic hits both the ipsecX interface and enc0. You might try putting the NAT rule on the "IPsec" interface in the GUI and not the assigned VTI interface.

                              Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                              Need help fast? Netgate Global Support!

                              Do not Chat/PM for help!

                              B M 2 Replies Last reply Oct 15, 2018, 6:28 PM Reply Quote 0
                              • B
                                BluRay @jimp
                                last edited by Oct 15, 2018, 6:28 PM

                                @jimp said in IPSEC VTI Tunnels:

                                @bluray said in IPSEC VTI Tunnels:

                                Now I am running into another (NAT) problem. Maybe one of you can test if you get the same result. If I add an NAT rule to masquerade traffic with IP-address of VTI interface I am not able to reach anything on the EdgeRouter side.

                                That's a known issue, NAT doesn't work with VTI currently. There is some weirdness between if_ipsec/VTI and pf where the traffic hits both the ipsecX interface and enc0. You might try putting the NAT rule on the "IPsec" interface in the GUI and not the assigned VTI interface.

                                Hello Jim,

                                Thank you for your feedback.
                                Unfortunately that doesn't work in my case. Is there a bugreport for this so I can track this issue? If not how can I make one?

                                1 Reply Last reply Reply Quote 0
                                • J
                                  jimp Rebel Alliance Developer Netgate
                                  last edited by Oct 15, 2018, 6:29 PM

                                  I don't recall if I made an issue for it on Redmine, but it's not one that we can address. It will need to be taken upstream to FreeBSD directly.

                                  Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                                  Need help fast? Netgate Global Support!

                                  Do not Chat/PM for help!

                                  B 1 Reply Last reply Oct 15, 2018, 6:35 PM Reply Quote 0
                                  • B
                                    BluRay @jimp
                                    last edited by Oct 15, 2018, 6:35 PM

                                    I already searched in Redmine, but I couldn't find one. Your explanation explains why there is no. (Due to it being a bug in FreeBSD IPSec implementation which must be fixed by FreeBSD Developers)

                                    I'm not familiair with reporting bugs. Will do some research how to submit one. Hopefully this can be resolved in the future.

                                    1 Reply Last reply Reply Quote 0
                                    • T
                                      The_Saint
                                      last edited by Nov 19, 2018, 8:19 PM

                                      Bumping this because I'm experiencing a similar problem.

                                      I have 2 IPsec Tunnels to an ISP.

                                      IPSec Phase 1 works good. When Phase 2 would come up for a short period of time, then fail. and the whole tunnel would collapse and restart.

                                      I observed a bunch of logs stating

                                      Nov 19 19:40:41	charon		14[KNL] <con2000|3> querying policy 10.0.255.248/30|/0 === 10.0.255.248/30|/0 in failed, not found
                                      Nov 19 19:40:41	charon		14[KNL] <con2000|3> querying policy 10.0.255.248/30|/0 === 10.0.255.248/30|/0 in failed, not found
                                      Nov 19 19:40:41	charon		14[KNL] <con2000|3> querying policy 10.0.255.248/30|/0 === 10.0.255.248/30|/0 in failed, not found
                                      Nov 19 19:40:41	charon		14[KNL] <con2000|3> querying policy 10.0.255.248/30|/0 === 10.0.255.248/30|/0 in failed, not found
                                      

                                      If I had the VTI settings to retain /0 on the remote, that's when my tunnels would constantly fail and restart after about a minute or so. So I executed the PFShell commands above to set /30. After doing so the IPSec/VTI 'appear' stable. But I can see my Security Association Database creating tons of new SPIs. So something still, just isn't right.

                                      I've setup my interfaces, OPT1 and OPT2, but I'm baffled as to why I don't see any Firewall Rule options for OPT1 like I saw in @ScottS post.
                                      0_1542657266406_1a21cd25-da3c-436c-842f-a3bcc077f072-image.png

                                      0_1542657320901_ae44bf7d-6320-4b07-9102-10c728ba698a-image.png

                                      I guess I'm curious if I should expect to see OPT1/OPT2 under my firewall rules and further, and further wonder if my accumulation of SPIs is an indication that things aren't quite right.

                                      I can PING

                                      Any help, or insights would be appreciated.

                                      1 Reply Last reply Reply Quote 0
                                      • B
                                        bradlay
                                        last edited by Nov 20, 2018, 2:42 AM

                                        I also have a similar issue with the same error message - the tunnel comes up fine, routing is working bi-directionally but if I attempt to do an iperf across the tunnel stops transferring packets almost immediately.
                                        Tried changing the MTU down to 1300, but didn't change anything.

                                        I have 2 SG-3100's setup with ikev2 VTI.

                                        $ tracert 192.168.11.10

                                        Tracing route to 192.168.11.10 over a maximum of 30 hops

                                        1 <1 ms <1 ms <1 ms 192.168.1.1
                                        2 35 ms 20 ms 31 ms 10.255.255.2
                                        3 18 ms 28 ms 32 ms 192.168.11.10

                                        $ tracert 192.168.1.150

                                        Tracing route to 192.168.1.150 over a maximum of 30 hops

                                        1 <1 ms <1 ms <1 ms 192.168.11.1
                                        2 18 ms 23 ms 29 ms 10.255.255.1
                                        3 25 ms 35 ms 29 ms 192.168.1.150

                                        Nov 19 21:37:02 charon 13[KNL] <con2000|2> querying policy 10.255.255.2/32|/0 === 10.255.255.1/32|/0 in failed, not found

                                        $ iperf3 -c 192.168.11.10
                                        Connecting to host 192.168.11.10, port 5201
                                        [ 4] local 192.168.1.150 port 51858 connected to 192.168.11.10 port 5201
                                        [ ID] Interval Transfer Bandwidth
                                        [ 4] 0.00-1.01 sec 256 KBytes 2.07 Mbits/sec
                                        [ 4] 1.01-2.00 sec 0.00 Bytes 0.00 bits/sec
                                        [ 4] 2.00-3.00 sec 0.00 Bytes 0.00 bits/sec
                                        [ 4] 3.00-4.00 sec 0.00 Bytes 0.00 bits/sec
                                        [ 4] 4.00-5.00 sec 0.00 Bytes 0.00 bits/sec
                                        [ 4] 5.00-6.00 sec 0.00 Bytes 0.00 bits/sec
                                        [ 4] 6.00-7.01 sec 0.00 Bytes 0.00 bits/sec
                                        [ 4] 7.01-8.00 sec 0.00 Bytes 0.00 bits/sec
                                        [ 4] 8.00-9.00 sec 0.00 Bytes 0.00 bits/sec
                                        [ 4] 9.00-10.02 sec 0.00 Bytes 0.00 bits/sec


                                        [ ID] Interval Transfer Bandwidth
                                        [ 4] 0.00-10.02 sec 256 KBytes 209 Kbits/sec sender
                                        [ 4] 0.00-10.02 sec 7.97 KBytes 6.52 Kbits/sec receiver

                                        The_Saint - check that your local and remote assignment is different, in the screenshot it looks like you have both local and remote as the same /32 (10.0.255.248)?

                                        T 1 Reply Last reply Nov 20, 2018, 3:49 PM Reply Quote 0
                                        • T
                                          The_Saint @bradlay
                                          last edited by Nov 20, 2018, 3:49 PM

                                          @bradlay
                                          Thank you for your reply Bradly. This is my config
                                          0_1542728259767_5661fd80-7d62-4ff0-b145-b4b41638cf9f-image.png
                                          I used PFShell to edit the fields manually though it doesn't seem to take the

                                          $config['ipsec']['phase2'][0]['remoteid']['type'] = "network";
                                          $config['ipsec']['phase2'][1]['remoteid']['type'] = "network";
                                          

                                          0_1542728435637_899e9ff2-098b-4885-b27c-b5a0c00910ef-image.png

                                          For your issue, does your FW actually have interfaces for your VTI (besides the IPSec)?
                                          Also, do you have MSS settings enabled?

                                          1 Reply Last reply Reply Quote 0
                                          • First post
                                            Last post
                                          Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.
                                            This community forum collects and processes your personal information.
                                            consent.not_received