Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Cannot route across IPSec VPN?

    Scheduled Pinned Locked Moved
    IPsec
    3
    7
    1.4k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • D
      djinnsour
      last edited by

      We have some new servers at Rackspace. With the setup they provided a Vyatta image for us to setup a VPN between our sites.

      At Rackspace we have two networks, 192.168.30.0/24 and 10.188.66.121/32.
      At our colo facility our internal network is 192.168.200.0/24.
      At our main office the internal network is 192.168.0.0/23.
      The connection between the main office subnet and colo subnet is an MPLS with a bare bones Linux router on each end. No NAT between those subnets. No iptables firewall.

      The connection from our pfSense to the Vyatta is an IPSec VPN, that appears to be working correctly. We can ping, SSH, and everything else between the two networks at Rackspace and the network at the colo without any problem.  We can even ping and SSH  networks all we want.  We can even SSH from a system at Rackspace, to a system on our main office network.

      What we cannot do is anything other than ping from out main office subnet to the Rackspace network. But we can ping, which doesn't make any sense to me.

      IPSec phase 1 configuration:
      protocol ipv4
      interface wan
      remote gateway (public IP for Vyatta)
      authentication method mutal psk
      negotiation mode main
      my identifier (my public IP)
      peer identified (public IP for Vyatta)
      pre-shared key xxxxxxxxxxxxxxx
      encryption algorith AES 256
      hash algorith SHA1
      DH key group 2
      Lifetime 86400

      phase 2 -1
      mode tunnel ipv4
      local network type lan
      remote network 192.168.30.0/24
      protocol esp
      encryption algorithms AES 256 bits
      has algorithms sha1
      pfs key group 2
      lifetime 3600

      phase 2-2
      same except
      remote network 10.188.66.121/32

      phase 2-3
      same except
      local network type network, address 192.168.0.0/23
      remote network 192.168.30.0/24

      phase 2-4
      same except
      local network type network, address 192.168.0.0/23
      remote network 192.168.30.0/24

      Firewall rules:
      Floating - nothing
      WAN - nothing
      LAN - anti-lockout rule, and IPv4****** basically wide open
      IPsec - IPv4****** basically wide open

      Like I said, I can SSH from 192.168.30.1 to 192.168.200.x. I can SSH from 192.168.200.x to 192.168.30.x. I can initiate an SSH connection from 192.168.30.2 to 192.168.0.187. I can connect to the database at 10.188.66.121 from the systems on 192.168.200.x. I cannot initiate an SSH session or database connection from 192.168.0.187 to 192.168.30.1, but I can ping it. Same usernames, same passwords, same SSH key.

      Taking a look at the connections on Vyatta I don't even see the connection being attempted from 192.168.0.187 to 192.168.30.1. Looking at the filter.log on pfSense shows the following when I SSH from 192.168.200.42 to 192.168.30.2:

      Jul 13 16:47:09 rackspacevpn-colov filterlog: 84,16777216,,1000104255,em0,match,pass,out,4,0x0,,64,47585,0,none,50,esp,120,97.65.238.10,104.239.231.219,datalength=100

      Attempting to SSH into the same system from 192.168.0.187 generates the following entries, and fails:

      ul 13 16:48:00 rackspacevpn-colov filterlog: 71,16777216,,1000004121,em1,match,pass,in,4,0x60,,59,49500,0,DF,6,tcp,60,192.168.0.187,192.168.200.41,49402,22,0,S,1614338842,,29200,,mss;sackOK;TS;nop;wscale
      Jul 13 16:48:01 rackspacevpn-colov filterlog: 71,16777216,,1000004121,em1,match,pass,in,4,0x60,,59,49501,0,DF,6,tcp,60,192.168.0.187,192.168.200.41,49402,22,0,S,1614338842,,29200,,mss;sackOK;TS;nop;wscale
      Jul 13 16:48:03 rackspacevpn-colov filterlog: 71,16777216,,1000004121,em1,match,pass,in,4,0x60,,59,49502,0,DF,6,tcp,60,192.168.0.187,192.168.200.41,49402,22,0,S,1614338842,,29200,,mss;sackOK;TS;nop;wscale
      Jul 13 16:48:04 rackspacevpn-colov filterlog: 67,16777216,,1000003811,em0,match,pass,out,4,0xb8,,64,14482,0,none,17,udp,76,97.65.238.10,208.75.88.4,123,123,56

      Mtr shows it is connects in exactly the path expected, but I can do nothing other than ping. What am I missing here?

      1 Reply Last reply Reply Quote 0
      • C
        cmb
        last edited by

        It's passing inbound at least, from those logs. Packet capture on IPsec, you see the TCP traffic leaving there?

        1 Reply Last reply Reply Quote 0
        • D
          djinnsour
          last edited by

          Ran a quick capture on IPsec.  Once doing a ping for a few responses, then ssh to the same host.  The packet capture shows the ICMP but nothing else even though I was capturing any protocol.  Started over, just SSH this time, and the packet capture was completely empty.

          21:06:57.186096 (authentic,confidential): SPI 0xc9bf72fe: IP (tos 0x0, ttl 58, id 32441, offset 0, flags [DF], proto ICMP (1), length 84, bad cksum 20e3 (->21e3)!)
              192.168.0.187 > 192.168.30.1: ICMP echo request, id 20392, seq 1, length 64
          21:06:57.225725 (authentic,confidential): SPI 0xca814bc8: IP (tos 0x0, ttl 64, id 37729, offset 0, flags [none], proto ICMP (1), length 84)
              192.168.30.1 > 192.168.0.187: ICMP echo reply, id 20392, seq 1, length 64
          21:06:58.187242 (authentic,confidential): SPI 0xc9bf72fe: IP (tos 0x0, ttl 58, id 32593, offset 0, flags [DF], proto ICMP (1), length 84, bad cksum 204b (->214b)!)
              192.168.0.187 > 192.168.30.1: ICMP echo request, id 20392, seq 2, length 64
          21:06:58.226716 (authentic,confidential): SPI 0xca814bc8: IP (tos 0x0, ttl 64, id 37775, offset 0, flags [none], proto ICMP (1), length 84)
              192.168.30.1 > 192.168.0.187: ICMP echo reply, id 20392, seq 2, length 64
          21:06:59.189348 (authentic,confidential): SPI 0xc9bf72fe: IP (tos 0x0, ttl 58, id 32709, offset 0, flags [DF], proto ICMP (1), length 84, bad cksum 1fd7 (->20d7)!)
              192.168.0.187 > 192.168.30.1: ICMP echo request, id 20392, seq 3, length 64
          21:06:59.228169 (authentic,confidential): SPI 0xca814bc8: IP (tos 0x0, ttl 64, id 37788, offset 0, flags [none], proto ICMP (1), length 84)
              192.168.30.1 > 192.168.0.187: ICMP echo reply, id 20392, seq 3, length 64

          Did it again, but set the interface to LAN instead of IPsec.

          21:11:57.443179 00:15:17:90:09:04 > 52:54:00:15:3f:7f, ethertype IPv4 (0x0800), length 74: (tos 0x60, ttl 59, id 44548, offset 0, flags [DF], proto TCP (6), length 60)
              192.168.0.187.36582 > 192.168.200.41.22: Flags ~~, cksum 0x91e7 (correct), seq 3040750311, win 29200, options [mss 1460,sackOK,TS val 97354979 ecr 0,nop,wscale 7], length 0
          21:11:57.449071 00:15:17:90:09:04 > 52:54:00:15:3f:7f, ethertype IPv4 (0x0800), length 60: (tos 0xa0, ttl 59, id 46834, offset 0, flags [DF], proto TCP (6), length 40)
              192.168.0.187.36582 > 192.168.200.41.22: Flags [R], cksum 0xf687 (correct), seq 3040750312, win 0, length 0
          21:11:58.440038 00:15:17:90:09:04 > 52:54:00:15:3f:7f, ethertype IPv4 (0x0800), length 74: (tos 0x60, ttl 59, id 44549, offset 0, flags [DF], proto TCP (6), length 60)
              192.168.0.187.36582 > 192.168.200.41.22: Flags ~~, cksum 0x90ed (correct), seq 3040750311, win 29200, options [mss 1460,sackOK,TS val 97355229 ecr 0,nop,wscale 7], length 0
          21:11:58.445045 00:15:17:90:09:04 > 52:54:00:15:3f:7f, ethertype IPv4 (0x0800), length 60: (tos 0xa0, ttl 59, id 47039, offset 0, flags [DF], proto TCP (6), length 40)
              192.168.0.187.36582 > 192.168.200.41.22: Flags [R], cksum 0xf687 (correct), seq 3040750312, win 0, length 0
          21:12:00.444029 00:15:17:90:09:04 > 52:54:00:15:3f:7f, ethertype IPv4 (0x0800), length 74: (tos 0x60, ttl 59, id 44550, offset 0, flags [DF], proto TCP (6), length 60)
              192.168.0.187.36582 > 192.168.200.41.22: Flags ~~, cksum 0x8ef8 (correct), seq 3040750311, win 29200, options [mss 1460,sackOK,TS val 97355730 ecr 0,nop,wscale 7], length 0
          21:12:00.451737 00:15:17:90:09:04 > 52:54:00:15:3f:7f, ethertype IPv4 (0x0800), length 60: (tos 0xa0, ttl 59, id 47187, offset 0, flags [DF], proto TCP (6), length 40)
              192.168.0.187.36582 > 192.168.200.41.22: Flags [R], cksum 0xf687 (correct), seq 3040750312, win 0, length 0

          I am at a loss.  I even added some routes just to see what happened but it didn't have any effect.  No idea where to go from here.  I may have to figure out how to create a MIP and just treat the 192.168.0.0/23 traffic as if it were on 192.168.200.0/24.~~~~~~

          1 Reply Last reply Reply Quote 0
          • C
            cmb
            last edited by

            That's a different destination, thought you meant you could ping but not SSH to and from the same iP. Probably missing a matching P2 for that source and destination network.

            1 Reply Last reply Reply Quote 0
            • D
              djinnsour
              last edited by

              Nope, same destination.  The pfsense box is 192.168.200.41, Vyatta on the other end is 192.168.30.1.  My desktop in this test was 192.168.0.187. I was able to ping, but not SSH. First capture was from IPSec, second was from LAN.  Both of them same test.  ICMP shows up on the IPsec capture, nothing else. LAN capture is just showing the traffic coming from the desktop to the pfsense system on the LAN, nothing on the IPsec capture.

              I even fired up another system with 2.2.2 just now to see if the problem was being caused by the bug in 2.2.3 people have been reporting with IPsec. Same problem.

              Could you elaborate on the "missing a matching P2 for that source and destination network"? Thanks for all your help.

              1 Reply Last reply Reply Quote 0
              • G
                georgeman
                last edited by

                Are you running pfSense on bare hardware or virtualized? I have seen similar "non-senses" with the vmxnet driver supplied by VMware

                If it ain't broke, you haven't tampered enough with it

                1 Reply Last reply Reply Quote 0
                • D
                  djinnsour
                  last edited by

                  We've actually tried it both ways and are getting the same result.  Using Intel Server adapters on the hardware version, e1000 virtualized adapters on the VM (KVM). They Vyatta is virtuallized, but Rackspace and other people use them all the time for similar configurations without a problem.

                  1 Reply Last reply Reply Quote 0
                  • First post
                    Last post
                  Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.