Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Site2Site does not work/route in both directions

    Scheduled Pinned Locked Moved
    OpenVPN
    openvpn site-to-site routing
    2
    9
    876
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • D
      demux
      last edited by demux

      Hi!

      I do have a strange problem with OpenVPN routing and I am pretty sure it was working some time ago.

      It is a site to site link conecting 172.27.72.0/24 (remote) with 10.252.200.0/22. 10.252.200.254 is the OpenVPN server on pfSense, 172.27.72.254 is the client side pfSense. OpenVPN tunnel network is 172.30.14.0/24 (172.30.14.1=server, 172.30.14.2=client).

      From remote (172.27.72.x) I can ping and reach the full 10.252.20x.y network.
      But I cannot reach the remote devices from the server side's network. I can only ping the remote pfSense but no other device.

      What I get is this from server's network - ping:
      PING 172.27.72.1 (172.27.72.1) 56(84) bytes of data.
      From 172.30.14.1 icmp_seq=1 Time to live exceeded
      From 172.30.14.1 icmp_seq=2 Time to live exceeded
      From 172.30.14.1 icmp_seq=3 Time to live exceeded
      From 172.30.14.1 icmp_seq=4 Time to live exceeded
      ...

      What I get is this from server's network - traceroute:
      traceroute to 172.27.72.1 (172.27.72.1), 30 hops max, 60 byte packets
      1 10.252.200.254 0.338 ms 0.362 ms 0.356 ms
      2 172.30.14.1 27.970 ms 35.843 ms 42.439 ms
      3 172.30.14.1 70.223 ms 70.251 ms 74.688 ms
      ...
      29 172.30.14.1 881.594 ms 871.410 ms 885.545 ms
      30 172.30.14.1 907.684 ms 895.962 ms 896.176 ms

      This traceroute is really strange.

      ssh from pfSense server to a device on the client network says:
      ssh -l user 172.27.72.1
      ssh: connect to host 172.27.72.1 port 22: No route to host

      But I see the route, below (***).

      The routing table on the server looks like this (two WANs and two additional OpenVPN servers):
      Destination Gateway Flags Netif Expire
      default 24.134.xx.yyy UGS igb1
      1.0.0.1 82.82.xx.yyy UGHS pppoe0
      1.1.1.1 24.134.xx.yyy UGHS igb1
      4.2.2.1 24.134.xx.yyy UGHS igb1
      4.2.2.2 82.82.xx.yyy UGHS pppoe0
      4.2.2.3 24.134.xx.yyy UGHS igb1
      4.2.2.4 pppoe0 UHS pppoe0
      8.8.4.4 82.82.xx.yyy UGHS pppoe0
      8.8.8.8 24.134.xx.yyy UGHS igb1
      9.9.9.10 24.134.xx.yyy UGHS igb1
      10.252.200.0/22 link#3 U igb2
      10.252.200.254 link#3 UHS lo0
      10.254.154.54 link#3 UHS lo0
      10.254.154.54/32 link#3 U igb2
      24.134.xx.yyy/30 link#2 U igb1
      24.134.xx.yyy link#2 UHS lo0
      82.82.xx.yyy link#12 UH pppoe0
      127.0.0.1 link#6 UH lo0
      172.25.254.0/24 link#11 U igb2.222
      172.25.254.254 link#11 UHS lo0
      172.27.72.0/24 172.30.14.2 UGS ovpns6 (***)
      172.30.1.0/24 172.30.1.2 UGS ovpns1
      172.30.1.1 link#17 UHS lo0
      172.30.1.2 link#17 UH ovpns1
      172.30.2.0/24 172.30.2.2 UGS ovpns3
      172.30.2.1 link#16 UHS lo0
      172.30.2.2 link#16 UH ovpns3
      172.30.14.1 link#14 UHS lo0
      172.30.14.2 link#14 UH ovpns6
      172.30.15.1 link#15 UHS lo0
      172.30.15.2 link#15 UH ovpns7
      176.95.xxx.yyy link#12 UHS lo0
      192.168.55.0/24 link#10 U igb2.55
      192.168.55.254 link#10 UHS lo0

      On the client I have the following routing table (two additional OpenVPN servers):
      Destination Gateway Flags Netif Expire
      default 90.187.xxx.yyy UGS igb0
      10.252.200.0/22 172.30.14.1 UGS ovpnc1
      90.187.xxx.yyy/30 link#1 U igb0
      90.187.xxx.yyy link#1 UHS lo0
      127.0.0.1 link#5 UH lo0
      172.27.72.0/24 link#3 U igb2
      172.27.72.254 link#3 UHS lo0
      172.30.14.1 link#11 UH ovpnc1
      172.30.14.2 link#11 UHS lo0
      172.30.15.1 link#12 UH ovpnc2
      172.30.15.2 link#12 UHS lo0
      172.30.50.0/24 172.30.50.2 UGS ovpns3
      172.30.50.1 link#9 UHS lo0
      172.30.50.2 link#9 UH ovpns3
      172.30.51.0/24 172.30.51.2 UGS ovpns4
      172.30.51.1 link#10 UHS lo0
      172.30.51.2 link#10 UH ovpns4
      192.168.55.0/24 link#8 U igb2.55
      192.168.55.254 link#8 UHS lo0

      The server config looks like this:
      dev ovpns6
      verb 1
      dev-type tun
      dev-node /dev/tun6
      writepid /var/run/openvpn_server6.pid
      #user nobody
      #group nobody
      script-security 3
      daemon
      keepalive 10 60
      ping-timer-rem
      persist-tun
      persist-key
      proto udp4
      cipher AES-128-CBC
      auth SHA1
      up /usr/local/sbin/ovpn-linkup
      down /usr/local/sbin/ovpn-linkdown
      local 24.134.xxx.yyy
      engine cryptodev
      ifconfig 172.30.14.1 172.30.14.2
      lport 1196
      management /var/etc/openvpn/server6.sock unix
      max-clients 16
      route 172.27.72.0 255.255.255.0
      secret /var/etc/openvpn/server6.secret
      fast-io
      sndbuf 1048576
      rcvbuf 1048576

      The client config looks like this:
      dev ovpnc1
      verb 1
      dev-type tun
      dev-node /dev/tun1
      writepid /var/run/openvpn_client1.pid
      #user nobody
      #group nobody
      script-security 3
      daemon
      keepalive 10 60
      ping-timer-rem
      persist-tun
      persist-key
      proto udp4
      cipher AES-128-CBC
      auth SHA1
      up /usr/local/sbin/ovpn-linkup
      down /usr/local/sbin/ovpn-linkdown
      local 90.187.xxx.yyy
      engine cryptodev
      lport 0
      management /var/etc/openvpn/client1.sock unix
      remote server 1196
      ifconfig 172.30.14.2 172.30.14.1
      route 10.252.200.0 255.255.252.0
      secret /var/etc/openvpn/client1.secret
      resolv-retry infinite
      fast-io
      sndbuf 1048576
      rcvbuf 1048576

      On the client I have some sort of fail-over using a gateway group. See https://forum.netgate.com/topic/147314/fail-over-client-config/9

      Can someone please have a look and help me find what's wrong? What am I missing?
      I need to reach all devices from both sides of the VPN.

      Thanks a lot!

      1 Reply Last reply Reply Quote 0
      • kiokomanK
        kiokoman LAYER 8
        last edited by

        both side are on the same subnet ?

        it seems the packet are bounced between the hops

        3 172.30.14.1 70.223 ms 70.251 ms 74.688 ms
        ...
        29 172.30.14.1 881.594 ms 871.410 ms 885.545 ms
        30 172.30.14.1 907.684 ms 895.962 ms 896.176 ms

        it's a routing problem

        ̿' ̿'\̵͇̿̿\з=(◕_◕)=ε/̵͇̿̿/'̿'̿ ̿
        Please do not use chat/PM to ask for help
        we must focus on silencing this @guest character. we must make up lies and alter the copyrights !
        Don't forget to Upvote with the 👍 button for any post you find to be helpful.

        1 Reply Last reply Reply Quote 0
        • D
          demux
          last edited by demux

          No, they are not on the same subnet.
          Server is on 10.252.200.0/22, client is on 172.27.72.0/24.
          Server pfSense has 10.252.200.254, client pfSense has 172.27.72.254.
          Both share the same tunnel network (172.30.14.0/24) with server pfSense at 172.30.14.1 and client pfSense on 172.30.14.2.
          Yes, it seems to be a routing problem. But I cannot see where. The routing table looks good to me.
          From client pfSense I can ping the whole 172.27.72.0/24 network.
          From client network I can ping the whole 10.252.200.0/22 network via VPN.
          From server pfSense network I can ping nothing on the client network, except client pfSense on client network. Not even the server pfSense can ping anything else than the client pfSense.
          And I don't see why!

          I just checked, the firewalls on both ends do not block anything.

          1 Reply Last reply Reply Quote 0
          • kiokomanK
            kiokoman LAYER 8
            last edited by

            @demux said in Site2Site does not work/route in both directions:
            client:
            172.27.72.0/24 link#3 U igb2
            172.27.72.254 link#3 UHS lo0
            172.30.14.1 link#11 UH ovpnc1
            172.30.14.2 link#11 UHS lo0
            10.252.200.0/22 172.30.14.1 UGS ovpnc1

            server:
            10.252.200.0/22 link#3 U igb2
            10.252.200.254 link#3 UHS lo0
            172.30.14.1 link#14 UHS lo0
            172.30.14.2 link#14 UH ovpns6
            172.27.72.0/24 172.30.14.2 UGS ovpns6

            from the client side it work but not from the server side
            🤔

            ̿' ̿'\̵͇̿̿\з=(◕_◕)=ε/̵͇̿̿/'̿'̿ ̿
            Please do not use chat/PM to ask for help
            we must focus on silencing this @guest character. we must make up lies and alter the copyrights !
            Don't forget to Upvote with the 👍 button for any post you find to be helpful.

            1 Reply Last reply Reply Quote 0
            • D
              demux
              last edited by

              Correct.

              1 Reply Last reply Reply Quote 0
              • kiokomanK
                kiokoman LAYER 8
                last edited by kiokoman

                when did it stop working? after an update maybe?
                a reboot didn't help i suppose?
                i can't see what's wrong ...
                can you make a screenshot of the firewall rules ?
                do you have any floating rules?

                ̿' ̿'\̵͇̿̿\з=(◕_◕)=ε/̵͇̿̿/'̿'̿ ̿
                Please do not use chat/PM to ask for help
                we must focus on silencing this @guest character. we must make up lies and alter the copyrights !
                Don't forget to Upvote with the 👍 button for any post you find to be helpful.

                1 Reply Last reply Reply Quote 0
                • D
                  demux
                  last edited by demux

                  I did reboot. Did not update.
                  I am not sure if it ever worked. I believe, but I'm not sure.
                  Maybe it stopped working after that gateway fail-over thing mentioned in the link. I did reverse that temporarily but did not help.
                  To me everything looks fine. And why the problems into this direction, not the other or both.
                  I have real problems testing that, because the distance between them is about 800km. If something breaks I have to travel about 9 hours. Like last weekend...

                  1 Reply Last reply Reply Quote 0
                  • D
                    demux
                    last edited by

                    I have no clue what happened.
                    I deleted all servers and clients and started over again.
                    I strictly followed this documentation:
                    https://docs.netgate.com/pfsense/en/latest/book/openvpn/openvpn-and-multi-wan.html#
                    Server uses GW group and client uses remote ...
                    And now it works.
                    Whatever happened to my previous config.

                    1 Reply Last reply Reply Quote 0
                    • kiokomanK
                      kiokoman LAYER 8
                      last edited by

                      the routing table now is the same ?
                      maybe it was something else on the configuration

                      ̿' ̿'\̵͇̿̿\з=(◕_◕)=ε/̵͇̿̿/'̿'̿ ̿
                      Please do not use chat/PM to ask for help
                      we must focus on silencing this @guest character. we must make up lies and alter the copyrights !
                      Don't forget to Upvote with the 👍 button for any post you find to be helpful.

                      1 Reply Last reply Reply Quote 0
                      • First post
                        Last post