Site-to-Site OpenVPN Connectivity Problem



  • I have followed standard setup instructions for a site-to-site OpenVPN config – there are plenty of guides out there and they are all very similar in nature; my settings are below. The VPN connection gets established fine, but I can’t reach anything from either side.

    This is packet capture on SITE A pfsense with ping running from 10.0.100.121 (site A) to 10.0.105.51 (site B):

    0_1538773705186_2018-10-05_15-39-51.jpg
    0_1538773714666_2018-10-05_15-42-44.jpg

    I have the packet capture running at the same time on pfsense in Site B on the ovpnc1 interface, which is the only OpenVPN interface on the client – it gets nothing. If I try reverse ping from Site B to Site A, same thing happens, i.e. I can see packets on Site B, but they never make it to site A.

    I’ve tried disabling block bogon networks on both WAN interfaces, but that did not make a difference. Both sites are at the same version (2.4.4).

    Anything else I can try?

    Thanks in advance!

    SITE A Config:

    WAN: 162.xxx.xxx.xxx
    LAN: 10.0.100.x/24
    Ovpn (Server)
    Mode: P2P SSL/TLS
    Proto: UDP on IP4
    Dev mode: tun
    Interface: WAN
    port: 1195
    IP4 Tunnel network: 172.16.0.0/24
    IP4 Local network: 10.0.100.0/24
    Remote networks: 10.0.105.0/24
    Topology: Subnet

    Routes:
    default 162.xxx.xxx.1 UGS 104441 1500 em0
    10.0.100.0/24 link#2 U 44993217 1500 em1
    10.0.100.1 link#2 UHS 0 16384 lo0
    10.0.101.0/24 10.0.101.2 UGS 0 1500 ovpns1
    10.0.101.1 link#7 UHS 0 16384 lo0
    10.0.101.2 link#7 UH 0 1500 ovpns1
    10.0.102.0/24 10.0.102.2 UGS 0 1500 ovpns3
    10.0.102.1 link#9 UHS 0 16384 lo0
    10.0.102.2 link#9 UH 28425 1500 ovpns3
    10.0.105.0/24 172.16.0.2 UGS 0 1500 ovpns2
    127.0.0.1 link#4 UH 4 16384 lo0
    162.xxx.xxx.0/22 link#1 U 26857 1500 em0
    162.xxx.xxx.xxx link#1 UHS 0 16384 lo0
    172.16.0.0/24 172.16.0.2 UGS 0 1500 ovpns2
    172.16.0.1 link#8 UHS 0 16384 lo0
    172.16.0.2 link#8 UH 0 1500 ovpns2

    FW Rules:
    1_1538774036712_2018-10-05_12-20-01.jpg 0_1538774036711_2018-10-05_12-19-14.jpg 0_1538774069839_2018-10-05_12-17-14.jpg

    SITE B Config:
    WAN: 10.10.186.94 (behind NAT)
    LAN: 10.0.105.x/24
    Ovpn (Client):
    Mode: P2P SSL/TLS
    Proto: UDP on IP4
    Dev mode: tun
    Interface: WAN
    Host: 162.xxx.xxx.xxx
    port: 1195
    IP4 Tunnel network: 172.16.0.0/24
    Remote networks: 10.0.100.0/24
    Topology: Subnet

    Routes:
    default 10.10.161.1 UGS 5121 1500 re0
    10.0.100.0/24 172.16.0.1 UGS 0 1500 ovpnc1
    10.0.105.0/24 link#2 U 264833 1500 re1
    10.0.105.1 link#2 UHS 0 16384 lo0
    10.10.160.0/19 link#1 U 26945 1500 re0
    10.10.186.94 link#1 UHS 0 16384 lo0
    10.17.33.5 00:30:18:a7:1a:49 UHS 0 1500 re0
    10.29.40.4 00:30:18:a7:1a:49 UHS 0 1500 re0
    127.0.0.1 link#4 UH 272 16384 lo0
    172.16.0.0/24 172.16.0.1 UGS 0 1500 ovpnc1
    172.16.0.1 link#7 UH 0 1500 ovpnc1
    172.16.0.2 link#7 UHS 0 16384 lo0

    FW Rules:

    1_1538774176236_2018-10-05_12-16-06.jpg 0_1538774176235_2018-10-05_12-11-47.jpg 0_1538774232556_2018-10-05_16-16-54.jpg



  • Set the tunnel to a /30 mask and assign interfaces to the OpenVPN instances on each site.


  • Netgate

    Yeah. The assigned interfaces are probably unnecessary but having a tunnel network larger than /30 puts an SSL/TLS server into server mode (PtMP) and you would need to add a CSO for the remote networks even if there is only one client.

    Simply changing that tunnel network to /30 kick the server into PtP will relieve you of that burden which is likely overkill on a true PtP connection.

    Also, in SSL/TLS mode you shouldn't need to specify a tunnel or remote networks on the client. Those should be pushed from the server. This allows a more centralized configuration.



  • Thanks for all the replies. I did both - the assignment and the tunnel set to /30 on both sites. Still no-go on the ping from one client to the other.

    However I am seeing a lot more ICMP activity on the packet capture now, even when no client pings are running....is that normal?

    Site A
    16:57:53.190902 IP 172.16.0.1 > 172.16.0.2: ICMP echo request, id 60436, seq 879, length 8
    16:57:53.243548 IP 172.16.0.6 > 172.16.0.5: ICMP echo request, id 19740, seq 808, length 8
    16:57:53.243586 IP 172.16.0.6 > 172.16.0.5: ICMP echo request, id 19740, seq 808, length 8
    16:57:53.719702 IP 172.16.0.1 > 172.16.0.2: ICMP echo request, id 60436, seq 880, length 8
    16:57:53.775623 IP 172.16.0.6 > 172.16.0.5: ICMP echo request, id 19740, seq 809, length 8
    16:57:53.775641 IP 172.16.0.6 > 172.16.0.5: ICMP echo request, id 19740, seq 809, length 8
    16:57:54.250884 IP 172.16.0.1 > 172.16.0.2: ICMP echo request, id 60436, seq 881, length 8

    Site B
    16:54:32.183967 IP 172.16.0.6 > 172.16.0.5: ICMP echo request, id 19740, seq 428, length 8
    16:54:32.716163 IP 172.16.0.6 > 172.16.0.5: ICMP echo request, id 19740, seq 429, length 8
    16:54:33.248410 IP 172.16.0.6 > 172.16.0.5: ICMP echo request, id 19740, seq 430, length 8
    16:54:33.780685 IP 172.16.0.6 > 172.16.0.5: ICMP echo request, id 19740, seq 431, length 8
    16:54:34.312925 IP 172.16.0.6 > 172.16.0.5: ICMP echo request, id 19740, seq 432, length 8
    16:54:34.845193 IP 172.16.0.6 > 172.16.0.5: ICMP echo request, id 19740, seq 433, length 8
    16:54:35.376665 IP 172.16.0.6 > 172.16.0.5: ICMP echo request, id 19740, seq 434, length 8
    16:54:35.908906 IP 172.16.0.6 > 172.16.0.5: ICMP echo request, id 19740, seq 435, length 8



  • I've also added IPv4 Any/any/any rules to the OVPNS2S interfaces on each side (that's the names i've assigned to them).

    Still no replies from pings.


  • Netgate

    Those look like gateway monitoring. you can disable that if you don't want it. Else filter on the source or target host IP address so you don't have to look at them.

    Are you sure the far side will respond to pings from a remote subnet? Windows firewall?

    Start at the source. pcap on the lan receiving the echo request. Then pcap on the OpenVPN on that side. Then pcap at the openvpn on the far side. Then the LAN on the far side. If the pings are going through but there is no response there, check the target host. Else see where the replies die.



  • Yup, you're right - those echos were gateway monitoring agents; they're off on both sides for now. To answer your questions:

    Yes, i'm sure the far side should respond. Windows firewall is off and I can ping it from pfsenseB>clientB.

    Start at the source - here are the results. I kind of did those before because other posts suggested that as a form of troubleshooting, but now i did it more thorough....here are the findings:

    pfsenseA > clientA = success
    pfsenseB > clientB = success
    pfsenseA > clientB = fail
    pfsenseB > clientA = fail

    I can try pinging some other-side gateway IPs from each pfsenseX, but i'm not sure which ones to ping. From other forum posts I also gathered that some shouldn't even reply so I don't want to add to confusion by including that in the above tests.

    So so far it still looks like there is no connectivity between the two sides. I also checked the firewall logs and I don't see any rejections that jump out, i.e on ovnpns2(A) or ocpnc1(B) interfaces....

    Appreciate your help. Anything else I can try, or should i be pcap'ing with different switches?


  • Netgate

    No. You need to packet capture the pings at progressive hop points, not report success or failure of the pings.



  • Ok, gotcha. Then the results are the same. I can't see any pings crossing the tunnel. I see them on each sides pfsense, but they don't come out on the other end.


  • Netgate

    That is not enough information to provide any help.

    Post a pcap of something like a ping -c3 dest at each hop, detailing exactly where each pcap is taken.



  • Sorry for the delay in my response - was out of office. Please let me know if you need more info, or if i misunderstood anything.

    clientA > pfsenseB (ping -c 1 10.0.105.51)

    clientA:

    C:\WINDOWS\system32>ping -c 1 10.0.105.1

    Pinging 10.0.105.1 with 32 bytes of data:
    Request timed out.
    Request timed out.
    Request timed out.
    Request timed out.

    Ping statistics for 10.0.105.1:
    Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),

    C:\WINDOWS\system32>

    pfsenseA:
    tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
    listening on ovpns2, link-type NULL (BSD loopback), capture size 262144 bytes
    14:17:33.076313 IP 10.0.100.121 > 10.0.105.1: ICMP echo request, id 1, seq 779, length 40
    14:17:37.670815 IP 10.0.100.121 > 10.0.105.1: ICMP echo request, id 1, seq 780, length 40
    14:17:42.681825 IP 10.0.100.121 > 10.0.105.1: ICMP echo request, id 1, seq 781, length 40
    14:17:47.687779 IP 10.0.100.121 > 10.0.105.1: ICMP echo request, id 1, seq 782, length 40

    pfsenseB:
    tcpdump is empty

    clientA > clientB (ping -c 1 10.0.105.51)

    ClientA:
    C:\WINDOWS\system32>ping -c 1 10.0.105.51

    Pinging 10.0.105.51 with 32 bytes of data:
    Request timed out.
    Request timed out.
    Request timed out.
    Request timed out.

    Ping statistics for 10.0.105.51:
    Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),

    C:\WINDOWS\system32>

    pfsenseA:
    tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
    listening on ovpns2, link-type NULL (BSD loopback), capture size 262144 bytes
    14:21:43.005849 IP 10.0.100.121 > 10.0.105.51: ICMP echo request, id 1, seq 787, length 40
    14:21:47.666048 IP 10.0.100.121 > 10.0.105.51: ICMP echo request, id 1, seq 788, length 40
    14:21:52.670818 IP 10.0.100.121 > 10.0.105.51: ICMP echo request, id 1, seq 789, length 40
    14:21:57.685270 IP 10.0.100.121 > 10.0.105.51: ICMP echo request, id 1, seq 790, length 40

    pfsenseB:
    tcpdump is empty.


  • Netgate

    @jamesva said in Site-to-Site OpenVPN Connectivity Problem:

    pfsenseB:
    tcpdump is empty.

    Listening where?

    Sorry. You don't need the -c 1 flag. I forget sometimes that some people still use Windows.

    Don't need any more ping results. We know it isn't working.



  • pfsenseB is the other side psfsense console.


  • Netgate

    Right but capturing on what interface? Need specifics here.

    And can you ping the pfSense LAN interface address on the other side?



  • The captures are running on the ovpn interfaces. Should i have been capturing on another interface?

    As far as ping LAN interface - no, i can't reach it from one side to another. However if the client is on the same side, pfsense's LAN int responds to pings just fine.



  • Having said that, it looks like I can ping clients on Side A if i'm sitting on the console of pfsenseB.....

    however it doesn't work the opposite way, i.e. if i'm on the console of pfsenseA, i can't reach anything on side B.


  • Netgate

    It doesn't make any sense that you are seeing traffic exit the VPN at A but it does not arrive in a capture on openvpn at B.



  • Could it have something to do with the fact that pfsenseB WAN IP is on 10.x.x.x subnet?


  • Netgate

    Not unless it's within or conflicting somehow eith the others. And you'd still see the packets in the capture on the OpenVPN.



  • hmm ok, what else can you suggest I try? I do have multiple ovpn instances on SideA, but i can't imagine that would conflict with any of them since they're running on different ports....


  • Netgate

    It's not about ports but addresses/networks. They all need to be unique.



  • they are; screenshot below.

    0_1539288653531_2018-10-11_15-09-37.jpg



  • what would be the next think i try that you would suggest?


  • Netgate

    Don't know. I asked for some files in chat. Did you upload those?



  • Done. Thanks for taking a look at it!



  • Just wanted to check in if you had to review the files i sent over.

    Thanks in advance!



  • guess not, no problem. thanks for giving it a shot, i do appreciate it.

    Just as an update... I thought... well maybe it's one of the packages that's interfering with the traffic... so I've set up a completely vanilla lab on the vmware workstation. The only package loaded on both sides is open-vm-tools....

    Same exact results.... so it makes me think i'm doing something wrong. I'll keep picking at it....

    Any other suggestions are welcome.



  • I kind of gave up on the openvpn and went with IPsec... seemed to work as expected on the first try.