Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Route WAN network to OVPN

    Scheduled Pinned Locked Moved General pfSense Questions
    19 Posts 2 Posters 1.8k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • stephenw10S
      stephenw10 Netgate Administrator
      last edited by

      I assume OPT1 is the assigned OpenVPN interface?

      Check the state table when you are trying to connect or successfully pinging. Is the traffic NAT'd to the OPT1 interface IP as expected?

      Run a packet capture on the OpenVPN interface. What actually being sent from there?

      Steve

      1 Reply Last reply Reply Quote 0
      • D
        DasK
        last edited by DasK

        I done the Packet Capture on the OPT1 interface while doing a ping and a traceroute on the proxmox serveur

        15:09:18.758929 IP 10.23.1.200 > 10.23.1.1: ICMP echo request, id 10458, seq 24999, length 8
15:09:18.768983 IP 10.23.1.1 > 10.23.1.200: ICMP echo reply, id 10458, seq 24999, length 8
15:09:19.289009 IP 10.23.1.200 > 10.23.1.1: ICMP echo request, id 10458, seq 25000, length 8
15:09:19.299123 IP 10.23.1.1 > 10.23.1.200: ICMP echo reply, id 10458, seq 25000, length 8
15:09:19.830176 IP 10.23.1.200 > 10.23.1.1: ICMP echo request, id 10458, seq 25001, length 8
15:09:19.840291 IP 10.23.1.1 > 10.23.1.200: ICMP echo reply, id 10458, seq 25001, length 8
15:09:20.371708 IP 10.23.1.200 > 10.23.1.1: ICMP echo request, id 10458, seq 25002, length 8
15:09:20.381646 IP 10.23.1.1 > 10.23.1.200: ICMP echo reply, id 10458, seq 25002, length 8
15:09:20.888891 IP 10.23.1.200 > 10.23.1.1: ICMP echo request, id 10458, seq 25003, length 8
15:09:20.898926 IP 10.23.1.1 > 10.23.1.200: ICMP echo reply, id 10458, seq 25003, length 8
15:09:21.418876 IP 10.23.1.200 > 10.23.1.100: ICMP echo request, id 16951, seq 1, length 64
15:09:21.419019 IP 10.23.1.200 > 10.23.1.1: ICMP echo request, id 10458, seq 25004, length 8
15:09:21.428766 IP 10.23.1.1 > 10.23.1.200: ICMP echo reply, id 10458, seq 25004, length 8
15:09:21.441468 IP 10.23.1.100 > 10.23.1.200: ICMP echo reply, id 16951, seq 1, length 64
15:09:21.937154 IP 10.23.1.200 > 10.23.1.1: ICMP echo request, id 10458, seq 25005, length 8
15:09:21.946987 IP 10.23.1.1 > 10.23.1.200: ICMP echo reply, id 10458, seq 25005, length 8
15:09:22.420103 IP 10.23.1.200 > 10.23.1.100: ICMP echo request, id 16951, seq 2, length 64
15:09:22.442994 IP 10.23.1.100 > 10.23.1.200: ICMP echo reply, id 16951, seq 2, length 64
15:09:22.448187 IP 10.23.1.200 > 10.23.1.1: ICMP echo request, id 10458, seq 25006, length 8
15:09:22.458130 IP 10.23.1.1 > 10.23.1.200: ICMP echo reply, id 10458, seq 25006, length 8
15:09:22.955888 IP 10.23.1.200 > 10.23.1.1: ICMP echo request, id 10458, seq 25007, length 8
15:09:22.965827 IP 10.23.1.1 > 10.23.1.200: ICMP echo reply, id 10458, seq 25007, length 8
15:09:23.420449 IP 10.23.1.200 > 10.23.1.100: ICMP echo request, id 16951, seq 3, length 64
15:09:23.443246 IP 10.23.1.100 > 10.23.1.200: ICMP echo reply, id 16951, seq 3, length 64
15:09:23.489014 IP 10.23.1.200 > 10.23.1.1: ICMP echo request, id 10458, seq 25008, length 8
15:09:23.498938 IP 10.23.1.1 > 10.23.1.200: ICMP echo reply, id 10458, seq 25008, length 8
15:09:24.030321 IP 10.23.1.200 > 10.23.1.1: ICMP echo request, id 10458, seq 25009, length 8
15:09:24.040293 IP 10.23.1.1 > 10.23.1.200: ICMP echo reply, id 10458, seq 25009, length 8
15:09:24.545142 IP 10.23.1.200 > 10.23.1.1: ICMP echo request, id 10458, seq 25010, length 8
15:09:24.555173 IP 10.23.1.1 > 10.23.1.200: ICMP echo reply, id 10458, seq 25010, length 8
15:09:25.058926 IP 10.23.1.200 > 10.23.1.1: ICMP echo request, id 10458, seq 25011, length 8
15:09:25.068957 IP 10.23.1.1 > 10.23.1.200: ICMP echo reply, id 10458, seq 25011, length 8
15:09:25.537847 IP 10.23.1.200.35330 > 10.23.1.100.33437: UDP, length 32
15:09:25.537863 IP 10.23.1.200.18025 > 10.23.1.100.33438: UDP, length 32
15:09:25.537871 IP 10.23.1.200.2270 > 10.23.1.100.33439: UDP, length 32
15:09:25.537879 IP 10.23.1.200.23400 > 10.23.1.100.33440: UDP, length 32
15:09:25.537886 IP 10.23.1.200.39105 > 10.23.1.100.33441: UDP, length 32
15:09:25.537894 IP 10.23.1.200.14561 > 10.23.1.100.33442: UDP, length 32
15:09:25.537917 IP 10.23.1.200.5445 > 10.23.1.100.33443: UDP, length 32
15:09:25.537925 IP 10.23.1.200.14700 > 10.23.1.100.33444: UDP, length 32
15:09:25.537933 IP 10.23.1.200.1751 > 10.23.1.100.33445: UDP, length 32
15:09:25.537954 IP 10.23.1.200.49371 > 10.23.1.100.33446: UDP, length 32
15:09:25.537961 IP 10.23.1.200.35188 > 10.23.1.100.33447: UDP, length 32
15:09:25.537969 IP 10.23.1.200.18223 > 10.23.1.100.33448: UDP, length 32
15:09:25.537992 IP 10.23.1.200.56560 > 10.23.1.100.33449: UDP, length 32
15:09:25.540117 IP 10.23.1.200.14513 > 10.23.1.100.33450: UDP, length 32
15:09:25.540135 IP 10.23.1.200.34198 > 10.23.1.100.33451: UDP, length 32
15:09:25.540147 IP 10.23.1.200.40572 > 10.23.1.100.33452: UDP, length 32
15:09:25.548161 IP 10.23.1.1 > 10.23.1.200: ICMP time exceeded in-transit, length 36
15:09:25.548208 IP 10.23.1.1 > 10.23.1.200: ICMP time exceeded in-transit, length 36
15:09:25.548233 IP 10.23.1.1 > 10.23.1.200: ICMP time exceeded in-transit, length 36
15:09:25.550430 IP 10.23.1.200.44090 > 10.23.1.100.33453: UDP, length 32
15:09:25.550444 IP 10.23.1.200.30073 > 10.23.1.100.33454: UDP, length 32
15:09:25.550453 IP 10.23.1.200.46325 > 10.23.1.100.33455: UDP, length 32
15:09:25.588888 IP 10.23.1.200 > 10.23.1.1: ICMP echo request, id 10458, seq 25012, length 8
15:09:25.598760 IP 10.23.1.1 > 10.23.1.200: ICMP echo reply, id 10458, seq 25012, length 8
15:09:26.130202 IP 10.23.1.200 > 10.23.1.1: ICMP echo request, id 10458, seq 25013, length 8
15:09:26.140134 IP 10.23.1.1 > 10.23.1.200: ICMP echo reply, id 10458, seq 25013, length 8
15:09:26.658825 IP 10.23.1.200 > 10.23.1.1: ICMP echo request, id 10458, seq 25014, length 8
15:09:26.668659 IP 10.23.1.1 > 10.23.1.200: ICMP echo reply, id 10458, seq 25014, length 8
15:09:27.180239 IP 10.23.1.200 > 10.23.1.1: ICMP echo request, id 10458, seq 25015, length 8
15:09:27.190104 IP 10.23.1.1 > 10.23.1.200: ICMP echo reply, id 10458, seq 25015, length 8
15:09:27.705401 IP 10.23.1.200 > 10.23.1.1: ICMP echo request, id 10458, seq 25016, length 8
15:09:27.715239 IP 10.23.1.1 > 10.23.1.200: ICMP echo reply, id 10458, seq 25016, length 8
15:09:28.220925 IP 10.23.1.200 > 10.23.1.1: ICMP echo request, id 10458, seq 25017, length 8
15:09:28.231023 IP 10.23.1.1 > 10.23.1.200: ICMP echo reply, id 10458, seq 25017, length 8
15:09:28.754789 IP 10.23.1.200 > 10.23.1.1: ICMP echo request, id 10458, seq 25018, length 8
15:09:28.764623 IP 10.23.1.1 > 10.23.1.200: ICMP echo reply, id 10458, seq 25018, length 8
15:09:29.273490 IP 10.23.1.200 > 10.23.1.1: ICMP echo request, id 10458, seq 25019, length 8

        The ip: 10.23.1.200 is the IP of the proxmox on the vpn network, so it look good.
        Like I said, if I do the same ping and traceroute directly via proxmox interface it working good.
        So the problem come from pfSense to send the data to proxmox server (10.1.1.2 to 10.1.1.1) it's like it's only going on one way.

        Doing a traceroute directly via pfSense interface (working one) result on this

        15:18:01.158901 IP 10.23.1.200 > 10.23.1.1: ICMP echo request, id 10458, seq 25985, length 8
15:18:01.168819 IP 10.23.1.1 > 10.23.1.200: ICMP echo reply, id 10458, seq 25985, length 8
15:18:01.670255 IP 10.23.1.200 > 10.23.1.1: ICMP echo request, id 10458, seq 25986, length 8
15:18:01.679957 IP 10.23.1.1 > 10.23.1.200: ICMP echo reply, id 10458, seq 25986, length 8
15:18:02.188947 IP 10.23.1.200 > 10.23.1.1: ICMP echo request, id 10458, seq 25987, length 8
15:18:02.198911 IP 10.23.1.1 > 10.23.1.200: ICMP echo reply, id 10458, seq 25987, length 8
15:18:02.631207 IP 10.23.1.200.47623 > 10.23.1.100.33435: UDP, length 12
15:18:02.641019 IP 10.23.1.1 > 10.23.1.200: ICMP time exceeded in-transit, length 36
15:18:02.641103 IP 10.23.1.200.47623 > 10.23.1.100.33436: UDP, length 12
15:18:02.651024 IP 10.23.1.1 > 10.23.1.200: ICMP time exceeded in-transit, length 36
15:18:02.651124 IP 10.23.1.200.47623 > 10.23.1.100.33437: UDP, length 12
15:18:02.661380 IP 10.23.1.1 > 10.23.1.200: ICMP time exceeded in-transit, length 36
15:18:02.661491 IP 10.23.1.200.47623 > 10.23.1.100.33438: UDP, length 12
15:18:02.684359 IP 10.23.1.100 > 10.23.1.200: ICMP 10.23.1.100 udp port 33438 unreachable, length 48
15:18:02.684430 IP 10.23.1.200.47623 > 10.23.1.100.33439: UDP, length 12
15:18:02.706763 IP 10.23.1.100 > 10.23.1.200: ICMP 10.23.1.100 udp port 33439 unreachable, length 48
15:18:02.706777 IP 10.23.1.200 > 10.23.1.1: ICMP echo request, id 10458, seq 25988, length 8
15:18:02.706821 IP 10.23.1.200.47623 > 10.23.1.100.33440: UDP, length 12
15:18:02.716579 IP 10.23.1.1 > 10.23.1.200: ICMP echo reply, id 10458, seq 25988, length 8
15:18:02.729263 IP 10.23.1.100 > 10.23.1.200: ICMP 10.23.1.100 udp port 33440 unreachable, length 48
15:18:03.239139 IP 10.23.1.200 > 10.23.1.1: ICMP echo request, id 10458, seq 25989, length 8
15:18:03.248924 IP 10.23.1.1 > 10.23.1.200: ICMP echo reply, id 10458, seq 25989, length 8
15:18:03.780349 IP 10.23.1.200 > 10.23.1.1: ICMP echo request, id 10458, seq 25990, length 8
15:18:03.790087 IP 10.23.1.1 > 10.23.1.200: ICMP echo reply, id 10458, seq 25990, length 8
15:18:04.321727 IP 10.23.1.200 > 10.23.1.1: ICMP echo request, id 10458, seq 25991, length 8
15:18:04.331676 IP 10.23.1.1 > 10.23.1.200: ICMP echo reply, id 10458, seq 25991, length 8
15:18:04.858850 IP 10.23.1.200 > 10.23.1.1: ICMP echo request, id 10458, seq 25992, length 8
15:18:04.868774 IP 10.23.1.1 > 10.23.1.200: ICMP echo reply, id 10458, seq 25992, length 8
        stephenw10S 1 Reply Last reply Reply Quote 0
        • stephenw10S
          stephenw10 Netgate Administrator
          last edited by stephenw10

          If Proxmox has an IP in the 10.23.1.0/24 tunnel subnet when you ping anything in the tunnel subnet from it I suspect you are seeing an icmp redirect at some point so the traffic in at least one direction goes directly rather than via pfSense.
          You have a high probability of asymmetric routing there.

          Do you see blocked traffic in the firewall log when it fails?

          Steve

          1 Reply Last reply Reply Quote 0
          • D
            DasK
            last edited by DasK

            Thanks for reply.

            No there is anything blocked, I allow everything into the firewall rules.
            Proxmox don't have an IP in the 10.23.1.0/24 tunnel.

            I just added a rule to tell proxmox to communicate to the 10.23.1.0/24 tunnel via 10.1.1.2(pfSense)

            ip route add 10.23.1.0/24 via 10.1.1.2

            So my proxmox server know where to go to communicate with 10.23.1.0/24 network.
            The thing I don't understand is why ping is working, but not traceroute or rpcinfo, if I do a traceroute -I it work.
            But well, in final I need this rpc working to etablish a NFS storage.

            1 Reply Last reply Reply Quote 0
            • stephenw10S
              stephenw10 Netgate Administrator @DasK
              last edited by

              @DasK said in Route WAN network to OVPN:

              The ip: 10.23.1.200 is the IP of the proxmox on the vpn network

              Then what did you mean by that? A different Proxmox server?

              Steve

              1 Reply Last reply Reply Quote 0
              • D
                DasK
                last edited by DasK

                Ok I'm sorry, I was meaning the IP of the pfSense, my bad

                Proxmox IP: 10.1.1.1
                PfSense WAN IP: 10.1.1.2
                PfSense LAN IP: 192.168.10.254
                PfSense VPN IP: 10.23.1.200
                VPN NETWORK: 10.23.1.0/24

                And the server I try to communicate with on the VPN is : 10.23.1.100

                Outbound rules (without this, proxmox can't ping 10.23.1.100 anymore)
                f7e68811-9f28-4dbb-80bb-d4cea4836e97-image.png

                Interfaces
                68580d68-37a7-4db9-b423-6d406710b9aa-image.png

                Wan rules
                9e837633-8910-412b-b1f1-40908c79675d-image.png

                Lan rules
                a846fc18-042b-4962-bb21-b6a28b9afbf7-image.png

                OPT1 rules
                921c8484-5d24-4bf6-8bf4-e8064d2f74d2-image.png

                Routes
                d2078683-c847-47d7-a22f-c8e1fc198c64-image.png

                pfSense traceroute:
                alt text

                pfSense rpcinfo:
                alt text

                Proxmox traceroute:
                ce7a828b-f205-4f1b-8acf-2ebc68e60065-image.png

                Proxmox rpcinfo:
                962e0997-2189-4e34-afb7-4d3a6efd504e-image.png

                Proxmox ping:
                29622625-b522-4db6-8ce6-d795089610e4-image.png

                1 Reply Last reply Reply Quote 0
                • stephenw10S
                  stephenw10 Netgate Administrator
                  last edited by

                  Hmm, you might have to check the OpenVPN server, it's able to reach that at 10.23.1.1 but no further. Something blocking or re-routing it there maybe?

                  Hard to imagine what the difference might be there though. It could only really be TTL....

                  You probably need that outbound NAT rule as those target devices otherwise have no route back.

                  Steve

                  1 Reply Last reply Reply Quote 0
                  • D
                    DasK
                    last edited by

                    All if good for the OpenVPN server.

                    Any example of how "outbound NAT rule as those target devices " ?

                    Best

                    1 Reply Last reply Reply Quote 0
                    • stephenw10S
                      stephenw10 Netgate Administrator
                      last edited by

                      The rule you already have for source NATing 10.1.1.0/30 to 10.23.1.200.

                      Devices at 10.23.1.100 (and .1) obviously know how to reach 10.23.1.200 but probably have no route to 10.1.1.0/30. That is the reason you need that rule.

                      Steve

                      1 Reply Last reply Reply Quote 0
                      • D
                        DasK
                        last edited by

                        I added the rule, but still the same problem

                        1 Reply Last reply Reply Quote 0
                        • D
                          DasK
                          last edited by

                          Well I think I'll give up and just also connect the proxmox server to the VPN, so there will be a direct connection to both server.. It just look weird to connect proxmox to VPN, when there is a VM pfSense already connected to this VPN

                          1 Reply Last reply Reply Quote 0
                          • stephenw10S
                            stephenw10 Netgate Administrator
                            last edited by

                            We haven't yet seen a packet capture of a TCP connection failing. That should be revealing. If there is no traffic coming back at all for example.

                            This still looks like some route asymmetry somewhere.

                            Does an icmp traceroute succeed?

                            Steve

                            1 Reply Last reply Reply Quote 0
                            • D
                              DasK
                              last edited by

                              Yes all ICMP traceroute work, like ping.

                              1 Reply Last reply Reply Quote 0
                              • stephenw10S
                                stephenw10 Netgate Administrator
                                last edited by

                                When you have ICMP works but TCP fails it's usually an asymmetric routing problem.

                                It can also be a packet size issue but that would not normally affect UDP traceroute.

                                Or it can be a hardware off loading problem if the a NIC/driver somewhere is not doing what it reports it can.

                                Steve

                                1 Reply Last reply Reply Quote 0
                                • D
                                  DasK
                                  last edited by

                                  How does I can solve this asymetric routing problem ? Or detect where is located the routing problem.

                                  1 Reply Last reply Reply Quote 0
                                  • stephenw10S
                                    stephenw10 Netgate Administrator
                                    last edited by

                                    Run a packet capture showing some TCP connection failing. If you see parts of the TCP setup missing they are either being blocked somewhere or routed some other way.

                                    Steve

                                    1 Reply Last reply Reply Quote 0
                                    • D
                                      DasK
                                      last edited by DasK

                                      There is a little diagram, if that can help to understand how is setup

                                      db312123-0f2a-4488-b534-0d7b9062e3c5-image.png

                                      1 Reply Last reply Reply Quote 0
                                      • stephenw10S
                                        stephenw10 Netgate Administrator
                                        last edited by

                                        Yeah, there is some difference between what happens when you run, say, rpcinfo from pfSense directly and when it's run from Proxmox. So I would run a pcap on the OpenVPN interface and compare them.

                                        rpcinfo will create quite a lot of output so you might want to use something simpler like just telneting to a port assuming that also fails from Proxmox.

                                        Steve

                                        1 Reply Last reply Reply Quote 0
                                        • First post
                                          Last post
                                        Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.