Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    VEs WANs can't reach each other

    Routing and Multi WAN
    2
    8
    288
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • I
      ites
      last edited by

      Dear Netgate-Community,

      I have the following problem:
      The VM’s behind one VE can’t ping VMs behind the other VE.

      The public IP net is 1.2.3.16/29.
      Thera are two hardware servers each one running a ProxmoxVE.
      The Gateway is PFsense2 on 1.2.3.17 (LAN IP / Gateway).
      PFsense1 on VE1 is on LAN interface 192.168.79.1 and public WAN interface 1.2.3.18 and
      PFsense3 on VE2 is on LAN interface 192.168.78.1 and public WAN interface 1.2.3.19. (I know, the names of the firewalls are a bit confusing).

      An example of the problem:
      VM1 on VE1 can’t reach VM2 on VE2 over 1.2.3.22, other direction - same problem, even the opposite WAN addresses are not reachable on WAN management interface. I know this should not be enabled for security reasons. Doesn’t matter here.
      But reaching the gateway 1.2.3.17 from both sides is possible.

      The problem is clear, because the IP’s are in the same net they don’t even try to leave the WAN. To solve I tried to link the two WAN interfaces to force him to leave the WAN searching IP’s of the same net. I have tried many different solutions, but I finally couldn’t handle it.

      Has anybody an idea how to solve that?

      V 1 Reply Last reply Reply Quote 0
      • V
        viragomann @ites
        last edited by

        @ites
        Can you post a network diagram, which shows, how the devices are connected?

        VM1 on VE1 can’t reach VM2 on VE2 over 1.2.3.22

        So both VMs are connected to the WAN network directly without the virtualized pfSense in between?

        I 1 Reply Last reply Reply Quote 0
        • I
          ites @viragomann
          last edited by ites

          Hi @viragomann

          Thanks for your reply.
          I made this chart an hope it makes everyone understand my setup better.
          network_firewall_pf1_pf_pf3.png

          All pfsense firewalls are virtualized.
          pf1 and pf3 have real hardware NICs for WAN and also software NICs on LAN side (Proxmox vSwitch).

          pf2 is also virtualized on pve2 and has has hardware NIC for WAN. The LAN interface is a bridge with a hardware and software NIC. PVE2 uses the software NIC in the bridge. PF1 is connected with the hardware NIC in the bridge.

          V 1 Reply Last reply Reply Quote 0
          • V
            viragomann @ites
            last edited by

            @ites said in VEs WANs can't reach each other:

            All pfsense firewalls are virtualized.

            So I pressume, that your drawing is showing the logical network setup.

            pf1 and pf3 have real hardware NICs for WAN and also software NICs on LAN side (Proxmox vSwitch).

            pf2 is also virtualized on pve2 and has has hardware NIC for WAN. The LAN interface is a bridge with a hardware and software NIC. PVE2 uses the software NIC in the bridge. PF1 is connected with the hardware NIC in the bridge.

            And PVE2 has at least three hardware NICs and pf1 has all interfaces bridged.

            If so on pf1 + 3 you just have to add the respective other pfSense WAN address as gateway and add a static route for the network behind the other using this gateway.

            VM1 on VE1 can’t reach VM2 on VE2 over 1.2.3.22

            Not clear, what this IP is, however.

            I 1 Reply Last reply Reply Quote 0
            • I
              ites @viragomann
              last edited by ites

              Hi @viragomann

              Thanks for for your answer.
              I will check later an let you know if this works.

              You are right 1.2.3.22 is a second virtual IP on the WAN interface of pf3. I forgot it in my network plan. It also doesn't work on 1.2.3.19.

              1 Reply Last reply Reply Quote 0
              • I
                ites
                last edited by

                Hi again @viragomann
                I didn't find the time to write this post yesterday and to try around with your suggestions.
                Today I tried it, but can't get it going. Maybe I don't get you right.
                Please explain how the static routes would look like.
                I want to connect to the gateway itself.

                I can't set a route for 1.2.3.16/29, as this gets in conflict with the WAN. Thats clear for sure.
                Bildschirmfoto 2024-06-06 um 21.17.04.png

                So I've set two additional gateways on pf1 and also those two static routes.
                gateways.png
                routes.png

                On pf3, no pictures here, I've also set a second gateway 1.2.3.18/29 and those static routes
                Network 192.168.79.0/24 ; Gateway 1.2.3.18
                Network 1.2.3.18/32 ; Gatewy 1.2.3.18

                With this configuration I don't get a connection.
                Any help appreciated. Thank you!

                V 1 Reply Last reply Reply Quote 0
                • V
                  viragomann @ites
                  last edited by

                  @ites
                  You don't need static routes for the WAN subnet, but only for networks behind the respective other pfSense.

                  So on pf1

                  • add a gateway on WAN interface with the IP 1.2.3.19, name pf3
                  • add a static route for the destination network 192.168.78.0/24 and select pf3 as gateway.

                  So on pf3

                  • add a gateway on WAN interface with the IP 1.2.3.18, name pf1
                  • add a static route for the destination network 192.168.79.0/24 and select pf1 as gateway.
                  I 1 Reply Last reply Reply Quote 0
                  • I
                    ites @viragomann
                    last edited by

                    Hi @viragomann

                    Thank you for your answer. This is what thougt to do at first. For whatever reason it doesn't work for me here. I ordered another /29 subnet and will check if it works with it. It didn't arrive yet.

                    1 Reply Last reply Reply Quote 0
                    • First post
                      Last post
                    Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.