Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    my openvpn site to site i cant seem to ping or access other site doesnt stay stable

    Scheduled Pinned Locked Moved General pfSense Questions
    22 Posts 2 Posters 572 Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • C
      comet424 @stephenw10
      last edited by

      @stephenw10 sorry delay ive had power out big storm saturday in canada and i just got my power back

      what is OBN? and i can switch i found an issue that is not working for me. and will this fix it with the hybrid?

      so on my unraid box i have a 10Gig nick and broken into some bridges

      br0 = LAN 192.168.0.0
      br0.10 = Vlan Cameras 192.168.10.0
      br0.20 = Vlan IOT 192.168.20.0

      now i cant seem to ping the 192.168.1.0 network
      i noticed the issue as i cant access the remote Home Assistant from my main home assistant

      on my home assistant i have
      enp1s0 192.168.0.12 gateway 192.168.0.1 dns 192.168.0.1
      enp2s0 192.168.20.12 gateway 192.168.20.1 dns 192.168.20.1
      enp3s0 192.168.10.12 gateway 192.168.10.1 dns 192.168.10.1

      now i try to ping 192.168.1.12 remote HA or 192.168.1.1 other pfsense
      and it cant
      even though in the rules i say IOT can have access to 192.168.1.12 same for camera

      i tried
      ping -I enp1s0 192.168.1.1
      ping -I enp1s0 192.168.1.12
      those work

      but if i try
      ping -I enp2s0 192.168.1.1
      ping -I enp2s0 192.168.1.1
      ping -I enp3s0 192.168.1.1
      ping -I enp3s0 192.168.1.1
      it doesnt work

      when i try the pinging in unraid terminal with the
      ping -I br0.10 192.168.1.1 or 192.168.10.1 it doesnt wanna work is there a rule or is that NAT outboard that is blocking it?

      reason i noticed Home assistant uses 1 ethernet bridge as the main one and i noticed sometimes it changes but id trying to ping from any of those bridges to go out the OPENVPN connection or the network
      is something conflicting ? i know when i just have 1 LAN network not issues but when i switched to having vlans to break up my network some i come across it..

      maybe complicated? maybe i just doing something wrong.. and if you need any pics let me know as i wanna be able to ping the 192.168.1.12 from the termainal and it should go out any of the vlan lan ports or be able to pick it off

      here is the IOT and Camera Rules as the LAN one pings ... maybe its outside of the scope or maybe i need a better video to watch to setup?
      and sorry if i confusing my dislyexia gets best of me sounds fine to me but i may not explain it right

      pf1.png pf2.png

      1 Reply Last reply Reply Quote 0
      • stephenw10S
        stephenw10 Netgate Administrator
        last edited by

        @comet424 said in my openvpn site to site i cant seem to ping or access other site doesnt stay stable:

        what is OBN?

        OutBound Nat.

        @comet424 said in my openvpn site to site i cant seem to ping or access other site doesnt stay stable:

        when i try the pinging in unraid terminal with the
        ping -I br0.10 192.168.1.1 or 192.168.10.1 it doesnt wanna work

        192.168.10.1 is in he br0.10 subnet so that would go directly if it's actually using the correct source IP. In which case pfSense never sees it and it must be something in the virtual infrastructure or the taret host blocking it.

        When you have multi-homed hosts like that it's common to see asymmetric routing issues. Check the firewall logs for blocked traffic.

        C 1 Reply Last reply Reply Quote 0
        • C
          comet424 @stephenw10
          last edited by comet424

          @stephenw10
          so i did screen capture the camera logs.. where you see its pinging from 192.168.10.12 which is the home assistant ip for Cameras.. and trying to reach 192.168.1.12

          firewall 1.png

          so
          192.168.0.1 LAN pfsense ip
          192.168.10.1 pfsense Camera VLAN
          192.168.20.1 pfsense IOT VLAN

          unraid
          192.168.0.3 BR0 LAN network
          192.168.10.3 BR0.10 Camera VLAN
          192.168.20.3 BR0.20 IOT Vlan

          home assistant
          192.168.0.12 LAN network
          192.168.10.12 Camera VLAN
          192.168.20.12 IOT Vlan

          now i found it would work if it static ip them
          but set it like this
          192.168.0.12 Gateway 192.168.0.1 DNS 192.168.0.1
          192.168.10.12 Gateway 192.168.0.1 DNS 192.168.0.1
          192.168.20.12 Gateway 192.168.0.1 DNS 192.168.0.1

          that made it work but it doesnt work if you specify it in the DHCP Server for doing the dhcp part it doesnt like you setting the gateway 192.168.0.1 for the camera or IOT says its out of the range

          but it made HA at least work..
          so how do you fix it to work properly

          and the

          192.168.0.1 LAN pfsense ip
          192.168.10.1 pfsense Camera VLAN
          192.168.20.1 pfsense IOT VLAN

          they are all from the single pfsense box so its not seperate machines just the one

          C stephenw10S 2 Replies Last reply Reply Quote 0
          • C
            comet424 @comet424
            last edited by

            oh and i also have a
            192.168.30.x vlan i made a management port from ym onboard network cards

            i found i had to disable the gateway on the desktop and servers.. i found that it wasnt using the 10gig network card 192.168.0.1 but was going through the 192.168.30.x and id have no internet
            even though i blocked it in the rules and left it 192.168.30.1 as a gateway it always tried to go through it and not the 10gig so i ended up removing the gateway in windows adapter for the onboard.. that was frustrating to figure out

            1 Reply Last reply Reply Quote 0
            • stephenw10S
              stephenw10 Netgate Administrator @comet424
              last edited by

              @comet424 said in my openvpn site to site i cant seem to ping or access other site doesnt stay stable:

              now i found it would work if it static ip them
              but set it like this
              192.168.0.12 Gateway 192.168.0.1 DNS 192.168.0.1
              192.168.10.12 Gateway 192.168.0.1 DNS 192.168.0.1
              192.168.20.12 Gateway 192.168.0.1 DNS 192.168.0.1

              Mmm, see that's the problem with multi-homed hosts. When you do that it sends all traffic via the LAN gateway on the LAN subnet so bypasses the other firewall rules entirely.

              So I assume the 192.168.1.0/24 subnet is at the other end of the tunnel?

              There are no hits in the firewall logs for 192.168.1.12. So if you're trying to ping that it's probably passing. You have specific rules to allow that and one of them has bytes recorded on it.

              Check the states for that IP when you're pinging.

              C 1 Reply Last reply Reply Quote 0
              • C
                comet424 @stephenw10
                last edited by comet424

                @stephenw10
                so is there an easy way to do mult-home? so like i dont seem to have issues when its 1 nic to a device

                just when i have 3 network lans going into the home assistant or the unraid
                as i have to ping -I <network interfrace>

                how come mult home is hard to do? that its all going to the same pfsense box etc?

                and ya on the openvpn tunnel the other side is the 192.168.1.0/24 and the remote home assistant is 192.168.1.12
                and with HA when you reboot sometimes it changes which is the main network card and it has option to only use 1 for multicast.. but its like it rotates it sometimes 192.168.0.0 or 192.168.20.0 or 192.168.10.0 and it can never find ips on the network of the other 2 lans unless i choose the 3
                here is it screen shot of HA the 3 networks.. the default will rotate from time to time from reboots.. so i want to be able to whichever its on when you just type ping 192.168.1.12 to test instead you gotta type ping -I enp1s0 192.168.1.12 as example cuz connecting doesnt work unless it can connect...
                ha.png

                i guess the Hybrid NAT cant fix this issue?
                so i rebooted the pfsense and re reset the HA nics to
                192.168.0.12 Gateway 192.168.0.1 DNS 192.168.0.1
                192.168.10.12 Gateway 192.168.10.1 DNS 192.168.10.1
                192.168.20.12 Gateway 192.168.20.1 DNS 192.168.20.1

                and back to cant ping or connect to the tunnel..

                as for checking the states of the ip of 192.168.1.12 not sure i did click on the the bytes and came up the states page.. i did type in 192.168.1.12
                but its showing nothing even when i doing pinging i probably doing that wrong

                pf3.png

                i guess its not a simple solution?

                i did google when you mentioned "multi-home" to try to learn if others done it but i finding 0 hits really 1.. but the person solved it and didnt show how he did it and others is about multiwan but ill also keep trying to look up more on multi-home

                i guess the multi home issue i have for the desktop and my servers too right
                where i have
                10gig nic 192.168.0.0
                onboard nic i use as Management port 192.168.30.0
                and only way i got it to work was block WAN address and subnets for the 192.168.30.0 and i had to remove gateway ip address from the server and desktop remove the 192.168.30.1 cuz thats where my 10g copying was going through the 1gb management ... guess its a learning curve but once you know how to do it its simple?

                1 Reply Last reply Reply Quote 0
                • stephenw10S
                  stephenw10 Netgate Administrator
                  last edited by

                  Don't filter by ruleID just look for all states with that IP address. You must be actively pinging at the time because those states expire quickly.

                  The only safe way to use multi-home is don't use multi-home! 😉

                  The problem is that other hosts from different VLANs may try to connect to it on an IP that must be routed through pfSense. But the HA host has direct access to all 3 subnets so it doesn't need to send replies back to pfSense to be routed, it just sends them dircetly. But doing so create an asymmetric route where the firewall only sees half of the conversation and because of that it blocks the unexpected traffic.

                  However, whilst that's bad, it should not have any affect on the ability to ping across the VPN so I would look at that for now.

                  C 1 Reply Last reply Reply Quote 0
                  • C
                    comet424 @stephenw10
                    last edited by comet424

                    @stephenw10
                    ok so i tried the pinging and i got these screen shots
                    ping1.png ping2.png ping3.png

                    so when you say HA creates asymmetric route
                    like for cameras or IOT devices it doesnt do

                    HA --> PFsense --> IOT Device 192.168.20.x

                    instead what its doing is

                    HA --> IOT Device

                    skipping PF Sense directly

                    but if i pinging 192.168.1.12 its neither on any of those 3 subnet interfaces so shouldnt it try to directly go out the pfsense?

                    and each time i do a reboot of HA a different interface becomes i guess the MASTER so on a reboot if 192.168.0.12 gets the default (master) it can ping 192.168.1.1 or 192.168.1.12
                    but when a reboot and if Cameras or IOT Interface gets the master i cant ping them..

                    so i dunno if this is doing anything i tried to make a NAT for cameras and IOT to use the openvpn connection now its probably totally wrong but i trying...

                    nat1.png

                    so if the idea is not to use multi home why do it? and isnt multi home same as like VPN_FAil over? so i have vpn fail over group i have 2 PIAs set to Teir 1 or its 1 and then 2 i forget.. and if one fails it goes out the other direction... but in esstance i doing the reverse using the 2 vpn fail overs and going out the 1.. using vpn fail over as like a camera and iot vlan just as example of trying to understand things

                    oh and if multihome is bad right how do you then use 1 interface.. in HA but push the 3 networks LAN, IOT,CAMERAS from pfsense split there forced into 1 interface and HA splits it back into 3 sections basiclly like a Hour glass
                    3 interfaces on PFSEnese goes to 1 interface on HA and HA re splits that 1 interface into 3 seperate networks

                    C stephenw10S 2 Replies Last reply Reply Quote 0
                    • C
                      comet424 @comet424
                      last edited by comet424

                      so with the HA defaulting to Camera network i ran the pinging and these were the results
                      and the NAT settings i have for Openvpn for iot and cameras did nothing so that didnt help my issue it was a try.. or is it working cuz its established but i cant ping

                      ha2.png
                      ping4.png

                      C 1 Reply Last reply Reply Quote 0
                      • C
                        comet424 @comet424
                        last edited by

                        so it seems ya that i needed the NAT i tested for Cameras network. it is the Master right now..

                        here is Before that NAT Camera
                        before.png

                        and after i enabled the NAT the openvpn to Camera source and Camera Address
                        then it started working. well not the pinging but the connection to the other Home Assistant

                        after.png

                        and what does the NAT Address do.. and do you always keep it the same as the Source
                        so Camera subnets Camera Address
                        IOT Subnets IOT Address?

                        and i having troubles posting this thing saying i got spam in it wish they would say what is spam by the akismet.com

                        C 1 Reply Last reply Reply Quote 0
                        • C
                          comet424 @comet424
                          last edited by

                          and for the IOT when its the master

                          Before
                          before 2.png

                          and after i enable the NAT openvpn iot subnet iot address
                          after 2.png

                          what i find weird is i dont have camera nat enable yet its established... but when its master i have to enable the NAT as it wont access untill i enable that NAT route.. its weird.. but that seems to work.. and you will say that shouldnt work like that im sure

                          1 Reply Last reply Reply Quote 0
                          • stephenw10S
                            stephenw10 Netgate Administrator @comet424
                            last edited by

                            @comet424 said in my openvpn site to site i cant seem to ping or access other site doesnt stay stable:

                            so when you say HA creates asymmetric route
                            like for cameras or IOT devices it doesnt do

                            HA --> PFsense --> IOT Device 192.168.20.x

                            instead what its doing is

                            HA --> IOT Device

                            skipping PF Sense directly

                            but if i pinging 192.168.1.12 its neither on any of those 3 subnet interfaces so shouldnt it try to directly go out the pfsense?

                            Yes exactly. You cannot have asymmetry to the remote subnet as the traffic must pass the firewall both ways.

                            But, yes, it could still try to send from any of those interfaces. However there should be a way to set one of those as priority in some way so it always uses 192.168.0.12 as the source for routed connections.

                            That last view showing it using the cameras subnet as source is still filtered by rule 137 so it's only showing states opened by that. You need to remove that so you see the outbound state it's using If it;s correctly opening a state on the openvpn interface then it's almost certainly blocked at the other end.

                            C 1 Reply Last reply Reply Quote 0
                            • C
                              comet424 @stephenw10
                              last edited by

                              @stephenw10
                              ah ok sorry about that it always defaults to 137 whatever that means... here is a new screen shot..
                              after 3.png

                              ill have to see if i can help from the Unraid group... there is boot order for the VM i guess so you can load the nics in order. as right now it just randomly picks whatever first.. it gets frustrating i tell ya..

                              i know on my host openvpn of the site to site 192.168.1.1 i added the openvpn connection for remote local i did
                              192.168.0.0/24,192.168.10.0/24,192.168.20.0/24

                              but here is a pic from the 192.168.1.1 pfsense the states
                              host.png host2.png host3.png

                              i guess sometimes its almost a guessing game? like the openvpn NAT i dont need one for LAN connection but i needed to add one for CAMERAS and IOT to get it to connect..

                              C stephenw10S 2 Replies Last reply Reply Quote 0
                              • C
                                comet424 @comet424
                                last edited by

                                and you mentioned to try to get HA to be master for the 192.168.0.12 is the reason it works all the time as its the physical LAN and the Camera and IOT are just VLANs on why it works differently ?

                                1 Reply Last reply Reply Quote 0
                                • stephenw10S
                                  stephenw10 Netgate Administrator @comet424
                                  last edited by

                                  @comet424 said in my openvpn site to site i cant seem to ping or access other site doesnt stay stable:

                                  it always defaults to 137

                                  It does that if you click on the state count on the rule to reach the states screen. But if you just go to Diag > States from the menu it should not have anything. If it does I'd check you don't have some auto-fill enabled that's adding it in the browser.

                                  But I don't see any pings in those tables. What we want to see is the pings states created by a failing ping. So start a continuous (or very long!) ping sources from the camera interface address then check the state table at each pfSense to see what it's doing.

                                  1 Reply Last reply Reply Quote 0
                                  • First post
                                    Last post
                                  Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.