my openvpn site to site i cant seem to ping or access other site doesnt stay stable
-
@stephenw10 sorry delay ive had power out big storm saturday in canada and i just got my power back
what is OBN? and i can switch i found an issue that is not working for me. and will this fix it with the hybrid?
so on my unraid box i have a 10Gig nick and broken into some bridges
br0 = LAN 192.168.0.0
br0.10 = Vlan Cameras 192.168.10.0
br0.20 = Vlan IOT 192.168.20.0now i cant seem to ping the 192.168.1.0 network
i noticed the issue as i cant access the remote Home Assistant from my main home assistanton my home assistant i have
enp1s0 192.168.0.12 gateway 192.168.0.1 dns 192.168.0.1
enp2s0 192.168.20.12 gateway 192.168.20.1 dns 192.168.20.1
enp3s0 192.168.10.12 gateway 192.168.10.1 dns 192.168.10.1now i try to ping 192.168.1.12 remote HA or 192.168.1.1 other pfsense
and it cant
even though in the rules i say IOT can have access to 192.168.1.12 same for camerai tried
ping -I enp1s0 192.168.1.1
ping -I enp1s0 192.168.1.12
those workbut if i try
ping -I enp2s0 192.168.1.1
ping -I enp2s0 192.168.1.1
ping -I enp3s0 192.168.1.1
ping -I enp3s0 192.168.1.1
it doesnt workwhen i try the pinging in unraid terminal with the
ping -I br0.10 192.168.1.1 or 192.168.10.1 it doesnt wanna work is there a rule or is that NAT outboard that is blocking it?reason i noticed Home assistant uses 1 ethernet bridge as the main one and i noticed sometimes it changes but id trying to ping from any of those bridges to go out the OPENVPN connection or the network
is something conflicting ? i know when i just have 1 LAN network not issues but when i switched to having vlans to break up my network some i come across it..maybe complicated? maybe i just doing something wrong.. and if you need any pics let me know as i wanna be able to ping the 192.168.1.12 from the termainal and it should go out any of the vlan lan ports or be able to pick it off
here is the IOT and Camera Rules as the LAN one pings ... maybe its outside of the scope or maybe i need a better video to watch to setup?
and sorry if i confusing my dislyexia gets best of me sounds fine to me but i may not explain it right
-
@comet424 said in my openvpn site to site i cant seem to ping or access other site doesnt stay stable:
what is OBN?
OutBound Nat.
@comet424 said in my openvpn site to site i cant seem to ping or access other site doesnt stay stable:
when i try the pinging in unraid terminal with the
ping -I br0.10 192.168.1.1 or 192.168.10.1 it doesnt wanna work192.168.10.1 is in he br0.10 subnet so that would go directly if it's actually using the correct source IP. In which case pfSense never sees it and it must be something in the virtual infrastructure or the taret host blocking it.
When you have multi-homed hosts like that it's common to see asymmetric routing issues. Check the firewall logs for blocked traffic.
-
@stephenw10
so i did screen capture the camera logs.. where you see its pinging from 192.168.10.12 which is the home assistant ip for Cameras.. and trying to reach 192.168.1.12
so
192.168.0.1 LAN pfsense ip
192.168.10.1 pfsense Camera VLAN
192.168.20.1 pfsense IOT VLANunraid
192.168.0.3 BR0 LAN network
192.168.10.3 BR0.10 Camera VLAN
192.168.20.3 BR0.20 IOT Vlanhome assistant
192.168.0.12 LAN network
192.168.10.12 Camera VLAN
192.168.20.12 IOT Vlannow i found it would work if it static ip them
but set it like this
192.168.0.12 Gateway 192.168.0.1 DNS 192.168.0.1
192.168.10.12 Gateway 192.168.0.1 DNS 192.168.0.1
192.168.20.12 Gateway 192.168.0.1 DNS 192.168.0.1that made it work but it doesnt work if you specify it in the DHCP Server for doing the dhcp part it doesnt like you setting the gateway 192.168.0.1 for the camera or IOT says its out of the range
but it made HA at least work..
so how do you fix it to work properlyand the
192.168.0.1 LAN pfsense ip
192.168.10.1 pfsense Camera VLAN
192.168.20.1 pfsense IOT VLANthey are all from the single pfsense box so its not seperate machines just the one
-
oh and i also have a
192.168.30.x vlan i made a management port from ym onboard network cardsi found i had to disable the gateway on the desktop and servers.. i found that it wasnt using the 10gig network card 192.168.0.1 but was going through the 192.168.30.x and id have no internet
even though i blocked it in the rules and left it 192.168.30.1 as a gateway it always tried to go through it and not the 10gig so i ended up removing the gateway in windows adapter for the onboard.. that was frustrating to figure out -
@comet424 said in my openvpn site to site i cant seem to ping or access other site doesnt stay stable:
now i found it would work if it static ip them
but set it like this
192.168.0.12 Gateway 192.168.0.1 DNS 192.168.0.1
192.168.10.12 Gateway 192.168.0.1 DNS 192.168.0.1
192.168.20.12 Gateway 192.168.0.1 DNS 192.168.0.1Mmm, see that's the problem with multi-homed hosts. When you do that it sends all traffic via the LAN gateway on the LAN subnet so bypasses the other firewall rules entirely.
So I assume the 192.168.1.0/24 subnet is at the other end of the tunnel?
There are no hits in the firewall logs for 192.168.1.12. So if you're trying to ping that it's probably passing. You have specific rules to allow that and one of them has bytes recorded on it.
Check the states for that IP when you're pinging.
-
@stephenw10
so is there an easy way to do mult-home? so like i dont seem to have issues when its 1 nic to a devicejust when i have 3 network lans going into the home assistant or the unraid
as i have to ping -I <network interfrace>how come mult home is hard to do? that its all going to the same pfsense box etc?
and ya on the openvpn tunnel the other side is the 192.168.1.0/24 and the remote home assistant is 192.168.1.12
and with HA when you reboot sometimes it changes which is the main network card and it has option to only use 1 for multicast.. but its like it rotates it sometimes 192.168.0.0 or 192.168.20.0 or 192.168.10.0 and it can never find ips on the network of the other 2 lans unless i choose the 3
here is it screen shot of HA the 3 networks.. the default will rotate from time to time from reboots.. so i want to be able to whichever its on when you just type ping 192.168.1.12 to test instead you gotta type ping -I enp1s0 192.168.1.12 as example cuz connecting doesnt work unless it can connect...
i guess the Hybrid NAT cant fix this issue?
so i rebooted the pfsense and re reset the HA nics to
192.168.0.12 Gateway 192.168.0.1 DNS 192.168.0.1
192.168.10.12 Gateway 192.168.10.1 DNS 192.168.10.1
192.168.20.12 Gateway 192.168.20.1 DNS 192.168.20.1and back to cant ping or connect to the tunnel..
as for checking the states of the ip of 192.168.1.12 not sure i did click on the the bytes and came up the states page.. i did type in 192.168.1.12
but its showing nothing even when i doing pinging i probably doing that wrong
i guess its not a simple solution?
i did google when you mentioned "multi-home" to try to learn if others done it but i finding 0 hits really 1.. but the person solved it and didnt show how he did it and others is about multiwan but ill also keep trying to look up more on multi-home
i guess the multi home issue i have for the desktop and my servers too right
where i have
10gig nic 192.168.0.0
onboard nic i use as Management port 192.168.30.0
and only way i got it to work was block WAN address and subnets for the 192.168.30.0 and i had to remove gateway ip address from the server and desktop remove the 192.168.30.1 cuz thats where my 10g copying was going through the 1gb management ... guess its a learning curve but once you know how to do it its simple? -
Don't filter by ruleID just look for all states with that IP address. You must be actively pinging at the time because those states expire quickly.
The only safe way to use multi-home is don't use multi-home!
The problem is that other hosts from different VLANs may try to connect to it on an IP that must be routed through pfSense. But the HA host has direct access to all 3 subnets so it doesn't need to send replies back to pfSense to be routed, it just sends them dircetly. But doing so create an asymmetric route where the firewall only sees half of the conversation and because of that it blocks the unexpected traffic.
However, whilst that's bad, it should not have any affect on the ability to ping across the VPN so I would look at that for now.
-
@stephenw10
ok so i tried the pinging and i got these screen shots
so when you say HA creates asymmetric route
like for cameras or IOT devices it doesnt doHA --> PFsense --> IOT Device 192.168.20.x
instead what its doing is
HA --> IOT Device
skipping PF Sense directly
but if i pinging 192.168.1.12 its neither on any of those 3 subnet interfaces so shouldnt it try to directly go out the pfsense?
and each time i do a reboot of HA a different interface becomes i guess the MASTER so on a reboot if 192.168.0.12 gets the default (master) it can ping 192.168.1.1 or 192.168.1.12
but when a reboot and if Cameras or IOT Interface gets the master i cant ping them..so i dunno if this is doing anything i tried to make a NAT for cameras and IOT to use the openvpn connection now its probably totally wrong but i trying...
so if the idea is not to use multi home why do it? and isnt multi home same as like VPN_FAil over? so i have vpn fail over group i have 2 PIAs set to Teir 1 or its 1 and then 2 i forget.. and if one fails it goes out the other direction... but in esstance i doing the reverse using the 2 vpn fail overs and going out the 1.. using vpn fail over as like a camera and iot vlan just as example of trying to understand things
oh and if multihome is bad right how do you then use 1 interface.. in HA but push the 3 networks LAN, IOT,CAMERAS from pfsense split there forced into 1 interface and HA splits it back into 3 sections basiclly like a Hour glass
3 interfaces on PFSEnese goes to 1 interface on HA and HA re splits that 1 interface into 3 seperate networks -
so with the HA defaulting to Camera network i ran the pinging and these were the results
and the NAT settings i have for Openvpn for iot and cameras did nothing so that didnt help my issue it was a try.. or is it working cuz its established but i cant ping
-
so it seems ya that i needed the NAT i tested for Cameras network. it is the Master right now..
here is Before that NAT Camera
and after i enabled the NAT the openvpn to Camera source and Camera Address
then it started working. well not the pinging but the connection to the other Home Assistant
and what does the NAT Address do.. and do you always keep it the same as the Source
so Camera subnets Camera Address
IOT Subnets IOT Address?and i having troubles posting this thing saying i got spam in it wish they would say what is spam by the akismet.com
-
and for the IOT when its the master
Before
and after i enable the NAT openvpn iot subnet iot address
what i find weird is i dont have camera nat enable yet its established... but when its master i have to enable the NAT as it wont access untill i enable that NAT route.. its weird.. but that seems to work.. and you will say that shouldnt work like that im sure
-
@comet424 said in my openvpn site to site i cant seem to ping or access other site doesnt stay stable:
so when you say HA creates asymmetric route
like for cameras or IOT devices it doesnt doHA --> PFsense --> IOT Device 192.168.20.x
instead what its doing is
HA --> IOT Device
skipping PF Sense directly
but if i pinging 192.168.1.12 its neither on any of those 3 subnet interfaces so shouldnt it try to directly go out the pfsense?
Yes exactly. You cannot have asymmetry to the remote subnet as the traffic must pass the firewall both ways.
But, yes, it could still try to send from any of those interfaces. However there should be a way to set one of those as priority in some way so it always uses 192.168.0.12 as the source for routed connections.
That last view showing it using the cameras subnet as source is still filtered by rule 137 so it's only showing states opened by that. You need to remove that so you see the outbound state it's using If it;s correctly opening a state on the openvpn interface then it's almost certainly blocked at the other end.
-
@stephenw10
ah ok sorry about that it always defaults to 137 whatever that means... here is a new screen shot..
ill have to see if i can help from the Unraid group... there is boot order for the VM i guess so you can load the nics in order. as right now it just randomly picks whatever first.. it gets frustrating i tell ya..
i know on my host openvpn of the site to site 192.168.1.1 i added the openvpn connection for remote local i did
192.168.0.0/24,192.168.10.0/24,192.168.20.0/24but here is a pic from the 192.168.1.1 pfsense the states
i guess sometimes its almost a guessing game? like the openvpn NAT i dont need one for LAN connection but i needed to add one for CAMERAS and IOT to get it to connect..
-
and you mentioned to try to get HA to be master for the 192.168.0.12 is the reason it works all the time as its the physical LAN and the Camera and IOT are just VLANs on why it works differently ?
-
@comet424 said in my openvpn site to site i cant seem to ping or access other site doesnt stay stable:
it always defaults to 137
It does that if you click on the state count on the rule to reach the states screen. But if you just go to Diag > States from the menu it should not have anything. If it does I'd check you don't have some auto-fill enabled that's adding it in the browser.
But I don't see any pings in those tables. What we want to see is the pings states created by a failing ping. So start a continuous (or very long!) ping sources from the camera interface address then check the state table at each pfSense to see what it's doing.
