Two firewall accessing each other when gateway is down
-
@Farh
Both firewalls give you diagnostic tools like ping or port probe. There is no need to attach an additional device the check this out.How are your WANs configured? Do they have static IP or DHCP?
-
@viragomann yes of course you're right but i can not take internet down on that time and i have no choice other than use another device.
It's static IP address with one wan ip and 4 virtual IP on pfsense and one static IP on opnsense
-
@Farh said in Two firewall accessing each other when gateway is down:
Until today i expected to access same wan subnet on layer 2 when gateway is down but today when my internet goes down i lose the connection.
No. It's not in ARP table.
No ARP, no layer 2 communication.
Seems you have an L2 issue.
Is there even an entry of the respective other WAN in the ARP table if the gateway is up?If not, to investigate the issue, go on one of the firewalls and start a packets capture of ARP. Then try to ping the other box.
Check after, what you got in the capture. -
@viragomann thanks for your guidance
pinging other side added it to ARP table.
is ICMP most effective rather than TCP for adding to ARP table or it's even required ??
But unfortunately even after both firewalls have each other MAC address in ARP table issue still persist. No gateway no connection.
Add other WANs to ARP table still need to pinging it. -
@Farh
No, the protocol doesn't matter. If an IP within the subnet is requested, it does an ARP resolution.
So my assumption is, that the other WAN isn't requested at all, when you try to access it from inside the LAN.Do you policy route the LAN traffic by any chance?
-
@viragomann said in Two firewall accessing each other when gateway is down:
So my assumption is, that the other WAN isn't requested at all, when you try to access it from inside the LAN.
It's impossible because my DNS return my WAN2 IP addresses and there is no other path to access.
Also traceroute result on LAN1 shows the LAN IP of pfsense but then it shows timeout.@viragomann said in Two firewall accessing each other when gateway is down:
Do you policy route the LAN traffic by any chance?
No. I don't think so. Which kind of police do you mean. It's outbound NAT
-
@Farh
Policy routing doesn't care about DNS.
And it has nothing to do with NAT.In you WAN rules for allowing upstream traffic, did you state a gateway?
If you're unsure, please show your rules. -
@viragomann if I'm getting correct you mean choosing a gateway on firewall > rules.
In my case every rule gateway on both LAN and WAN is set to * -
First of all i want to thanks everyone reply to this topic and help me find the problem
I do the following to resolve my problem. Maybe somebody else face this problem and may it helps:
1- first i check the ARP table on both firewall and discover none of them have each other MAC addresses. It's so strange for me because i connected to them with https and http but pinging other firewall inside each other even without response that because of firewall rule added it to ARP table.
2- step one doesn't resolve the problem but i believe it was required. Secondly i try many thing s that doesn't work. After several hours of try and error i discovered that base on MAC addresses pfsense send packages directly to opnsense but opnsense reply through defaut gateway to pfsense. After searching on internet i found this link:
https://forum.opnsense.org/index.php?topic=5615.0
Some of opnsense guys says enabling "disable reply-to" option in opensense may resolve the problem.
3- surprisingly enabling " disable reply-to " resolve the problem.
I don't disconnecting the internet yet but i believe problem is resolved.
I'm glad if anyone can explain why this option worked ? Because I'm confused a little bit.
Thanks -
@Farh
Disabling reply-to on the accessed node - yeah, this could be a reason.
When enabled, replies are directed to the gateway, which is stated in the interface settings.Disabling reply-to could lead into issues with multi-WAN setup, however.
To avoid this, you can add pass rules to the top of the WAN rule set only for the source of the WAN subnet and disable reply-to in the advanced options.