@robh-0 What happens if you restart the dpinger service instead?
There are situations where the pings are responding properly (do you have a monitoring IP configured?) and a dpinger bounce will help that.

Are all your shared gateway rules switched to the new gateway group or did you simply update your original failover group to reflect the change?

Lastly is it a case of some but not all of the traffic is routing out WAN1 after fail-back? If so that's a states issue and they will reset when the states clear out for the WAN2 connection.

@jemadsen
Thank you Steve for the recommendations.
I will look into reflection and it sound like it will help.
I have been doing traceroutes, tcpdumps, netcats. I setup a dual setup, where the VM FW handles the mail, VoIP and Web traffic. The Proctectli handles the rest of the traffic thru Starlink.
I scanned some of the troubleshooting document and tried the recommendations.
I initially started with the VM configuration, but when it didn't work, I reset and manually configured the Proctectli. The VM was my learning environment with more "STUFF" to take into account to troubleshoot.
I have built up several pfSense FWs over the years and I know most of my mestakes, unfortunately, I am also good making new ones. 😊
Having the VM FW to use as reference. I searched for some example similar to mine, but all were failover/load balancing.
I need to get my servers back online so I am using both FWs one that works for the CLWAN and the other for the SLWAN. It working now. Next I will build up a test setup on my VM Server and trouble with that.
I will continue to look for an example or recipe. It is my birthday this weekend. 🎂

@robh-0 Hi , but i dont have another option. i have a server with 256 GB ram and i will use 128 GB for Pfsense and other for creating more virtual machns.

i have similar setup on esxi with 8GB ram and 2 ISP links with morethan 400-600 users connection and i never faced any issues.

@cool_corona blocking bogon on a lan side interface is normally never a good idea.. That rule is not a destination rule that is a source rule. How would you have source traffic coming into your interface with a bogon source?

And if your rules limit to the net your on, say lan net then the default deny would stop anything with a bogon source anyway.

Does anyone have any idea on the implementation of this please?^

@bygiuse Short answer is Yes to all questions.

@chuck1968 Talk to the ISP, you can’t do much with 2 overlapping IP addresses.

Maybe the WISP can provide another SSID with a diffferent subnet.

Nobody has an explanation or a clue ?
Do i need to fill a bug request ?

@justconfused I can marked it solved for you. So your currently working? While as I said I really don't think this is a pfsense issue - want to help in anyway we can to get you working.

@hugoeyng Use a DynDNS service and connect to the hostname associated.

Guys, I just started over on this. I know I should troubleshoot these types of issues instead of starting over. But I did start over and I have a working Pfsense firewall. Something I've been working on for a long time. Now the hard part will be for me to build and configure my pentesting lab behind that firewall.

@pfpv I don't know. I did see that emails have a timeout, default is 20 seconds. When I pull the wan1 link, the failover happens in <5 seconds.

I've run multi wan for decades... never had problems w/ this feature even when I used linksys/dlink.

The second part worked on second attempt. Below is the third last part, hopefully. It looks like I need to write more not to be flagged as spam.

Aug 21 05:23:24 php-fpm 453 /rc.dyndns.update: phpDynDNS (): No change in my IP address and/or 15 days has not passed. Not updating dynamic DNS entry. Aug 21 05:23:24 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:24 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:24 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:23 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:23 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:23 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:23 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:23 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:22 php-fpm 453 /rc.dyndns.update: phpDynDNS (): No change in my IP address and/or 15 days has not passed. Not updating dynamic DNS entry. Aug 21 05:23:22 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:22 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:22 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:22 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:21 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:21 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:21 php-fpm 453 /rc.dyndns.update: phpDynDNS (): No change in my IP address and/or 15 days has not passed. Not updating dynamic DNS entry. Aug 21 05:23:21 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:21 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:21 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:21 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:21 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:21 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:21 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:21 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:20 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:20 php-fpm 453 /rc.dyndns.update: phpDynDNS (): No change in my IP address and/or 15 days has not passed. Not updating dynamic DNS entry. Aug 21 05:23:20 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:20 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:19 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:19 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:19 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:19 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:19 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:19 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:19 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:19 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:19 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:19 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:19 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:19 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:19 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:19 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:19 php-fpm 453 /rc.dyndns.update: phpDynDNS (): No change in my IP address and/or 15 days has not passed. Not updating dynamic DNS entry. Aug 21 05:23:19 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:19 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:19 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:19 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:19 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:18 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:18 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:18 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:18 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:18 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:18 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:18 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:18 php 33487 notify_monitor.php: Message sent to Email OK Aug 21 05:23:18 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:18 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:17 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:17 php-fpm 453 /rc.dyndns.update: phpDynDNS (): No change in my IP address and/or 15 days has not passed. Not updating dynamic DNS entry. Aug 21 05:23:17 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:17 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:16 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:16 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:16 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:16 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:16 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:16 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:16 php-fpm 453 /rc.dyndns.update: phpDynDNS (): (Success) IP Address Updated Successfully! Aug 21 05:23:16 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:16 php-fpm 453 /rc.dyndns.update: phpDynDNS: updating cache file /conf/dyndns_PreferWANcustom''5.cache: q.q.q.q(WAN2_IP) Aug 21 05:23:16 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:16 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:16 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:16 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:16 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:16 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:16 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:16 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:16 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:16 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:15 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:15 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:15 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:15 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:15 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:15 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:15 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:15 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:14 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:14 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:14 php-fpm 453 /rc.dyndns.update: phpDynDNS (): (Success) IP Address Updated Successfully! Aug 21 05:23:14 php-fpm 453 /rc.dyndns.update: phpDynDNS: updating cache file /conf/dyndns_PreferWANcustom''4.cache: q.q.q.q(WAN2_IP) Aug 21 05:23:14 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:14 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:14 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:14 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:14 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:14 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:14 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:14 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:14 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:14 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:14 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:14 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:13 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:13 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:13 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:13 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:13 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:13 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:13 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:13 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:13 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:12 php-fpm 453 /rc.dyndns.update: phpDynDNS (): (Success) IP Address Updated Successfully! Aug 21 05:23:12 php-fpm 453 /rc.dyndns.update: phpDynDNS: updating cache file /conf/dyndns_PreferWANcustom''3.cache: q.q.q.q(WAN2_IP) Aug 21 05:23:12 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:12 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:12 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:12 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:12 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:12 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:12 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:11 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:11 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:11 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:11 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:11 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:11 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:11 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:11 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:11 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:11 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:11 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:11 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:11 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:11 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:11 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:11 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:10 php-fpm 453 /rc.dyndns.update: phpDynDNS (): (Success) IP Address Updated Successfully! Aug 21 05:23:10 php-fpm 453 /rc.dyndns.update: phpDynDNS: updating cache file /conf/dyndns_PreferWANcustom''2.cache: q.q.q.q(WAN2_IP) Aug 21 05:23:10 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:10 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:10 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:10 php-fpm 40773 /rc.filter_configure_sync: GW States: Gateway is down but its IP address cannot be located. Skipping state kill.: WAN_DHCP Aug 21 05:23:10 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:10 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:10 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:10 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:10 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:10 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:10 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:10 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:09 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:09 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:09 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:09 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:09 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:09 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:09 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:09 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:09 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:09 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:08 php-fpm 453 /rc.dyndns.update: phpDynDNS (): (Success) IP Address Updated Successfully! Aug 21 05:23:08 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:08 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:08 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:08 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:08 php-fpm 453 /rc.dyndns.update: phpDynDNS: updating cache file /conf/dyndns_PreferWANcustom''1.cache: q.q.q.q(WAN2_IP) Aug 21 05:23:08 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:08 php-fpm 64592 /rc.filter_configure_sync: GW States: Gateway is down but its IP address cannot be located. Skipping state kill.: WAN_DHCP Aug 21 05:23:08 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:08 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:08 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:07 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:07 php 33487 notify_monitor.php: Message sent to Email OK Aug 21 05:23:07 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:07 php-fpm 453 /rc.dyndns.update: phpDynDNS (): (Success) IP Address Updated Successfully! Aug 21 05:23:07 php-fpm 453 /rc.dyndns.update: phpDynDNS: updating cache file /conf/dyndns_PreferWANcustom''0.cache: q.q.q.q(WAN2_IP) Aug 21 05:23:07 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:07 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:06 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:06 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:06 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:06 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:06 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:06 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:06 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:06 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:06 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:06 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:06 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:06 kernel arpresolve: can't allocate llinfo for y.y.y.y(WAN1_GW_IP) on igb0 Aug 21 05:23:06 php-fpm 454 /rc.openvpn: Default gateway setting Interface WAN2_PPPOE Gateway as default. Aug 21 05:23:06 php-fpm 454 /rc.openvpn: Gateway, switch to: WAN2_PPPOE Aug 21 05:23:05 php-fpm 64592 z.z.z.z(WAN1_monitoring_IP)|192.168.100.2|WAN_DHCP|0ms|0ms|100%|down|highloss Aug 21 05:23:05 php-fpm 64592 /rc.filter_configure_sync: MONITOR: WAN_DHCP has packet loss, omitting from routing group PreferWAN Aug 21 05:23:05 check_reload_status 483 Reloading filter Aug 21 05:23:05 check_reload_status 483 Restarting OpenVPN tunnels/interfaces Aug 21 05:23:05 check_reload_status 483 Restarting IPsec tunnels Aug 21 05:23:05 check_reload_status 483 updating dyndns WAN_DHCP Aug 21 05:23:05 rc.gateway_alarm 25620 >>> Gateway alarm: WAN_DHCP (Addr:z.z.z.z(WAN1_monitoring_IP) Alarm:1 RTT:0.000ms RTTsd:0.000ms Loss:100%) Aug 21 05:23:03 check_reload_status 483 Reloading filter Aug 21 05:23:03 php-fpm 40773 /rc.linkup: Removing static route for monitor p.p.p.p(WAN3_monitoring_IP) and adding a new route through k.k.k.k(WAN3_IP) Aug 21 05:23:01 php-fpm 40773 /rc.linkup: DEVD Ethernet detached event for wan Aug 21 05:23:00 check_reload_status 483 Linkup starting igb0 Aug 21 05:23:00 kernel igb0: link state changed to DOWN

overview.png

I made a sketch to clarify