Gateway drops and never comes back
-
I might also running into this problem.
I have 4 WANS, with one dynamic IP and the other 3 of them having a fixed IP. Once in a while I notice that pfsense thinks one of them is down ("Offline, Packetloss"). It is not; I also monitor them from the outside with Uptime Robot, so I do know when they have been up/down in case I need to do something about it.
So I go to System -> Routing -> Gateways, edit the gateway, remove the monitor IP, save changes, and it comes back up. Edit the gateway again, configure the same monitor IP it had, and now it will stay up.
I would say that this does not happen to the WAN that has dynamic IP, but I am not so sure. I will keep an eye of this.
Next time I will go the disable/enable gateway route to see if it also works.
-
That was me above ^^
-
So I had this happen to me again tonight. On two different pfSenses, both of them with 2 WANs, were the second WAN has fixed IP and is on the same ISP. They both went down tonight at the same time, and they both came back 13 minutes later. But on pfSense they remained offline.
I tried to disable/enable the gateway as @scottmsilver did above, but I was unable to do so, since they are part of a gateway group.
So I did as usual, remove the monitor IP, so it will use my own routr as monitor IP. A few seconds later the gateway is back up. And reconfigure again the same monitor IP I had.
-
I am having a similar (same?) issue on 22.01 as well as 21.05.1 and 21.05.2 on a SG-2100. I have two gateways, a bog standard configuration composed of a DHCP WAN interface gateway, as well as an OVPN Gateway on a virtual interface. The WAN interface represents a remote cellular connection, and as you might expect, it isn't that stable. I have a gateway monitor applied pinging Cloudflare DNS Servers, and this works until the first time the Gateway goes down. At that point, the Gateway sticks in "Pending, Gathering Data" in the Gateway Group. Just as @scottmsilver points out, in the logs they'll be entries showing sendto errors for a couple of tries, and then nothing more. The Gateway is forever in pending.
For me, the fix is simpler. If I go to System -> Routing -> Edit the WAN_DHCP Gateway, then simply scroll to the bottom and click Save without changing anything on the page, and finally Apply Changes, the Gateway immediately comes Online. In the logs, dpinger immediately logs the configuration of the monitor:
send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr 1.1.1.1 bind_addr <ipaddress> identifier "WAN_DHCP "
From there, the Gateway will stay up again until the cellular link is lost, then it will go back to pending. It just seems like dpinger gives up monitoring the new WAN interface every time DHCP applies a new lease on link restoration.
-
I got the same issue here. instable celluar connection, gateway goes to pending, saving the gateway again makes it work again. Now i changed it from DHCP to static , will see if this will be better...
-
-
@scottmsilver thanks for that. I had seen it while making my post and I can confirm it does the job on my side too. Was searching to see if you had opened a bug on redmine for this (I couldn't find one). If you had not, I was going to so that there's at least a chance this can get fixed in a future revision.
-
-
@jimp
I got some systems with multiple WAN on unstable celluar connections, with GW groups.Is there a chance to get this fix or changeset
https://redmine.pfsense.org/projects/pfsense/repository/1/revisions/ec73bb89489d830ec21c4e04ffa3ec401791b55d/diff
for 2.7 as a patch for 2.6?
-
@pete35 In System Patches, Add New Patch and use the ID on that diff page (ec73bb89489d830ec21c4e04ffa3ec401791b55d). The patches just apply the diff to the files on disk.
-
@steveits That's pretty cool. I didn't know about that!
-
@steveits
ok thank you, done that. -
Old subject, but in case it can help someone, I found the solution this morning. :-)
Add the line "setup_gateways_monitor();" in the file /etc/rc.newwanip, at the end, after the last instruction ( "filter_configure();" ), still in the "else" condition.
This way, after a disconnection, gateway_monitor and dpinger will be reconfigured all the same even if WAN IP hasn't changed.
After a few seconds, WAN gateway will be automatically up again.Tested with v2.6.0.
-
-
@steveits Sorry, didn't know where to find this. Thanks.
-
This is still an ongoing issue for me. Not a big deal, but I really would not feel confortable myself telneting on my pfSense boxes and manually editing some files (those changes I guess might be loss when pfSense updates itself?)
Running 22.05-RELEASE, and the issue is a described above. When a WAN connection with fixed IP goes down, it will come back up, but still appear as offline on pfSense until I change the monitor IP to something else.
My quick fix would be to just leave my ISP router IP as the monitor IP for that WAN connection, but of course then pfSense would not notice when the connection is really down (it rarely happens anyway).
-
@mindtwist You can use the System Patches package and enter the ID for the patch (ec73bb89489d830ec21c4e04ffa3ec401791b55d) and it will update the files.
I'm curious, does it work if you view the System\Routing\Gateways page or only if you change the IP? 2-3 years ago there was a similar issue but viewing the page fixed it so we set up a cron to run every so often...don't recall without looking but I think it called a function.
-
@mindtwist : I confirm this change is permanent (I tested it, it survives a reboot), until an OS upgrade of course.
If you really don't feel comfortable, you may backup the original file, and work on a copy, so that you can restore it in case something goes wrong (WinSCP is perfect for this). But adding a single line is not a big deal, it's easy to revert. ;-)@SteveITS : I tried the patch you mentioned, it actually doesn't work. Fortunately, the files are very well commented (thanks to the devs !), which make them quite easy to understand. The issue comes from the fact that dpinger is reset only when the new WAN IP is different from the previous one, whereas it would need to be even if the IP is the same (because of firewall states). This is corrected by adding this only code line.
-
@steveits said in Gateway drops and never comes back:
I'm curious, does it work if you view the System\Routing\Gateways page or only if you change the IP? 2-3 years ago there was a similar issue but viewing the page fixed it so we set up a cron to run every so often...don't recall without looking but I think it called a function.
The interface does come back online just by going to System/Routing/Gateways, hitting "Save", and then "Apply". I do not need to change the IP, but I do have to do a Save/Apply.
-
@ludo9176 said in Gateway drops and never comes back:
@mindtwist : I confirm this change is permanent (I tested it, it survives a reboot), until an OS upgrade of course.
If you really don't feel comfortable, you may backup the original file, and work on a copy, so that you can restore it in case something goes wrong (WinSCP is perfect for this). But adding a single line is not a big deal, it's easy to revert. ;-)I am not even sure what is the file I have to modify, or what change to apply.
I think I need to make the change scottmsilver posted on 16/Dic/21 on line 204 of rc.newwanip, but I am unable to find that rc.newwanip file.