Gateway monitor IPs are being put into the routing table
-
Well, i'd like to have this wrong behaviour fixed, simple as that. That's not a feature request in my book.
In scenario 1, the firewall is sending traffic out to interfaces it's not supposed to do
In scenario 2, the monitor IP becomes completely unreachable although it's reachable just fine over the other WAN interface
In scenario 3, the monitor IP becomes completely unreachable, just because the currently inactive WAN connection goes down
All of the above three issues are not documented anywhere and looking at the forum posts regarding Multi-WAN failover, I can tell other people are experiencing these issues also.
Let me ask you a direct question: How can you call the above three issues "working as designed"? What would you name that feature that fixes the three above mentioned scenarios?
-
Your use case is lets call it ODD to say the least..
When monitoring your wan.. The NORMAL scenario is to ping the gateway of that wan interface.. Its not some IP out of the public internet that needs to be gotten to.
You are choosing to use some IP that you want to also use, like 8.8.8.8 - pick something else for starters.
So yeah I would call what your asking for a "feature"..
-
Your use case is lets call it ODD to say the least..
When monitoring your wan.. The NORMAL scenario is to ping the gateway of that wan interface.. Its not some IP out of the public internet that needs to be gotten to.
No, it's not odd to do that, not at all. It's necessary, otherwise many outages are not detected as the gateway often stays up because something further upstream breaks.
In fact, you guys explain it in the documentation and recommend not using the WAN gateway:
By default pfSense will ping the gateway to determine the quality of the WAN. In some cases, that is not an accurate measure. For instance, if the WAN gateway is actually a device that is local and not on the other side of the ISP circuit, then the actual WAN link could be down and pinging the gateway would never show it. Also, if the ISP gateway is up but the ISP experiences upstream failures, those cannot be detected by pinging only the gateway.
A custom IP address can be entered to monitor here that will be used to determine the WAN quality. A public website, Google public DNS, or any IP on the Internet that responds to pings can be used. The downside is that should that IP ever go offline, or suffer a failure of its own, the WAN could be marked down when it’s really up.
(https://docs.netgate.com/pfsense/en/latest/routing/multi-wan.html)You are choosing to use some IP that you want to also use, like 8.8.8.8 - pick something else for starters.
See, that's the hard part. I've already mentioned it in the first post of this thread. There are not that many reliable anycast IPs out there where I can be sure that they are never used by my users. I know for example they use Google DNS, so 8.8.8.8 and 8.8.4.4 are not possible anymore. I also know they use Cloudflare DNS, so 1.1.1.1 is also not possible anymore.
I'd need two IPs that are:
- reliable
- never used by my users
That's not easy to find.
-
No it is ODD.. you pay your isp for connectivity... If something upstream breaks like the site your monitoring goes down.. Your internet connection still works.
They mention that scenario for when say your gateway is local to pfsense, say your behind a double nat. Or an upstream provider router for example that is on site.
"if the WAN gateway is actually a device that is local and not on the other side of the ISP circuit"Also, where did they say you should do that in a multi wan scenario?
Pick and ip just upstream of your gateway.. Do a traceroute, whats the next hop past your gateway? Use that!
The monitoring of your wan connection is meant to monitor the connection, not that path to internet is broken to some X ip out on the internet.. Or that the path is much longer because some peer connection when down somewhere upstream in the internet. So now your asking to use the monitoring of the connection as best path selection..
So yeah what your asking for is a "feature" And the current system works as designed... Be it all the things doesn't work how you want them to work, doesn't mean its broken in any way.
While your at is requesting this feature you want - you should also ask to measure latency to every dest IP gone to, and if isp A is faster than B, use A vs B ;) And if at some point later B becomes faster switch over to B for that destination.
There is nothing saying that the monitor IP you pick has to be anycast. Monitoring is not meant to monitor the internet and some specific ISPs connectivity to it.. Its meant to monitor if your ISP connection is up and working.
Pretty much any major dns providers IPs are going to be anycast.. Pick one your not using if that is what you want. Why are you letting users use other dns other than your local dns in the first place ;)
-
They mention that scenario for when say your gateway is local to pfsense, say your behind a double nat. Or an upstream provider router for example that is on site.
"if the WAN gateway is actually a device that is local and not on the other side of the ISP circuit"Yes, that is exactly the case for me, the gateway is local. It's a pretty much standard internet connection as done by many ISPs around the world. A /29 network with one of those IPs being configured on a router in the same building. And even if the gateway wouldn't be local, there is still a lot of stuff between that gateway and "the internet", so no, it doesn't make sense to monitor only the gateway. Also not something further upstream. Not at all. And actually, there are not many cases where that makes sense.
Also, where did they say you should do that in a multi wan scenario?
It says "Using Multiple IPv4 WAN Connections" in the headline. Did you actually ever read the documentation? Or atleast looked at the URL, which also contains the words "multi-wan"?
Pick and ip just upstream of your gateway.. Do a traceroute, whats the next hop past your gateway? Use that!
Bad idea because of several reasons:
- sometimes ISPs re-arrange their network and routing which could lead to that IP becoming unavailable
- those IPs belong to routers whose job is to forward traffic, not answer ICMP requests, often they don't answer reliably
- sometimes, ISPs just decide to not have their routers answer ICMP echo requests anymore
So yeah what your asking for is a "feature" And the current system works as designed... Be it all the things doesn't work how you want them to work, doesn't mean its broken in any way.
Let me ask you the same direct question I asked Derelict: How can you call the above three error scenarios "working as designed"? What would you name that feature that fixes the three above mentioned scenarios?
The monitoring of your wan connection is meant to monitor the connection, not that path to internet is broken to some X ip out on the internet.. Or that the path is much longer because some peer connection when down somewhere upstream in the internet. So now your asking to use the monitoring of the connection as best path selection..
The monitoring is meant to make sure that "the internet connection" works. Meaning I can actually reach "the internet". For that, I have to ping an IP that is actually on "the internet". It's completely pointless knowing that some IP inside my ISPs core network is still pingeable, but the internet doesn't work. In that case, the automatic failover won't occur and users will stay offline.
There is nothing saying that the monitor IP you pick has to be anycast. Monitoring is not meant to monitor the internet and some specific ISPs connectivity to it.. Its meant to monitor if your ISP connection is up and working.
Ofcourse it doesn't need to be anycast per se, it only needs to be reliable. Which anycast usually is, because, well, it's anycast, i.e. not dependent on a single host or path. Like I explained above,
the monitor IPs need to be reliable, otherwise people will suffer from unecessary false-alarm WAN failover switching caused by the monitor IPs being down but not the path to the internet.But whatever, I see it's pointless, you guys declare this working as designed and that's it. No, I'm not going to request a "don't route monitor IP traffic the wrong way" feature.
Thanks for re-assuring me to never go for a support contract or buy hardware from Netgate.
-
As was specified in the redmine, it is not a bug and is necessary to ensure the packets to go out the gateway being monitored. So there's your answer.
-
Hi A Former User
I had the same problem, but you can do a policy route for the monitoring IPs on the LAN interface with the gateway group as gateway.
-
Yes, if the pings are sourcing from something on the LAN and not the firewall itself, that should work fine.
-
@dynach do you think policy routing could be useful in this case as well?
Basically the issue it's the same of "A former user".https://forum.netgate.com/category/32/ha-carp-vips
-
necro'ing this thread to point out that Comcast (at least in my area) is no longer responding to pings on the DHCP assigned gateway, which might push this scenario a little further towards normal on the normal to odd continuum. This has forced me to put a public ip in dpinger which is not pfSense's fault, and may still be more ODD than not.
-
This may help:
-
i tried your patch but it doesnt seem to be correct, any ideas ?
/usr/bin/patch --directory=/ -f -p2 -i /var/patches/6214fbf0c578a.patch --check --reverse --ignore-whitespace
Hmm... Looks like a unified diff to me...
The text leading up to this was:|From ad2dd2c1debaf2281eaed685ad8464c027c3b0b1 Mon Sep 17 00:00:00 2001
|From: luckman212
|Date: Tue, 18 Jan 2022 16:36:58 -0500
|Subject: [PATCH] squash
|
|---
| src/etc/inc/gwlb.inc | 54 +++++++++++--------
| .../include/www/system_advanced_misc.inc | 1 +
| src/usr/local/www/system_advanced_misc.php | 9 ++++
| src/usr/local/www/system_gateways_edit.php | 12 +++++
| 4 files changed, 54 insertions(+), 22 deletions(-)
|
|diff --git a/src/etc/inc/gwlb.inc b/src/etc/inc/gwlb.inc
|index 31dde0ee4ad..2b471571944 100644
|--- a/src/etc/inc/gwlb.inc+++ b/src/etc/inc/gwlb.inc Patching file etc/inc/gwlb.inc using Plan A... Hunk #1 failed at 239. Hunk #2 failed at 282. Hunk #3 failed at 2078. 3 out of 3 hunks failed while patching etc/inc/gwlb.inc Hmm... The next patch looks like a unified diff to me... The text leading up to this was:
|diff --git a/src/usr/local/pfSense/include/www/system_advanced_misc.inc b/src/usr/local/pfSense/include/www/system_advanced_misc.inc
|index 6cb826693cb..aa8c24d1c37 100644
|--- a/src/usr/local/pfSense/include/www/system_advanced_misc.inc+++ b/src/usr/local/pfSense/include/www/system_advanced_misc.inc Patching file usr/local/pfSense/include/www/system_advanced_misc.inc using Plan A... Hunk #1 failed at 56. 1 out of 1 hunks failed while patching usr/local/pfSense/include/www/system_advanced_misc.inc Hmm... The next patch looks like a unified diff to me... The text leading up to this was:
|diff --git a/src/usr/local/www/system_advanced_misc.php b/src/usr/local/www/system_advanced_misc.php
|index 9806aac040e..4f676b58feb 100644
|--- a/src/usr/local/www/system_advanced_misc.php+++ b/src/usr/local/www/system_advanced_misc.php Patching file usr/local/www/system_advanced_misc.php using Plan A... Hunk #1 failed at 304. 1 out of 1 hunks failed while patching usr/local/www/system_advanced_misc.php Hmm... The next patch looks like a unified diff to me... The text leading up to this was:
|diff --git a/src/usr/local/www/system_gateways_edit.php b/src/usr/local/www/system_gateways_edit.php
|index 96b80171790..57afa7ce7f7 100644
|--- a/src/usr/local/www/system_gateways_edit.php+++ b/src/usr/local/www/system_gateways_edit.php Patching file usr/local/www/system_gateways_edit.php using Plan A... Hunk #1 failed at 72. Hunk #2 failed at 223. 2 out of 2 hunks failed while patching usr/local/www/system_gateways_edit.php done
