Monitor is FALSE detecting one of my WANs as DOWN and another WAN as UP

johnpoz

You have bad checksums.. and sending multiple requests and getting back 1 reply..

You have something borked there… So looks like your monitor is going out BAD... And what your dump is showing as answer is your normal ping..

dims

@johnpoz:

You have bad checksums.. and sending multiple requests and getting back 1 reply..

Yes. Why can it happen? It never happen with normal ping. At least I was running it for dozens of minutes and saw no case of packet loss.

And what your dump is showing as answer is your normal ping..

I did only 5 pings, then I pressed Ctrl-C. All other pings are from someone else, I expect dpinger.

johnpoz

exactly if the monitors are going out bad and not getting a response then yes it would show it offline since its not getting an answer to its ping..

Derelict

@dims:

@Derelict:

host route is created to steer all traffic to that address out a specific interface.

It should not happen silently. A link to this route should appear near appropriate configuration window so that administrator could check if this route interferes with something he set in other places. This is bad design case.

There is no choice. It MUST force that route out the set gateway or it will go out the default gateway. The whole point is to bind the traffic to the specific interface. You can always look at the entire routing table in Diagnostics > Routes which is what a competent administrator would do.

If the system doesn't smart enogh to run out of the box automatically, it should be configurable. If it is not configurable, it should be smart enough to run automatically.

You have the special case. That is never going to work out-of-the-box. It will require configuration and tuning to make multi-wan work how you need it to work.

You can't do Apple iOS which doesn't run and unable to configure.

I have no idea what that means.
Diagnostics > Routes

It is smart enough to run automatically. You have the special case/requirements.

Derelict

@johnpoz:

You have bad checksums.. and sending multiple requests and getting back 1 reply..

You have something borked there… So looks like your monitor is going out BAD... And what your dump is showing as answer is your normal ping..

The bad checksums are probably the result of checksum offloading. The OS doesn't calculate the checksums so they are 0 when tcpdump sees it. They are calculated and inserted by the NIC hardware. They are only being displayed because of his tcpdump flags.


17:24:27.148884 IP (tos 0x0, ttl 64, id 4107, offset 0, flags [none], proto ICMP (1), length 28, bad cksum 0 (->6685)!)
    192.168.100.2 > 95.165.128.1: ICMP echo request, id 44623, seq 774, length 8
17:24:37.150298 IP (tos 0x0, ttl 64, id 4605, offset 0, flags [none], proto ICMP (1), length 28, bad cksum 0 (->6493)!)
    192.168.100.2 > 95.165.128.1: ICMP echo request, id 44623, seq 775, length 8
17:24:37.151978 IP (tos 0x20, ttl 254, id 4605, offset 0, flags [none], proto ICMP (1), length 28)
    95.165.128.1 > 192.168.100.2: ICMP echo reply, id 44623, seq 775, length 8
17:24:47.152278 IP (tos 0x0, ttl 64, id 13649, offset 0, flags [none], proto ICMP (1), length 28, bad cksum 0 (->413f)!)
    192.168.100.2 > 95.165.128.1: ICMP echo request, id 44623, seq 776, length 8
17:24:57.154279 IP (tos 0x0, ttl 64, id 56463, offset 0, flags [none], proto ICMP (1), length 28, bad cksum 0 (->9a00)!)
    192.168.100.2 > 95.165.128.1: ICMP echo request, id 44623, seq 777, length 8
17:25:07.155922 IP (tos 0x0, ttl 64, id 37697, offset 0, flags [none], proto ICMP (1), length 28, bad cksum 0 (->e34e)!)
    192.168.100.2 > 95.165.128.1: ICMP echo request, id 44623, seq 778, length 8
17:25:07.157606 IP (tos 0x20, ttl 254, id 37697, offset 0, flags [none], proto ICMP (1), length 28)
    95.165.128.1 > 192.168.100.2: ICMP echo reply, id 44623, seq 778, length 8

You are not getting a LOT of responses. Of course the GW is marked as down.

The default ping interval is two per second. You are doing one every 10 seconds. What else have you changed? How about you post the settings for that gateway - particularly the advanced settings which you have obviously changed from the defaults.

dims

@Derelict:

You are not getting a LOT of responses.

Why? How does dpinger able to achieve this?

The default ping interval is two per second. You are doing one every 10 seconds. What else have you changed?

I played with intervals because I was thinking target host has some sort of flood protection. Neither setting helped, including default one.

How about you post the settings for that gateway - particularly the advanced settings which you have obviously changed from the defaults.

I would not change defaults if they worked.

You are welcome:

dims

@Derelict:

It is smart enough to run automatically. You have the special case/requirements.

Sure, this is what I am saying: pfSense does not work in "special case" of having 3 WANs round robbin.

dims

@johnpoz:

exactly if the monitors are going out bad and not getting a response

Why this can happen? Taking into consideration, that normal ping works?

Derelict

Only your ISP can tell you why sent echo requests are not responded to.

Set all of those settings back to the default. Does it work?

If not, take a quick packet capture for posting here then try setting the data payload to 2. Does it work?

If not, take a quick packet capture for posting here then try setting the data payload to 64. Does it work?

If not, take a quick packet capture for posting here then post all of the above.

dims

@Derelict:

Only your ISP can tell you why sent echo requests are not responded to.

How provider can technically distinguish pings from ping command and from dpinger?

Derelict

The payload size of a normal ping from the ping command is 56 bytes. The payload size of a dpinger ping is 0 by default.

Hence why I asked you to do what I did.

It's pretty rare but some devices freak out with the 0-byte payload even though it's completely legal.

dims

Setting payload to 56 immediately made monitor think gateway is up, thank you!

I have set it to 56 in other monitors, and picture also became better: now zeros in RTT and RTTsd columns!

dims

No, last statement was wrong: after some time these columns got some positive values.

pwood999

I had flase GW down reports when I first configured multi-wan. Then changed Monitor IP's from google to OpenDNS, and problems went away.

Also this gives RTT & RTTsd figures which are probably more realistic for internet connectivity, rather than using the next-hop provider IP.

DSL GW = 7.5mS & 0.2mS
Cable GW = 15mS & 3.5mS

Cable is always longer due to the way Docsis works !!

Gektor

I am have same issue and same scheme as in author of topic with gateway group:
1 gateway is tier 2 (connected directly to the internet provider, used as backup)
2 gateway is tier 1 (connected to external router 1 with VLAN2 by DHCP)
3 gateway is tier 1 (connected to external router 2 with VLAN3 by DHCP)
and last 3 gateway can't resolve "UP" state of gateway WAN when external router 2 changes WAN IP. Monitoring IP on 2 gateway is 8.8.4.4, on 3 gateway 8.8.8.8.
Payload and changing monitoring IP did not help at all, when i set "Trigger Level' as "High Latency" it works a little better, but when i set it to "Packet Loss" and in time when router 2 on 3 gateway changes WAN ip - on pfSense in100% case it will marks 3 gateway as offline forever, but in Diagnostics -> Ping i can ping any address from 3 gateway without any problems. If i will go to System -> Routing and save and apply any gateway without changes - 3 gateway will back to Online state till router 2 changes WAN IP address.
2 gateway have same type router as on 3 gateway, but when it's change IP - pfSense make it Offline for few minutes and then make it back to Online state in any types of Trigger Level.
It seems that pfSense buggy with dpinger on some scenarios.

Gektor

It's more interesting, i switch Trigger Level to High Latency, and some time later pfSense himself switch it to Packet Loss! I didn't understand, why it happens.

And for sure it's dpinger bugs related, case i have checked "Disable Gateway Monitoring Action" and then made reconnect on router 2 (3 gateway) and in "Gateway status" i get on 3 gateway "Danger, Packetloss: 100%" on 3 gateway, i have check - traffic still goes through 3 gateway (router 2) without any problems, but dpinger thinks that it's dead for sure forever, till i make "save and apply" in any gateway settings.

I didn't now how to make monitoring work in pfSense. :(