Navigation

    Netgate Discussion Forum
    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search

    Monitor is FALSE detecting one of my WANs as DOWN and another WAN as UP

    Routing and Multi WAN
    7
    39
    1279
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • D
      dims last edited by

      I have 3 WANs, each WAN has static outer IP address, given by provider.

      The topology is following:

      When I have no money on provider2 or I have any other problems with it, I can't ping IP2 from inside my LAN and from AWS hosting.

      Nevertheless, monitoring of this gateway indicates green.

      I.e. I have FALSE POSITIVE detection.

      An opposite is happening with provoder3. If I have everything OK with it, and I can ping IP3 from inside my LAN and from AWS hosting, apinger reports it is down.

      I.e. I have FALSE NEGATIVE detection.

      All gateways are cofigured as parts of one Load_Balancing_Group at tier 1.

      provider1 is configured as default and works ok.

      How it can be?

      1 Reply Last reply Reply Quote 0
      • C
        ccmks last edited by

        What are the monitor IP you use?

        1 Reply Last reply Reply Quote 0
        • D
          dims last edited by

          Outer IP of my provider. I can ping it from outside at the very same moment when apinger reporting it's down.

          1 Reply Last reply Reply Quote 0
          • R
            rudger_wolvram last edited by

            I had as similar problem with ATT, you couldn't ping their gateway IP from the same network (WAN interface IP) but you could from outside that subnet.
            I had to set up my monitor IP to 8.8.8.8 because it's google, they can handle the traffic, it's up pretty much all the time, and it monitors through the ISP gateway.
            Downside to this method, your interface response time reporting is skewed higher because you're hitting an actual internet host instead of the first hop.

            1 Reply Last reply Reply Quote 0
            • D
              dims last edited by

              I can ping IP3 from everywhere. Their router has WEB-interface, and it has PING page there. So, I tried to ping from:

              1. workstation inside LAN

              2. pfSense command line

              3. provider's router.

              Ping works from everywhere.

              Only apinger thinks interface is down, by unknown reason, probably BUG.

              1 Reply Last reply Reply Quote 0
              • Derelict
                Derelict LAYER 8 Netgate last edited by

                Probably NOT a bug. Try pinging the monitor IP address from the firewall itself. Diagnostics > Ping or ping from the ssh/console shell.

                Chattanooga, Tennessee, USA
                The pfSense Book is free of charge!
                DO NOT set a source port in a port forward or firewall rule unless you KNOW you need it!
                Do Not Chat For Help! NO_WAN_EGRESS(TM)

                1 Reply Last reply Reply Quote 0
                • D
                  dims last edited by

                  If by "firewall" you mean pfSense box, that I can ping from it. See above.

                  1 Reply Last reply Reply Quote 0
                  • R
                    rudger_wolvram last edited by

                    Wait, what version of pfsense are you on?
                    apinger was removed from pfsense somewhere around 2.3.x and replaced with dpinger. I'm on 2.4.2 and can't find apinger or dpinger (unless dpinger is the underlying pinger for gateways) packages.
                    You may be using a package that shouldn't even be there.

                    1 Reply Last reply Reply Quote 0
                    • Derelict
                      Derelict LAYER 8 Netgate last edited by

                      Yeah if you have a version that uses apinger, the solution is to upgrade. 2.4.2_1 is current.

                      Chattanooga, Tennessee, USA
                      The pfSense Book is free of charge!
                      DO NOT set a source port in a port forward or firewall rule unless you KNOW you need it!
                      Do Not Chat For Help! NO_WAN_EGRESS(TM)

                      1 Reply Last reply Reply Quote 0
                      • D
                        dims last edited by

                        I am using 2.3.2-RELEASE (amd64)

                        I don't see 2.4.2_1 as upgrade option. It writes Latest Base System 2.3.3_1

                        If I enable unstable and experimental releases, it writes 2.3.6.a.20180103.1249

                        The date is yesterday.

                        Are you really pfSense guys, people?

                        1 Reply Last reply Reply Quote 0
                        • Derelict
                          Derelict LAYER 8 Netgate last edited by

                          2.3.2 does not have apinger, it has dpinger. I don't recall any issues with it since then.

                          You should upgrade anyway. Take a configuration backup and give it a go. The reported version from there does not always match what you end up with, unfortunately.

                          Chattanooga, Tennessee, USA
                          The pfSense Book is free of charge!
                          DO NOT set a source port in a port forward or firewall rule unless you KNOW you need it!
                          Do Not Chat For Help! NO_WAN_EGRESS(TM)

                          1 Reply Last reply Reply Quote 0
                          • D
                            dims last edited by

                            Ah, I found newer version on site. Updater just doesn't see it…

                            1 Reply Last reply Reply Quote 0
                            • D
                              dims last edited by

                              I don't beleive it will work. If this is not recognized as a bug or problem, then unprobably it was solved…

                              1 Reply Last reply Reply Quote 0
                              • Derelict
                                Derelict LAYER 8 Netgate last edited by

                                Are you really pfSense guys, people?

                                Insults? Really?

                                Chattanooga, Tennessee, USA
                                The pfSense Book is free of charge!
                                DO NOT set a source port in a port forward or firewall rule unless you KNOW you need it!
                                Do Not Chat For Help! NO_WAN_EGRESS(TM)

                                1 Reply Last reply Reply Quote 0
                                • Derelict
                                  Derelict LAYER 8 Netgate last edited by

                                  @dims:

                                  I don't beleive it will work. If this is not recognized as a bug or problem, then unprobably it was solved…

                                  That is because it is probably not a bug or a problem. You have a unique situation and you need to figure out what to monitor so you get the results you are looking for.

                                  Sometimes when an ISP administratively shuts down a circuit for things like "no more money" they still respond to pings for some close addresses, sometimes they hijack DNS or forward all port 80 "you're out of money" page, etc.

                                  Chattanooga, Tennessee, USA
                                  The pfSense Book is free of charge!
                                  DO NOT set a source port in a port forward or firewall rule unless you KNOW you need it!
                                  Do Not Chat For Help! NO_WAN_EGRESS(TM)

                                  1 Reply Last reply Reply Quote 0
                                  • Derelict
                                    Derelict LAYER 8 Netgate last edited by

                                    1. workstation inside LAN

                                    2. pfSense command line

                                    3. provider's router.

                                    And, to add some clarity. NOTHING but pinging from the firewall itself matters for gateway monitoring. That is the only case that has any impact on the monitoring process. It does not care if you can or cannot ping the target from AWS or LAN or the "provider's router."

                                    What do you have for DNS servers in System > General? Do you have gateways set on those?

                                    What do you have for monitor IP addresses on each gateway? Are they the same or different than the DNS servers and gateways?

                                    Are you trying to use any VPN endpoints as monitor addresses?

                                    Chattanooga, Tennessee, USA
                                    The pfSense Book is free of charge!
                                    DO NOT set a source port in a port forward or firewall rule unless you KNOW you need it!
                                    Do Not Chat For Help! NO_WAN_EGRESS(TM)

                                    1 Reply Last reply Reply Quote 0
                                    • D
                                      dims last edited by

                                      @Derelict:

                                      Sometimes when an ISP administratively shuts down a circuit for things like "no more money" they still respond to pings

                                      This is not the case since I tried to ping by ping command line command.

                                      1 Reply Last reply Reply Quote 0
                                      • D
                                        dims last edited by

                                        @Derelict:

                                        NOTHING but pinging from the firewall itself matters for gateway monitoring.

                                        Okay. So ping from firewall works, while monitor says gateway is down. How can it be?

                                        What do you have for DNS servers in System > General?

                                        How DNS can affect pinging?

                                        Do you have gateways set on those?

                                        Of course not. DNS servers are given by each provider and the matter of change without notice. So I can't set static DNS addresses on General page. This is design error of pfSense (aka deliberate bug).

                                        What do you have for monitor IP addresses on each gateway?

                                        I am pinging my outer IPs for each provider. This is the only thing I can know, because I pay for them.

                                        Are they the same or different than the DNS servers and gateways?

                                        Of course they are different. I can't set DNS server to ping, because DNS server is not obliged to respong to pings.

                                        Are you trying to use any VPN endpoints as monitor addresses?

                                        I would write this, if I did.

                                        1 Reply Last reply Reply Quote 0
                                        • Derelict
                                          Derelict LAYER 8 Netgate last edited by

                                          Because there are a lot of things that have to happen to make Multi-WAN and gateway monitoring work.

                                          All of the things I mentioned are because those are instances where the firewall has no choice but to install host routes out a specific interface.

                                          When you set a gateway monitor IP address, a host route is created to steer all traffic to that address out a specific interface.

                                          When you set a gateway on a DNS server in System > General the same thing happens a host route for that DNS server out that interface.

                                          When you set an interface on an IPsec configuration, the same thing happens.

                                          I know you think this should all "just work" but in your (uncommon, complicated) situation you have to have everything just right.

                                          So you can either listen and answer questions without all the snark, or don't. Completely up to you.

                                          Okay. So ping from firewall works, while monitor says gateway is down. How can it be?

                                          Show me a packet capture on that interface where the dpinger echo requests are being sent and replies are reliably being received and the gateway is still showing as down.

                                          Chattanooga, Tennessee, USA
                                          The pfSense Book is free of charge!
                                          DO NOT set a source port in a port forward or firewall rule unless you KNOW you need it!
                                          Do Not Chat For Help! NO_WAN_EGRESS(TM)

                                          1 Reply Last reply Reply Quote 0
                                          • D
                                            dims last edited by

                                            Currently I tried to monitor the "gateway" host for the provider's modem. Situation with this address is the same: I can ping it from pfSense, but Monitor says it is down.

                                            Here is the screenshot of Monitor:

                                            1 Reply Last reply Reply Quote 0
                                            • D
                                              dims last edited by

                                              Here is the proof that ping is OK:

                                              1 Reply Last reply Reply Quote 0
                                              • D
                                                dims last edited by

                                                And here is the tcpdump. Address 192.168.100.2 is an address of pfSense re3 interface, which is connected to provider3 modem.

                                                I made a gap at the moment before I started a ping from another session in parallel of tcpdump running.

                                                 tcpdump -vnni re3 icmp
tcpdump: listening on re3, link-type EN10MB (Ethernet), capture size 65535 bytes
17:24:27.148884 IP (tos 0x0, ttl 64, id 4107, offset 0, flags [none], proto ICMP (1), length 28, bad cksum 0 (->6685)!)
    192.168.100.2 > 95.165.128.1: ICMP echo request, id 44623, seq 774, length 8
17:24:37.150298 IP (tos 0x0, ttl 64, id 4605, offset 0, flags [none], proto ICMP (1), length 28, bad cksum 0 (->6493)!)
    192.168.100.2 > 95.165.128.1: ICMP echo request, id 44623, seq 775, length 8
17:24:37.151978 IP (tos 0x20, ttl 254, id 4605, offset 0, flags [none], proto ICMP (1), length 28)
    95.165.128.1 > 192.168.100.2: ICMP echo reply, id 44623, seq 775, length 8
17:24:47.152278 IP (tos 0x0, ttl 64, id 13649, offset 0, flags [none], proto ICMP (1), length 28, bad cksum 0 (->413f)!)
    192.168.100.2 > 95.165.128.1: ICMP echo request, id 44623, seq 776, length 8
17:24:57.154279 IP (tos 0x0, ttl 64, id 56463, offset 0, flags [none], proto ICMP (1), length 28, bad cksum 0 (->9a00)!)
    192.168.100.2 > 95.165.128.1: ICMP echo request, id 44623, seq 777, length 8
17:25:07.155922 IP (tos 0x0, ttl 64, id 37697, offset 0, flags [none], proto ICMP (1), length 28, bad cksum 0 (->e34e)!)
    192.168.100.2 > 95.165.128.1: ICMP echo request, id 44623, seq 778, length 8
17:25:07.157606 IP (tos 0x20, ttl 254, id 37697, offset 0, flags [none], proto ICMP (1), length 28)
    95.165.128.1 > 192.168.100.2: ICMP echo reply, id 44623, seq 778, length 8
17:25:11.218697 IP (tos 0x0, ttl 64, id 64431, offset 0, flags [DF], proto ICMP (1), length 84, bad cksum 0 (->f5a4)!)
    192.168.100.2 > 192.168.100.1: ICMP echo request, id 33766, seq 0, length 64
17:25:11.219159 IP (tos 0x0, ttl 64, id 5506, offset 0, flags [none], proto ICMP (1), length 84)
    192.168.100.1 > 192.168.100.2: ICMP echo reply, id 33766, seq 0, length 64
17:25:12.219773 IP (tos 0x0, ttl 64, id 64514, offset 0, flags [DF], proto ICMP (1), length 84, bad cksum 0 (->f551)!)
    192.168.100.2 > 192.168.100.1: ICMP echo request, id 33766, seq 1, length 64
17:25:12.220202 IP (tos 0x0, ttl 64, id 5567, offset 0, flags [none], proto ICMP (1), length 84)
    192.168.100.1 > 192.168.100.2: ICMP echo reply, id 33766, seq 1, length 64
17:25:13.220858 IP (tos 0x0, ttl 64, id 64601, offset 0, flags [DF], proto ICMP (1), length 84, bad cksum 0 (->f4fa)!)
    192.168.100.2 > 192.168.100.1: ICMP echo request, id 33766, seq 2, length 64
17:25:13.221296 IP (tos 0x0, ttl 64, id 5593, offset 0, flags [none], proto ICMP (1), length 84)
    192.168.100.1 > 192.168.100.2: ICMP echo reply, id 33766, seq 2, length 64
17:25:17.157276 IP (tos 0x0, ttl 64, id 61699, offset 0, flags [none], proto ICMP (1), length 28, bad cksum 0 (->858c)!)
    192.168.100.2 > 95.165.128.1: ICMP echo request, id 44623, seq 779, length 8
17:25:17.158900 IP (tos 0x20, ttl 254, id 61699, offset 0, flags [none], proto ICMP (1), length 28)
    95.165.128.1 > 192.168.100.2: ICMP echo reply, id 44623, seq 779, length 8
17:25:27.159275 IP (tos 0x0, ttl 64, id 53364, offset 0, flags [none], proto ICMP (1), length 28, bad cksum 0 (->a61b)!)
    192.168.100.2 > 95.165.128.1: ICMP echo request, id 44623, seq 780, length 8
17:25:37.161277 IP (tos 0x0, ttl 64, id 4829, offset 0, flags [none], proto ICMP (1), length 28, bad cksum 0 (->63b3)!)
    192.168.100.2 > 95.165.128.1: ICMP echo request, id 44623, seq 781, length 8
17:25:47.163278 IP (tos 0x0, ttl 64, id 34933, offset 0, flags [none], proto ICMP (1), length 28, bad cksum 0 (->ee1a)!)
    192.168.100.2 > 95.165.128.1: ICMP echo request, id 44623, seq 782, length 8
17:25:57.165273 IP (tos 0x0, ttl 64, id 18232, offset 0, flags [none], proto ICMP (1), length 28, bad cksum 0 (->2f58)!)
    192.168.100.2 > 95.165.128.1: ICMP echo request, id 44623, seq 783, length 8

17:26:00.753899 IP (tos 0x0, ttl 64, id 13197, offset 0, flags [none], proto ICMP (1), length 84, bad cksum 0 (->42cb)!)
    192.168.100.2 > 95.165.128.1: ICMP echo request, id 62267, seq 0, length 64
17:26:00.758603 IP (tos 0x20, ttl 254, id 13197, offset 0, flags [none], proto ICMP (1), length 84)
    95.165.128.1 > 192.168.100.2: ICMP echo reply, id 62267, seq 0, length 64
17:26:01.755266 IP (tos 0x0, ttl 64, id 38153, offset 0, flags [none], proto ICMP (1), length 84, bad cksum 0 (->e14e)!)
    192.168.100.2 > 95.165.128.1: ICMP echo request, id 62267, seq 1, length 64
17:26:01.756999 IP (tos 0x20, ttl 254, id 38153, offset 0, flags [none], proto ICMP (1), length 84)
    95.165.128.1 > 192.168.100.2: ICMP echo reply, id 62267, seq 1, length 64
17:26:02.756272 IP (tos 0x0, ttl 64, id 49644, offset 0, flags [none], proto ICMP (1), length 84, bad cksum 0 (->b46b)!)
    192.168.100.2 > 95.165.128.1: ICMP echo request, id 62267, seq 2, length 64
17:26:02.758051 IP (tos 0x20, ttl 254, id 49644, offset 0, flags [none], proto ICMP (1), length 84)
    95.165.128.1 > 192.168.100.2: ICMP echo reply, id 62267, seq 2, length 64
17:26:03.757266 IP (tos 0x0, ttl 64, id 18970, offset 0, flags [none], proto ICMP (1), length 84, bad cksum 0 (->2c3e)!)
    192.168.100.2 > 95.165.128.1: ICMP echo request, id 62267, seq 3, length 64
17:26:03.759090 IP (tos 0x20, ttl 254, id 18970, offset 0, flags [none], proto ICMP (1), length 84)
    95.165.128.1 > 192.168.100.2: ICMP echo reply, id 62267, seq 3, length 64
17:26:04.758269 IP (tos 0x0, ttl 64, id 39956, offset 0, flags [none], proto ICMP (1), length 84, bad cksum 0 (->da43)!)
    192.168.100.2 > 95.165.128.1: ICMP echo request, id 62267, seq 4, length 64
17:26:04.760083 IP (tos 0x20, ttl 254, id 39956, offset 0, flags [none], proto ICMP (1), length 84)
    95.165.128.1 > 192.168.100.2: ICMP echo reply, id 62267, seq 4, length 64
17:26:07.167272 IP (tos 0x0, ttl 64, id 19983, offset 0, flags [none], proto ICMP (1), length 28, bad cksum 0 (->2881)!)
    192.168.100.2 > 95.165.128.1: ICMP echo request, id 44623, seq 784, length 8
17:26:07.169021 IP (tos 0x20, ttl 254, id 19983, offset 0, flags [none], proto ICMP (1), length 28)
    95.165.128.1 > 192.168.100.2: ICMP echo reply, id 44623, seq 784, length 8
17:26:11.265949 IP (tos 0x0, ttl 64, id 10046, offset 0, flags [DF], proto ICMP (1), length 84, bad cksum 0 (->ca16)!)
    192.168.100.2 > 192.168.100.1: ICMP echo request, id 53386, seq 0, length 64
17:26:11.266466 IP (tos 0x0, ttl 64, id 5877, offset 0, flags [none], proto ICMP (1), length 84)
    192.168.100.1 > 192.168.100.2: ICMP echo reply, id 53386, seq 0, length 64
17:26:12.267013 IP (tos 0x0, ttl 64, id 10061, offset 0, flags [DF], proto ICMP (1), length 84, bad cksum 0 (->ca07)!)
    192.168.100.2 > 192.168.100.1: ICMP echo request, id 53386, seq 1, length 64
17:26:12.267436 IP (tos 0x0, ttl 64, id 5911, offset 0, flags [none], proto ICMP (1), length 84)
    192.168.100.1 > 192.168.100.2: ICMP echo reply, id 53386, seq 1, length 64
17:26:13.268090 IP (tos 0x0, ttl 64, id 10073, offset 0, flags [DF], proto ICMP (1), length 84, bad cksum 0 (->c9fb)!)
    192.168.100.2 > 192.168.100.1: ICMP echo request, id 53386, seq 2, length 64
17:26:13.268560 IP (tos 0x0, ttl 64, id 5913, offset 0, flags [none], proto ICMP (1), length 84)
    192.168.100.1 > 192.168.100.2: ICMP echo reply, id 53386, seq 2, length 64
17:26:17.169272 IP (tos 0x0, ttl 64, id 18055, offset 0, flags [none], proto ICMP (1), length 28, bad cksum 0 (->3009)!)
    192.168.100.2 > 95.165.128.1: ICMP echo request, id 44623, seq 785, length 8
17:26:17.170811 IP (tos 0x20, ttl 254, id 18055, offset 0, flags [none], proto ICMP (1), length 28)
    95.165.128.1 > 192.168.100.2: ICMP echo reply, id 44623, seq 785, length 8
                                                1 Reply Last reply Reply Quote 0
                                                • D
                                                  dims last edited by

                                                  @Derelict:

                                                  host route is created to steer all traffic to that address out a specific interface.

                                                  It should not happen silently. A link to this route should appear near appropriate configuration window so that administrator could check if this route interferes with something he set in other places. This is bad design case.

                                                  If the system doesn't smart enogh to run out of the box automatically, it should be configurable. If it is not configurable, it should be smart enough to run automatically.

                                                  You can't do Apple iOS which doesn't run and unable to configure.

                                                  1 Reply Last reply Reply Quote 0
                                                  • johnpoz
                                                    johnpoz LAYER 8 Global Moderator last edited by

                                                    You have bad checksums.. and sending multiple requests and getting back 1 reply..

                                                    You have something borked there… So looks like your monitor is going out BAD... And what your dump is showing as answer is your normal ping..

                                                    An intelligent man is sometimes forced to be drunk to spend time with his fools
                                                    Please don't Chat/PM me for help, unless mod related
                                                    Read https://www.netgate.com/docs/pfsense/book/
                                                    SG-2440 2.4.5p1 | 4x SG-3100 2.4.4p3 | SG-4860 21.05

                                                    1 Reply Last reply Reply Quote 0
                                                    • D
                                                      dims last edited by

                                                      @johnpoz:

                                                      You have bad checksums.. and sending multiple requests and getting back 1 reply..

                                                      Yes. Why can it happen? It never happen with normal ping. At least I was running it for dozens of minutes and saw no case of packet loss.

                                                      And what your dump is showing as answer is your normal ping..

                                                      I did only 5 pings, then I pressed Ctrl-C. All other pings are from someone else, I expect dpinger.

                                                      1 Reply Last reply Reply Quote 0
                                                      • johnpoz
                                                        johnpoz LAYER 8 Global Moderator last edited by

                                                        exactly if the monitors are going out bad and not getting a response then yes it would show it offline since its not getting an answer to its ping..

                                                        An intelligent man is sometimes forced to be drunk to spend time with his fools
                                                        Please don't Chat/PM me for help, unless mod related
                                                        Read https://www.netgate.com/docs/pfsense/book/
                                                        SG-2440 2.4.5p1 | 4x SG-3100 2.4.4p3 | SG-4860 21.05

                                                        1 Reply Last reply Reply Quote 0
                                                        • Derelict
                                                          Derelict LAYER 8 Netgate last edited by

                                                          @dims:

                                                          @Derelict:

                                                          host route is created to steer all traffic to that address out a specific interface.

                                                          It should not happen silently. A link to this route should appear near appropriate configuration window so that administrator could check if this route interferes with something he set in other places. This is bad design case.

                                                          There is no choice. It MUST force that route out the set gateway or it will go out the default gateway. The whole point is to bind the traffic to the specific interface. You can always look at the entire routing table in Diagnostics > Routes which is what a competent administrator would do.

                                                          If the system doesn't smart enogh to run out of the box automatically, it should be configurable. If it is not configurable, it should be smart enough to run automatically.

                                                          You have the special case. That is never going to work out-of-the-box. It will require configuration and tuning to make multi-wan work how you need it to work.

                                                          You can't do Apple iOS which doesn't run and unable to configure.

                                                          I have no idea what that means.
                                                          Diagnostics > Routes

                                                          It is smart enough to run automatically. You have the special case/requirements.

                                                          Chattanooga, Tennessee, USA
                                                          The pfSense Book is free of charge!
                                                          DO NOT set a source port in a port forward or firewall rule unless you KNOW you need it!
                                                          Do Not Chat For Help! NO_WAN_EGRESS(TM)

                                                          1 Reply Last reply Reply Quote 0
                                                          • Derelict
                                                            Derelict LAYER 8 Netgate last edited by

                                                            @johnpoz:

                                                            You have bad checksums.. and sending multiple requests and getting back 1 reply..

                                                            You have something borked there… So looks like your monitor is going out BAD... And what your dump is showing as answer is your normal ping..

                                                            The bad checksums are probably the result of checksum offloading. The OS doesn't calculate the checksums so they are 0 when tcpdump sees it. They are calculated and inserted by the NIC hardware. They are only being displayed because of his tcpdump flags.

                                                            
17:24:27.148884 IP (tos 0x0, ttl 64, id 4107, offset 0, flags [none], proto ICMP (1), length 28, bad cksum 0 (->6685)!)
    192.168.100.2 > 95.165.128.1: ICMP echo request, id 44623, seq 774, length 8
17:24:37.150298 IP (tos 0x0, ttl 64, id 4605, offset 0, flags [none], proto ICMP (1), length 28, bad cksum 0 (->6493)!)
    192.168.100.2 > 95.165.128.1: ICMP echo request, id 44623, seq 775, length 8
17:24:37.151978 IP (tos 0x20, ttl 254, id 4605, offset 0, flags [none], proto ICMP (1), length 28)
    95.165.128.1 > 192.168.100.2: ICMP echo reply, id 44623, seq 775, length 8
17:24:47.152278 IP (tos 0x0, ttl 64, id 13649, offset 0, flags [none], proto ICMP (1), length 28, bad cksum 0 (->413f)!)
    192.168.100.2 > 95.165.128.1: ICMP echo request, id 44623, seq 776, length 8
17:24:57.154279 IP (tos 0x0, ttl 64, id 56463, offset 0, flags [none], proto ICMP (1), length 28, bad cksum 0 (->9a00)!)
    192.168.100.2 > 95.165.128.1: ICMP echo request, id 44623, seq 777, length 8
17:25:07.155922 IP (tos 0x0, ttl 64, id 37697, offset 0, flags [none], proto ICMP (1), length 28, bad cksum 0 (->e34e)!)
    192.168.100.2 > 95.165.128.1: ICMP echo request, id 44623, seq 778, length 8
17:25:07.157606 IP (tos 0x20, ttl 254, id 37697, offset 0, flags [none], proto ICMP (1), length 28)
    95.165.128.1 > 192.168.100.2: ICMP echo reply, id 44623, seq 778, length 8

                                                            You are not getting a LOT of responses. Of course the GW is marked as down.

                                                            The default ping interval is two per second. You are doing one every 10 seconds. What else have you changed? How about you post the settings for that gateway - particularly the advanced settings which you have obviously changed from the defaults.

                                                            Chattanooga, Tennessee, USA
                                                            The pfSense Book is free of charge!
                                                            DO NOT set a source port in a port forward or firewall rule unless you KNOW you need it!
                                                            Do Not Chat For Help! NO_WAN_EGRESS(TM)

                                                            1 Reply Last reply Reply Quote 0
                                                            • D
                                                              dims last edited by

                                                              @Derelict:

                                                              You are not getting a LOT of responses.

                                                              Why? How does dpinger able to achieve this?

                                                              The default ping interval is two per second. You are doing one every 10 seconds. What else have you changed?

                                                              I played with intervals because I was thinking target host has some sort of flood protection. Neither setting helped, including default one.

                                                              How about you post the settings for that gateway - particularly the advanced settings which you have obviously changed from the defaults.

                                                              I would not change defaults if they worked.

                                                              You are welcome:

                                                              1 Reply Last reply Reply Quote 0
                                                              • D
                                                                dims last edited by

                                                                @Derelict:

                                                                It is smart enough to run automatically. You have the special case/requirements.

                                                                Sure, this is what I am saying: pfSense does not work in "special case" of having 3 WANs round robbin.

                                                                1 Reply Last reply Reply Quote 0
                                                                • D
                                                                  dims last edited by

                                                                  @johnpoz:

                                                                  exactly if the monitors are going out bad and not getting a response

                                                                  Why this can happen? Taking into consideration, that normal ping works?

                                                                  1 Reply Last reply Reply Quote 0
                                                                  • Derelict
                                                                    Derelict LAYER 8 Netgate last edited by

                                                                    Only your ISP can tell you why sent echo requests are not responded to.

                                                                    Set all of those settings back to the default. Does it work?

                                                                    If not, take a quick packet capture for posting here then try setting the data payload to 2. Does it work?

                                                                    If not, take a quick packet capture for posting here then try setting the data payload to 64. Does it work?

                                                                    If not, take a quick packet capture for posting here then post all of the above.

                                                                    Chattanooga, Tennessee, USA
                                                                    The pfSense Book is free of charge!
                                                                    DO NOT set a source port in a port forward or firewall rule unless you KNOW you need it!
                                                                    Do Not Chat For Help! NO_WAN_EGRESS(TM)

                                                                    1 Reply Last reply Reply Quote 0
                                                                    • D
                                                                      dims last edited by

                                                                      @Derelict:

                                                                      Only your ISP can tell you why sent echo requests are not responded to.

                                                                      How provider can technically distinguish pings from ping command and from dpinger?

                                                                      1 Reply Last reply Reply Quote 0
                                                                      • Derelict
                                                                        Derelict LAYER 8 Netgate last edited by

                                                                        The payload size of a normal ping from the ping command is 56 bytes. The payload size of a dpinger ping is 0 by default.

                                                                        Hence why I asked you to do what I did.

                                                                        It's pretty rare but some devices freak out with the 0-byte payload even though it's completely legal.

                                                                        Chattanooga, Tennessee, USA
                                                                        The pfSense Book is free of charge!
                                                                        DO NOT set a source port in a port forward or firewall rule unless you KNOW you need it!
                                                                        Do Not Chat For Help! NO_WAN_EGRESS(TM)

                                                                        1 Reply Last reply Reply Quote 0
                                                                        • D
                                                                          dims last edited by

                                                                          Setting payload to 56 immediately made monitor think gateway is up, thank you!

                                                                          I have set it to 56 in other monitors, and picture also became better: now zeros in RTT and RTTsd columns!

                                                                          1 Reply Last reply Reply Quote 0
                                                                          • D
                                                                            dims last edited by

                                                                            No, last statement was wrong: after some time these columns got some positive values.

                                                                            1 Reply Last reply Reply Quote 0
                                                                            • P
                                                                              pwood999 last edited by

                                                                              I had flase GW down reports when I first configured multi-wan.  Then changed Monitor IP's from google to OpenDNS, and problems went away.

                                                                              Also this gives RTT & RTTsd figures which are probably more realistic for internet connectivity, rather than using the next-hop provider IP.

                                                                              DSL GW = 7.5mS & 0.2mS
                                                                              Cable GW = 15mS & 3.5mS

                                                                              Cable is always longer due to the way Docsis works !!

                                                                              1 Reply Last reply Reply Quote 0
                                                                              • G
                                                                                Gektor last edited by Gektor

                                                                                I am have same issue and same scheme as in author of topic with gateway group:
                                                                                1 gateway is tier 2 (connected directly to the internet provider, used as backup)
                                                                                2 gateway is tier 1 (connected to external router 1 with VLAN2 by DHCP)
                                                                                3 gateway is tier 1 (connected to external router 2 with VLAN3 by DHCP)
                                                                                and last 3 gateway can't resolve "UP" state of gateway WAN when external router 2 changes WAN IP. Monitoring IP on 2 gateway is 8.8.4.4, on 3 gateway 8.8.8.8.
                                                                                Payload and changing monitoring IP did not help at all, when i set "Trigger Level' as "High Latency" it works a little better, but when i set it to "Packet Loss" and in time when router 2 on 3 gateway changes WAN ip - on pfSense in100% case it will marks 3 gateway as offline forever, but in Diagnostics -> Ping i can ping any address from 3 gateway without any problems. If i will go to System -> Routing and save and apply any gateway without changes - 3 gateway will back to Online state till router 2 changes WAN IP address.
                                                                                2 gateway have same type router as on 3 gateway, but when it's change IP - pfSense make it Offline for few minutes and then make it back to Online state in any types of Trigger Level.
                                                                                It seems that pfSense buggy with dpinger on some scenarios.

                                                                                1 Reply Last reply Reply Quote 0
                                                                                • G
                                                                                  Gektor last edited by Gektor

                                                                                  It's more interesting, i switch Trigger Level to High Latency, and some time later pfSense himself switch it to Packet Loss! I didn't understand, why it happens.

                                                                                  And for sure it's dpinger bugs related, case i have checked "Disable Gateway Monitoring Action" and then made reconnect on router 2 (3 gateway) and in "Gateway status" i get on 3 gateway "Danger, Packetloss: 100%" on 3 gateway, i have check - traffic still goes through 3 gateway (router 2) without any problems, but dpinger thinks that it's dead for sure forever, till i make "save and apply" in any gateway settings.

                                                                                  I didn't now how to make monitoring work in pfSense. :(

                                                                                  1 Reply Last reply Reply Quote 0
                                                                                  • First post
                                                                                    Last post

                                                                                  Products

                                                                                  • Platform Overview
                                                                                  • TNSR
                                                                                  • pfSense
                                                                                  • Appliances

                                                                                  Services

                                                                                  • Training
                                                                                  • Professional Services

                                                                                  Support

                                                                                  • Subscription Plans
                                                                                  • Contact Support
                                                                                  • Product Lifecycle
                                                                                  • Documentation

                                                                                  News

                                                                                  • Media Coverage
                                                                                  • Press
                                                                                  • Events

                                                                                  Resources

                                                                                  • Blog
                                                                                  • FAQ
                                                                                  • Find a Partner
                                                                                  • Resource Library
                                                                                  • Security Information

                                                                                  Company

                                                                                  • About Us
                                                                                  • Careers
                                                                                  • Partners
                                                                                  • Contact Us
                                                                                  • Legal
                                                                                  Our Mission

                                                                                  We provide leading-edge network security at a fair price - regardless of organizational size or network sophistication. We believe that an open-source security model offers disruptive pricing along with the agility required to quickly address emerging threats.

                                                                                  Subscribe to our Newsletter

                                                                                  Product information, software announcements, and special offers. See our newsletter archive to sign up for future newsletters and to read past announcements.

                                                                                  © 2021 Rubicon Communications, LLC | Privacy Policy