Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Multi WAN on OffLine State

    Scheduled Pinned Locked Moved Routing and Multi WAN
    17 Posts 5 Posters 3.5k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • H
      heper
      last edited by

      start by providing a detailed schematic (no ascii art)

      also provide logs/screenshots of whatever useful things you might have. Without good information, its impossible to get useful feedback

      1 Reply Last reply Reply Quote 0
      • V
        v.smaldino
        last edited by

        Similar problem here; i'll try to illustrate my case, hoping this can help.

        The configuration is relatively simple: 1 LAN - 3 WAN (grouped)

        When one of the 3 WANs goes down (due to latency or packet loss), sometimes, the relative GW remains marked OFFLINE even when the WAN comes back up and the monitor ip is pingable from the pfsense shell.

        If i edit and save (without any modification) any of the 3 GWs, the state of the GW immediately changes to ONLINE.

        The problem appears randomly and i'm not able to replicate it manually.

        My system is 2.3.1 (see attechment)

        Few words on the log (see attachment)
        00:06 two WANs and relatively GWs go down (WANSEGR1GW & GW_ITG)
        00:46 WANSEGR1GW automatically comes back ONLINE (GW_ITG is still OFFLINE)
        01:11 - 01:13 another WANSEGR1GW failure, automatically recovered

        at about 07:45 i log on pfsense box and manually ping the GW_ITG monitor IP, realizing that the wan connection is ok

        at 07:50 i edit and save, without any modification, the GW_FIBRAITIS and immediately GW_ITG goes ONLINE.

        Thanks for any help.

        versione.png
        versione.png_thumb
        log.png
        log.png_thumb

        1 Reply Last reply Reply Quote 0
        • H
          heper
          last edited by

          you appear to have 3 wans within the same subnet. (unless i'm reading this wrong)

          this won't work

          @v.smaldino:

          Similar problem here; i'll try to illustrate my case, hoping this can help.

          The configuration is relatively simple: 1 LAN - 3 WAN (grouped)

          When one of the 3 WANs goes down (due to latency or packet loss), sometimes, the relative GW remains marked OFFLINE even when the WAN comes back up and the monitor ip is pingable from the pfsense shell.

          If i edit and save (without any modification) any of the 3 GWs, the state of the GW immediately changes to ONLINE.

          The problem appears randomly and i'm not able to replicate it manually.

          My system is 2.3.1 (see attechment)

          Few words on the log (see attachment)
          00:06 two WANs and relatively GWs go down (WANSEGR1GW & GW_ITG)
          00:46 WANSEGR1GW automatically comes back ONLINE (GW_ITG is still OFFLINE)
          01:11 - 01:13 another WANSEGR1GW failure, automatically recovered

          at about 07:45 i log on pfsense box and manually ping the GW_ITG monitor IP, realizing that the wan connection is ok

          at 07:50 i edit and save, without any modification, the GW_FIBRAITIS and immediately GW_ITG goes ONLINE.

          Thanks for any help.

          1 Reply Last reply Reply Quote 0
          • V
            v.smaldino
            last edited by

            No heper, the configuration is right and it works fine because the mask of the 3 GWs is /26 and not the standard /24  ;)
            The problem is not there  :(

            192.168.1.34/26 GW 192.168.1.1 aka GW_ITG1 (range 192.168.1.1-62)
            192.168.1.68/26 GW 192.168.1.111 aka WANSEGR1GW (range 192.168.1.65-126)
            192.168.1.194/26 GW 192.168.1.211 aka GW_FIBRAITIS (range 192.168.1.193-254)

            Addendum: The 3 WANs are on 3 different NICs

            In any case pfSense WebGUI avoids that kind of error because it checks the overlap of the subnet GWs  :)

            Thanks for your help

            1 Reply Last reply Reply Quote 0
            • H
              heper
              last edited by

              have you tried changing monitor ip's ?

              1 Reply Last reply Reply Quote 0
              • DerelictD
                Derelict LAYER 8 Netgate
                last edited by

                You should probably at least upgrade to current release code level.

                Post some diagnostics from the WAN that is showing down when it should be showing up. Things like manual pings from the firewall to the monitor IP address, quality graphs, etc.

                For the ping do something like ping -i.5 -c 60 monitor.ip.address. That will be basically what dpinger does.

                Chattanooga, Tennessee, USA
                A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                Do Not Chat For Help! NO_WAN_EGRESS(TM)

                1 Reply Last reply Reply Quote 0
                • V
                  v.smaldino
                  last edited by

                  Until few days ago, every time i changed the monitor ips, but the problem doesn't change.

                  Today it happened two times :-( , i searched the logs for something useful, but nothing!

                  I attach the ping on the monitor_ip of the WAN and the gateways logs.

                  In my opinion the problem is related with dpinger; i'm not able to check what happend when it triggerred the alarm, but i'm SURE that when i successfully pinged the monitor ip, the WAN is marked as OFFLINE.

                  I don't know how dpinger integrates with the system, but checking dpingers PIDs, i observed that dpinger restarts after saving any GW config and then all goes ok.

                  In order to debug the problem:

                  1. Is there any way to check (from the shell) what dpinger "thinks" about the status of its monitor_ip at any time?
                  2. Is there any way to manually restart (from the shell) the instance of dpinger relatively to the WAN marked offline?

                  Thanks to all
                  V

                  ping.png
                  ping.png_thumb
                  log1.png
                  log1.png_thumb
                  lo2.png
                  lo2.png_thumb

                  1 Reply Last reply Reply Quote 0
                  • H
                    heper
                    last edited by

                    fetchin straws here but, have you tried changing dpingers data payload?

                    1 Reply Last reply Reply Quote 0
                    • V
                      v.smaldino
                      last edited by

                      Just changed all payloads from 0 to 56. Let's see  :)

                      Thanks

                      1 Reply Last reply Reply Quote 0
                      • G
                        Gildresh
                        last edited by

                        Where can you change the payload?

                        1 Reply Last reply Reply Quote 0
                        • V
                          v.smaldino
                          last edited by

                          @Gildresh
                          in System->Routing->Edit your GW->Advanced->Data Payload.

                          Read below  ;)

                          1 Reply Last reply Reply Quote 0
                          • V
                            v.smaldino
                            last edited by

                            I just ended a cycle of tests about this matter. I share with you what happened:
                            I put in service a Zeroshell box acting as a failover/load balancer on a test LAN using the same GWs as pfSense box, on the same ethernet infrastructure and with the same monitor ip and parameters.

                            1. Changing data payloads doesn't solve the problem
                            2. When for some reason any of the GWs went down, both pfSense and Zeroshell marked it as Offline
                            3. When the GW came up again, ZS recognized the event and marked it Online, PFS kept it Offline until dpinger restarted

                            Restarting dpinger seems to be the only solution :(

                            Attachments:

                            • Logs of pfSense boxes
                            • Log of Zeroshell box

                            cinque.png
                            cinque.png_thumb
                            quattro.png
                            quattro.png_thumb
                            tre.png
                            tre.png_thumb

                            1 Reply Last reply Reply Quote 0
                            • A
                              Alesk13Fr
                              last edited by

                              Hello,

                              I've done the same things, I've try to compare it with ZeroShell too and OPNSense, and see the same things, with PfSense Gateway never goes back in Online mode when link goes back, with OPNSense, problem is present but only 25% of time (when PfSense go offline, it never go back, when OPNSense go back, 1 of 4 times gateway stay down, and I can use a cron task with to launch back /usr/local/sbin/apinger -c /var/etc/apinger.conf, but this don't exist on PfSense, I hope I'm wrong ?), with ZeroShell I've 100% success.

                              I'm pretty sure dpinger or apinger aren't going to test anymore the gateway and stay stuck to OffLine mode …
                              It would be really cool if somebody can help in first time, to give us a way to restart dpinger via cron if gateway goes down, and in second time to correct this things, for me it's just a little bug, to stop this behaviour ...

                              I'm using PfSense since 6 years now, and this problem is recuring since the begining ... I Hope this time my report will be understand and earn !

                              Thanks in advance, I love so much PfSense since the begining, but this things is driving me nuts, I'm tired to go don't do any changes on each gateway down and apply no changes just to get dpinger restarting, this is done 4 to 25 times a day ...

                              Best regards,
                              Alex.

                              1 Reply Last reply Reply Quote 0
                              • V
                                v.smaldino
                                last edited by

                                Just an addendum.

                                I'm not able to manually replicate the problem. Every time i shutdown and restart or i disconnect and reconnect wan or lan cable of any of my adsl modems, pfsense works correctly, very strange  :-\

                                Thanks all

                                V

                                1 Reply Last reply Reply Quote 0
                                • First post
                                  Last post
                                Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.