Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Multi WAN on OffLine State

    Scheduled Pinned Locked Moved Routing and Multi WAN
    17 Posts 5 Posters 3.5k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • A
      Alesk13Fr
      last edited by

      Hello,
      I'm using Multi WAN configuration, four Gateways, all are wired, 2 are wireless WISP connection, but Wireless access pooint are dedicated to this connection, when connected they are connected to the FAI Wireless, and got internal IP, access point give internal IP to PfSense, like this :

      FAI –-----------------> Wired Connection ---> PfSense WAN 1
                                                                          Public IP Address

      FAI -------------------> Wired Connection ---> PfSense WAN 2
                                                                          Public IP Address

      FAI DHCP ---> FAI --------------> Wireless Connection ---> Access Point ----> Wired Connection ---> PfSense WAN 3
      Public IP Address                              192.168.0.30                                                  NAT                  192.168.50.1

      FAI DHCP ---> FAI --------------> Wireless Connection ---> Access Point ----> Wired Connection ---> PfSense WAN 4
      Public Address                                  192.168.1.20                                                  NAT                  192.168.60.1

      Regulary my WAN 3 and 4 are going in OffLine state, If I don't do anything OffLine is always the current state ! If I'm just editing this two gateway 3 or 4, or the two, and without changing anything just saving already stored settings, and applying the no changes my Gateway goes back OnLine.

      I'm wondering why dpinger is failing to renew state, or check again OffLine Gateways.

      If you have any suggestiont o solve this I'll be happty to earn you !

      My idea for bypassing this so anoying problem is to make bash script to check if one gateway is down, and if it's down, juste reload the dpinger check. Do you know how I can do to do this ?

      Thanks in advance for your help !

      Best regards,
      Alex.

      1 Reply Last reply Reply Quote 0
      • A
        Alesk13Fr
        last edited by

        Hello,

        I'm surprised to not see anybody have at least one idea on how achieve this ?!

        Feel free to give me your opinion !

        Best regards,
        Alex.

        1 Reply Last reply Reply Quote 0
        • G
          Gildresh
          last edited by

          I am having a similar issue and have not gotten any feedback. I will likely begin looking at other options to replace pfSence with.

          1 Reply Last reply Reply Quote 0
          • H
            heper
            last edited by

            start by providing a detailed schematic (no ascii art)

            also provide logs/screenshots of whatever useful things you might have. Without good information, its impossible to get useful feedback

            1 Reply Last reply Reply Quote 0
            • V
              v.smaldino
              last edited by

              Similar problem here; i'll try to illustrate my case, hoping this can help.

              The configuration is relatively simple: 1 LAN - 3 WAN (grouped)

              When one of the 3 WANs goes down (due to latency or packet loss), sometimes, the relative GW remains marked OFFLINE even when the WAN comes back up and the monitor ip is pingable from the pfsense shell.

              If i edit and save (without any modification) any of the 3 GWs, the state of the GW immediately changes to ONLINE.

              The problem appears randomly and i'm not able to replicate it manually.

              My system is 2.3.1 (see attechment)

              Few words on the log (see attachment)
              00:06 two WANs and relatively GWs go down (WANSEGR1GW & GW_ITG)
              00:46 WANSEGR1GW automatically comes back ONLINE (GW_ITG is still OFFLINE)
              01:11 - 01:13 another WANSEGR1GW failure, automatically recovered

              at about 07:45 i log on pfsense box and manually ping the GW_ITG monitor IP, realizing that the wan connection is ok

              at 07:50 i edit and save, without any modification, the GW_FIBRAITIS and immediately GW_ITG goes ONLINE.

              Thanks for any help.

              versione.png
              versione.png_thumb
              log.png
              log.png_thumb

              1 Reply Last reply Reply Quote 0
              • H
                heper
                last edited by

                you appear to have 3 wans within the same subnet. (unless i'm reading this wrong)

                this won't work

                @v.smaldino:

                Similar problem here; i'll try to illustrate my case, hoping this can help.

                The configuration is relatively simple: 1 LAN - 3 WAN (grouped)

                When one of the 3 WANs goes down (due to latency or packet loss), sometimes, the relative GW remains marked OFFLINE even when the WAN comes back up and the monitor ip is pingable from the pfsense shell.

                If i edit and save (without any modification) any of the 3 GWs, the state of the GW immediately changes to ONLINE.

                The problem appears randomly and i'm not able to replicate it manually.

                My system is 2.3.1 (see attechment)

                Few words on the log (see attachment)
                00:06 two WANs and relatively GWs go down (WANSEGR1GW & GW_ITG)
                00:46 WANSEGR1GW automatically comes back ONLINE (GW_ITG is still OFFLINE)
                01:11 - 01:13 another WANSEGR1GW failure, automatically recovered

                at about 07:45 i log on pfsense box and manually ping the GW_ITG monitor IP, realizing that the wan connection is ok

                at 07:50 i edit and save, without any modification, the GW_FIBRAITIS and immediately GW_ITG goes ONLINE.

                Thanks for any help.

                1 Reply Last reply Reply Quote 0
                • V
                  v.smaldino
                  last edited by

                  No heper, the configuration is right and it works fine because the mask of the 3 GWs is /26 and not the standard /24  ;)
                  The problem is not there  :(

                  192.168.1.34/26 GW 192.168.1.1 aka GW_ITG1 (range 192.168.1.1-62)
                  192.168.1.68/26 GW 192.168.1.111 aka WANSEGR1GW (range 192.168.1.65-126)
                  192.168.1.194/26 GW 192.168.1.211 aka GW_FIBRAITIS (range 192.168.1.193-254)

                  Addendum: The 3 WANs are on 3 different NICs

                  In any case pfSense WebGUI avoids that kind of error because it checks the overlap of the subnet GWs  :)

                  Thanks for your help

                  1 Reply Last reply Reply Quote 0
                  • H
                    heper
                    last edited by

                    have you tried changing monitor ip's ?

                    1 Reply Last reply Reply Quote 0
                    • DerelictD
                      Derelict LAYER 8 Netgate
                      last edited by

                      You should probably at least upgrade to current release code level.

                      Post some diagnostics from the WAN that is showing down when it should be showing up. Things like manual pings from the firewall to the monitor IP address, quality graphs, etc.

                      For the ping do something like ping -i.5 -c 60 monitor.ip.address. That will be basically what dpinger does.

                      Chattanooga, Tennessee, USA
                      A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                      DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                      Do Not Chat For Help! NO_WAN_EGRESS(TM)

                      1 Reply Last reply Reply Quote 0
                      • V
                        v.smaldino
                        last edited by

                        Until few days ago, every time i changed the monitor ips, but the problem doesn't change.

                        Today it happened two times :-( , i searched the logs for something useful, but nothing!

                        I attach the ping on the monitor_ip of the WAN and the gateways logs.

                        In my opinion the problem is related with dpinger; i'm not able to check what happend when it triggerred the alarm, but i'm SURE that when i successfully pinged the monitor ip, the WAN is marked as OFFLINE.

                        I don't know how dpinger integrates with the system, but checking dpingers PIDs, i observed that dpinger restarts after saving any GW config and then all goes ok.

                        In order to debug the problem:

                        1. Is there any way to check (from the shell) what dpinger "thinks" about the status of its monitor_ip at any time?
                        2. Is there any way to manually restart (from the shell) the instance of dpinger relatively to the WAN marked offline?

                        Thanks to all
                        V

                        ping.png
                        ping.png_thumb
                        log1.png
                        log1.png_thumb
                        lo2.png
                        lo2.png_thumb

                        1 Reply Last reply Reply Quote 0
                        • H
                          heper
                          last edited by

                          fetchin straws here but, have you tried changing dpingers data payload?

                          1 Reply Last reply Reply Quote 0
                          • V
                            v.smaldino
                            last edited by

                            Just changed all payloads from 0 to 56. Let's see  :)

                            Thanks

                            1 Reply Last reply Reply Quote 0
                            • G
                              Gildresh
                              last edited by

                              Where can you change the payload?

                              1 Reply Last reply Reply Quote 0
                              • V
                                v.smaldino
                                last edited by

                                @Gildresh
                                in System->Routing->Edit your GW->Advanced->Data Payload.

                                Read below  ;)

                                1 Reply Last reply Reply Quote 0
                                • V
                                  v.smaldino
                                  last edited by

                                  I just ended a cycle of tests about this matter. I share with you what happened:
                                  I put in service a Zeroshell box acting as a failover/load balancer on a test LAN using the same GWs as pfSense box, on the same ethernet infrastructure and with the same monitor ip and parameters.

                                  1. Changing data payloads doesn't solve the problem
                                  2. When for some reason any of the GWs went down, both pfSense and Zeroshell marked it as Offline
                                  3. When the GW came up again, ZS recognized the event and marked it Online, PFS kept it Offline until dpinger restarted

                                  Restarting dpinger seems to be the only solution :(

                                  Attachments:

                                  • Logs of pfSense boxes
                                  • Log of Zeroshell box

                                  cinque.png
                                  cinque.png_thumb
                                  quattro.png
                                  quattro.png_thumb
                                  tre.png
                                  tre.png_thumb

                                  1 Reply Last reply Reply Quote 0
                                  • A
                                    Alesk13Fr
                                    last edited by

                                    Hello,

                                    I've done the same things, I've try to compare it with ZeroShell too and OPNSense, and see the same things, with PfSense Gateway never goes back in Online mode when link goes back, with OPNSense, problem is present but only 25% of time (when PfSense go offline, it never go back, when OPNSense go back, 1 of 4 times gateway stay down, and I can use a cron task with to launch back /usr/local/sbin/apinger -c /var/etc/apinger.conf, but this don't exist on PfSense, I hope I'm wrong ?), with ZeroShell I've 100% success.

                                    I'm pretty sure dpinger or apinger aren't going to test anymore the gateway and stay stuck to OffLine mode …
                                    It would be really cool if somebody can help in first time, to give us a way to restart dpinger via cron if gateway goes down, and in second time to correct this things, for me it's just a little bug, to stop this behaviour ...

                                    I'm using PfSense since 6 years now, and this problem is recuring since the begining ... I Hope this time my report will be understand and earn !

                                    Thanks in advance, I love so much PfSense since the begining, but this things is driving me nuts, I'm tired to go don't do any changes on each gateway down and apply no changes just to get dpinger restarting, this is done 4 to 25 times a day ...

                                    Best regards,
                                    Alex.

                                    1 Reply Last reply Reply Quote 0
                                    • V
                                      v.smaldino
                                      last edited by

                                      Just an addendum.

                                      I'm not able to manually replicate the problem. Every time i shutdown and restart or i disconnect and reconnect wan or lan cable of any of my adsl modems, pfsense works correctly, very strange  :-\

                                      Thanks all

                                      V

                                      1 Reply Last reply Reply Quote 0
                                      • First post
                                        Last post
                                      Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.