Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Using second WAN speeds up first WAN ?????

    Scheduled Pinned Locked Moved General pfSense Questions
    11 Posts 3 Posters 2.5k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • H
      hackin8
      last edited by

      I have a very strange problem. I believe it started after I installed pfSense but it could be a coincidence - unfortunately I cannot go back as the old dual WAN router is now dead.

      Config:
      WAN1  80/20 connection. (PPPoE fibre from BT)
      WAN2  10/1 ADSL2 connection (PPPoE copper from Nildram)
      LAN    Simple setup - 6 PC's
      pfS      Running on Compaq SFF with three NIC's

      Settings:
      Usually use failover but have also tried load balancing
      No other packages running
      I am running a web server on one PC - with forwarding rules through both WANs. (Uses HFS but have also tried Apache)

      Everything works well in terms of routing etc.

      Problem:
      My daughter (ADSL2 12/1) in Australia regularly downloads from my server - until recently with no problems.
      She used to get 500KB/s+ through WAN1 or 90KB/s through WAN2 - both stable
      Now she usually gets 10-40KB/s (continuously variable) through WAN1 but still 90KB/s through WAN2
      BUT
      If she starts DL on WAN1 (getting 10-40KB/s)
      If she then starts DL on WAN2 (getting 90KB/s)
      WAN1 DL then increases to higher speed - anywhere from 200KB/s to 500KB/s - but with stable speed
      If she then stops DL on WAN2
      After about 30 seconds WAN1 again drops down to variable 10-40KB/s

      This also shows if trying to use Skype - too slow to work on WAN1 unless she starts a DL on WAN2!
      She can Skype to many other people in UK (and globally) without any problem

      For other users (In UK) both WAN's work perfectly at high speed independently - DL and Skype

      Any thoughts / suggestions?  Any ideas on what to change or how to troubleshoot?

      Thanks

      1 Reply Last reply Reply Quote 0
      • H
        hackin8
        last edited by

        I have a theory:

        When connected to wan1 it seems some of the packets may be going out through wan2, as the routing via wan2 has lower latency. This creates a problem with packets out of sequence.

        When starting a dl on wan2 it fills the line (Only a relatively slow adsl2) to the latency increases

        Then traffic through wan1 ONLY returns through wan1 and packets are in order - and speed increases?

        Does that make sense?  I plan to try out the theory by disconnecting wan2 when next trying.  If it works how can I resolve the problem using pfSense?

        1 Reply Last reply Reply Quote 0
        • marcellocM
          marcelloc
          last edited by

          How did you configured your gateways? Is there a default gateway configured? On LAN rules did you forced a gateway or a load balance?

          Treinamentos de Elite: http://sys-squad.com

          Help a community developer! ;D

          1 Reply Last reply Reply Quote 0
          • H
            hackin8
            last edited by

            WAN1 has default gateway selected.  I have tried with "allow default gateway switching" enabled and disabled - no difference.

            On Firewall-Lan-Rules I normally use "wan1failover" as gateway. (WAN1 on tier1, wan2 on tier2) But I have also tried WAN1 (Only) and load balanced between wan1 and wan2 (Both tier1 - tried with different and same weightings).
            Again - no difference between them all.

            I haven't tried "use sticky connections" as I believe they relate to connections for incoming only - and I further gather they may not work n 2.0.1, which I am using.

            Any help appreciated

            1 Reply Last reply Reply Quote 0
            • H
              hackin8
              last edited by

              Well, now I am totally puzzled. I tried with WAN2 disabled - hoping this would prove a point.

              See attached bandwidth graphs - read from right to left

              I disabled the WAN2 and she then tried dl  - it started slow but then increased and I thought it was ok. Then it dropped and again increased, repeating the "sawtooth" pattern 4 times. Then, unfortunately, it dropped and stayed low and variable.

              After running for some time I then re-enabled the second interface. Lo and behold - the speed increased immediately.  I then disabled the second interface - which reset the firewall so both stopped. Finally tried just WAN2 - again it ran more or less steadily - maxing out the interface.

              If it was just the "sawtooth" variation I could believe it was some sort of throttling - but that still doesn't explain why the speed increases when wan2 starts dl.

              Any suggestions on what I can try to further investigate?

              speed.png
              speed.png_thumb

              1 Reply Last reply Reply Quote 0
              • H
                hackin8
                last edited by

                Just a last appeal - please.

                Ignoring (for the moment) all the details, how is it possible that DL through WAN2 increase DL speed through WAN1?
                Removing WAN2 completely has no effect.
                The only way to get fast dl through WAN1 is to start a DL through WAN2.

                Any thoughts or suggestions as to how to troubleshoot would be much appreciated.

                1 Reply Last reply Reply Quote 0
                • W
                  wallabybob
                  last edited by

                  @hackin8:

                  Ignoring (for the moment) all the details, how is it possible that DL through WAN2 increase DL speed through WAN1?
                  Removing WAN2 completely has no effect.
                  The only way to get fast dl through WAN1 is to start a DL through WAN2.

                  OK, ignoring the details of your "problem" and the details required to make my suggestion happen, suppose the downloader's ISP has a web cache. Suppose the paths to WAN1 and WAN2 have significantly different round trip times. Downloading from WAN2 might prefill the cache so a download from WAN1 is handled from the cache rather than the real WAN1.

                  @hackin8:

                  Any thoughts or suggestions as to how to troubleshoot would be much appreciated.

                  Comparing traceroute to WAN1 and WAN2 might be interesting. If I recall correctly, a major submarine cable connecting Western Australia to the rest of the world was broken a few months ago. That might have affected the route to WAN1 but not WAN2 (or vice versa)

                  1 Reply Last reply Reply Quote 0
                  • H
                    hackin8
                    last edited by

                    Thanks for taking the time and making suggestions.

                    Unfortunately first suggestion would only be true if downloading same thing from both WANs - whereas WAN1 speeds up regardless of what is DL on WAN2.  ANY DL on WAN2 will speed up WAN1.

                    Second Idea is a good one - I have actually asked them to try MTR on both routes to compare - waiting for the result. I am hoping this might help - but not sure it would explain why WAN2 DL would immediately speed up WAN1 DL. (And WAN1 slows down again if WAN2 stops)

                    Thanks again for suggestions - I had already run out of ideas / theories..

                    1 Reply Last reply Reply Quote 0
                    • H
                      hackin8
                      last edited by

                      Thanks to all taking time to make suggestions.  Unfortunately still no further on.

                      As I have no way of knowing whether the problem actually relates to pfS or something entirely different - any suggestions for a more general network forum that I could ask for suggestions of how to troubleshoot / find the root cause of the problem?

                      Thanks again

                      1 Reply Last reply Reply Quote 0
                      • W
                        wallabybob
                        last edited by

                        Did you get traceroutes to both IPs? Care to post them? (blank out source IP if you wish)

                        I suspect what you have reported is a problem in the sense of "unexplained behaviour" rather than a problem in the sense of something doing something it should not do.

                        1 Reply Last reply Reply Quote 0
                        • H
                          hackin8
                          last edited by

                          Solved - I think!!

                          It appears that the problem has been solved - although I will wait a few days before final confirmation.

                          The solution - if it was the cause - was for the ISP in Australia to put a check mark (i.e. enable) in the "force MTU" option in the exchange.

                          I have no idea how / why this would cause the strange problem - but so far it appears to have fixed it.

                          I will mark the thread "Solved" once I have verified over a few days.

                          1 Reply Last reply Reply Quote 0
                          • First post
                            Last post
                          Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.