Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Uploading files from any machine KILLS latency for everyone on the network

    Scheduled Pinned Locked Moved General pfSense Questions
    11 Posts 5 Posters 3.5k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • A
      azcire
      last edited by

      We've had a mysterious issue for a long time now that I was finally able to replicate.  On ANY machine, when I upload files on FileZilla to any FTP server, or upload a big file to senduit.com or similar (basically, any activity that maxes out our upload), the entire network's latency increases to like 500ms or worse when pinging google.  This negatively affects our VOIP quality and causes us to scramble to figure out which workstation is raping the network.

      This doesn't seem like it should be happening though.  How can I protect against this?

      Specs:

      1.2.3-RELEASE
      PC Engines / Netgate ALIX embedded Firewall
      3158/10000 states
      259 /1920 MBUF Usage
      41% Memory usage
      11% Disk usage

      1 Reply Last reply Reply Quote 0
      • stephenw10S
        stephenw10 Netgate Administrator
        last edited by

        Why do you think this wouldn't happen? If you have used all your upstream bandwidth it's bound to cause trouble for other traffic using the connection.
        To prevent this happening you need to use some form of traffic shaping:
        http://doc.pfsense.org/index.php/Traffic_Shaping_Guide#PFSense_2.0_Traffic_Shaping_Guide_.28Work_In_progress.29

        Steve

        1 Reply Last reply Reply Quote 0
        • A
          azcire
          last edited by

          Logically, it's unreasonable to expect that anytime anyone is uploading a file on a corporate network that it would hose latency for EVERYONE.  There has to be an easy way to solve for this.  This does not happen when directly connected to the cable modem, and it does not happen on my home network with a crappy DD-WRT router.  Some more clues…

          With traffic shaping OFF, and upload bandwidth saturated, there are no issues (average 78ms ping time).

          With traffic shaping ON, and upload bandwidth saturated, there ARE issues (average 350ms ping time).

          Here are my traffic shaping settings:

          http://imgur.com/a/dfpE8

          What's wrong?

          1 Reply Last reply Reply Quote 0
          • stephenw10S
            stephenw10 Netgate Administrator
            last edited by

            Ah OK. It does seem at least unacceptable!
            What is your connection speed? The ALIX has limited computing power, is it running 100% CPU trying to do traffic shaping?

            Steve

            1 Reply Last reply Reply Quote 0
            • A
              azcire
              last edited by

              Advertised connection speed is 50mb down, 5mb up. CPU does not appear to be maxing out (or even spiking) when I saturate upload, according to top via SSH.

              Edit: I read it wrong, I was just looking at "user".  Here's the pic from top during an upload with traffic shaping on:

              http://screencast.com/t/ijcnt1mV

              And here it is with traffic shaping off:

              http://screencast.com/t/AgKU0zXf

              So, idle is 0% with traffic shaping on and idle is 62% with traffic shaping off.  "Nice" and "system" do spike. And if idle is at 0% I assume that means CPU is 100% utilized.

              1 Reply Last reply Reply Quote 0
              • stephenw10S
                stephenw10 Netgate Administrator
                last edited by

                Yes, I'd say you've found the problem.
                I'm no expert on traffic shaping so someone else can almost certainly give you better advise. Less queues?
                Different type of shaping?  :-\

                Steve

                1 Reply Last reply Reply Quote 0
                • D
                  dhatz
                  last edited by

                  @azcire:

                  Logically, it's unreasonable to expect that anytime anyone is uploading a file on a corporate network that it would hose latency for EVERYONE.  There has to be an easy way to solve for this.  This does not happen when directly connected to the cable modem, and it does not happen on my home network with a crappy DD-WRT router.  Some more clues…

                  The behavior you describe is in fact very common, and would be fully expected if one doesn't use traffic shaping. It is however very strange that you say it doesn't happen when you're connected to the cable modem directly… are you sure about that?

                  1 Reply Last reply Reply Quote 0
                  • A
                    azcire
                    last edited by

                    @dhatz:

                    @azcire:

                    Logically, it's unreasonable to expect that anytime anyone is uploading a file on a corporate network that it would hose latency for EVERYONE.  There has to be an easy way to solve for this.  This does not happen when directly connected to the cable modem, and it does not happen on my home network with a crappy DD-WRT router.  Some more clues…

                    The behavior you describe is in fact very common, and would be fully expected if one doesn't use traffic shaping. It is however very strange that you say it doesn't happen when you're connected to the cable modem directly… are you sure about that?

                    Ya. On top of that I showed screenshots both with and without traffic shaping turned on. It's clear that it's the actual traffic shaping and resulting CPU usage that is causing this latency problem.

                    Here's what we're dealing with:

                    1 Reply Last reply Reply Quote 0
                    • C
                      cmb
                      last edited by

                      It's normal without shaping to have significant increases in latency when your download or upload especially is pegged, just a fact of networking. Not as much as with shaping because the ICMP falls in lower priority queues which by the nature of shaping get delayed more.

                      Ping response time has no relation to latency of VoIP traffic or even latency in general when shaping, you're not queuing ICMP in a high priority queue, hence it falls into the default which will (and must) add latency when congestion occurs in order to get higher priority traffic out. Without shaping it's FIFO, so your VoIP is getting delayed behind the ICMP that gets out more quickly.

                      1 Reply Last reply Reply Quote 0
                      • A
                        azcire
                        last edited by

                        @cmb:

                        It's normal without shaping to have significant increases in latency when your download or upload especially is pegged, just a fact of networking. Not as much as with shaping because the ICMP falls in lower priority queues which by the nature of shaping get delayed more.

                        Ping response time has no relation to latency of VoIP traffic or even latency in general when shaping, you're not queuing ICMP in a high priority queue, hence it falls into the default which will (and must) add latency when congestion occurs in order to get higher priority traffic out. Without shaping it's FIFO, so your VoIP is getting delayed behind the ICMP that gets out more quickly.

                        True, bad ping times don't necessarily mean VOIP traffic is suffering. However, in this case, when ping times go to crap on my network, VOIP quality does suffer significantly. People experience a delay on calls and customers start cutting in and out. All evidence seems to point to the CPU being overloaded, and ALL traffic suffering as a result.

                        Also interesting is when shaping is off, I get 75mbit down on speed tests.  With traffic shaping on, the most I've ever seen is 25mbit down.

                        1 Reply Last reply Reply Quote 0
                        • chpalmerC
                          chpalmer
                          last edited by

                          I was able to control my voip qos issues by using the soproxd package which I actually installed for another reason. The "qos" settings (Ill call them) seem to have made a huge difference for my office. May or may not be an option for you.

                          But for reference…  http://cable-dsl.navasgroup.com/#Asymmetry

                          Are you using a cable gateway (has own router) device at the office location? Or just a bridge?

                          Triggering snowflakes one by one..
                          Intel(R) Core(TM) i5-4590T CPU @ 2.00GHz on an M400 WG box.

                          1 Reply Last reply Reply Quote 0
                          • First post
                            Last post
                          Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.