Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    IPSect Site to Site (Slow Upload) - (Fast Download) issue

    Scheduled Pinned Locked Moved
    IPsec
    5
    24
    2.0k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • A
      AMD_infinium05
      last edited by AMD_infinium05

      Hello Guys,

      I need help troubleshooting this problem.

      • Site to Site is established
      • IPSec Firewall rules are set to "Any to Any" on both ends
      • Both links have 100Mbps Up/Down speed
      • Both hardware has plenty of resources

      SiteA
      Pfsense 2.4.5 Dev
      Subnet 192.168.20.0/24

      SiteB
      Pfsense 2.4.4 Stable
      Subnet 192.168.45.0/24
      Site A to Site B uploading to Qnap via Samba/CIFS the upload can only max out at 2.5mbits/sec

      Test Scenarios:

      1. Site A to Site B download from Qnap via Samba/CIFS the download runs betweent 70-90Mbit/sec

      2. Site A to Site B remote desktop protocol to a computer in Site B file transfer via RDP is 70-90mbits/sec

      3. Site B to Site A remote desktop protocol to a computer in Site A files transfers via RDP is 70-90mbits/sec

      The only problem I am encountering is any computer from SiteB uploading data to Qnap Device in Site A by Samba/CIFS is very slow. Other than that everything is working normal.

      I have scoured both end's pfsense for any traffic shapers, there is none. I have not configured any shapers since the deployments. I have been looking at this for over 48 hrs already and couldn't figure out whats wrong.

      DerelictD 1 Reply Last reply Reply Quote 0
      • P
        P3R
        last edited by

        Why do you suspect pfSense and not the Qnap?

        1 Reply Last reply Reply Quote 0
        • A
          AMD_infinium05
          last edited by

          Because Qnap works just fine on SiteA.
          Do you have anything in mind where else to look at?

          1 Reply Last reply Reply Quote 0
          • DerelictD
            Derelict LAYER 8 Netgate @AMD_infinium05
            last edited by

            @amd_infinium05 said in IPSect Site to Site (Slow Upload) - (Fast Download) issue:

            The only problem I am encountering is any computer from SiteB uploading data to Qnap Device in Site A by Samba/CIFS is very slow. Other than that everything is working normal.

            How are you testing?

            What is the latency between the sites?

            SMB is probably the slowest, chattiest, crappiest protocol you can use for WAN transfers.

            Chattanooga, Tennessee, USA
            A comprehensive network diagram is worth 10,000 words and 15 conference calls.
            DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
            Do Not Chat For Help! NO_WAN_EGRESS(TM)

            A 1 Reply Last reply Reply Quote 0
            • P
              P3R
              last edited by P3R

              @derelict said in IPSect Site to Site (Slow Upload) - (Fast Download) issue:

              SMB is probably the slowest, chattiest, crappiest protocol you can use for WAN transfers.

              The advantage with CIFS/SMB is that it's very convenient and user friendly on Windows and it's the recommended protocol on MacOS. I've been running it over an IPsec VPN to different NASes for many years now. I just tried copying a 1 TB file to and from a very (8 year) old Qnap over a symmetric 100 Mb/s connection. When writing to the NAS it had one very short drop down to 9.89 MB/s but was otherwise steadily above 10 MB/s. When reading from the Qnap it's rock steady at 10.6 MB/s with extremely small variations. In my opinion that's not a terrible protocol overhead.

              I see no reason to use anything else regularly but for troubleshooting this issue it may be interesting to try something else but let's hear the answer to your questions first.

              1 Reply Last reply Reply Quote 0
              • DerelictD
                Derelict LAYER 8 Netgate
                last edited by

                Not much in the firewall could cause what he's seeing. if SMB works fine in one direction it should work as well in the other.

                Chattanooga, Tennessee, USA
                A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                Do Not Chat For Help! NO_WAN_EGRESS(TM)

                1 Reply Last reply Reply Quote 0
                • A
                  AMD_infinium05
                  last edited by

                  It is not just SMB, even Https/Http browsing from SiteB to Site seems slow.
                  I have also tested with iPerf 3.1.3

                  Site B to Site A - speed maxes out at 2-3Mbits/sec
                  Site A to Sibe B - speed maxes out the links which is 75-95Mbits/sec

                  Commands used:
                  server: iperf.exe --server
                  client: iperf.exe --c <serverIP> -t 60

                  DerelictD 1 Reply Last reply Reply Quote 0
                  • A
                    AMD_infinium05 @Derelict
                    last edited by

                    @derelict
                    Hello!

                    The latency between sites in the tunnel is 3-6ms.

                    K 1 Reply Last reply Reply Quote 0
                    • K
                      Konstanti @AMD_infinium05
                      last edited by

                      @amd_infinium05

                      Hey

                      1. if you test iperf with udp - will the speed change ?
                      2. what network cards are used ?
                      A 1 Reply Last reply Reply Quote 0
                      • A
                        AMD_infinium05 @Konstanti
                        last edited by

                        @konstanti I still have to test.

                        all nodes uses Intel based NIC cards, workstations, pfsense boxes.
                        Afaik all Intel Gbe.

                        1 Reply Last reply Reply Quote 0
                        • DerelictD
                          Derelict LAYER 8 Netgate @AMD_infinium05
                          last edited by Derelict

                          @amd_infinium05 said in IPSect Site to Site (Slow Upload) - (Fast Download) issue:

                          It is not just SMB, even Https/Http browsing from SiteB to Site seems slow.
                          I have also tested with iPerf 3.1.3

                          Site B to Site A - speed maxes out at 2-3Mbits/sec
                          Site A to Sibe B - speed maxes out the links which is 75-95Mbits/sec

                          Commands used:
                          server: iperf.exe --server
                          client: iperf.exe --c <serverIP> -t 60

                          That conflicts with what you said here:

                          Site A to Site B remote desktop protocol to a computer in Site B file transfer via RDP is 70-90mbits/sec

                          If it is just sending to the Q-NAP that is problematic then that points to something in the Q-NAP.

                          Perhaps a limiter on transfers in from non-local subnets?

                          There really isn't anything in the firewall that would cause this unless a limiter/shaper was deliberately placed.

                          I assume both sites can run traditional speed tests to internet test sites at or near 100/100.

                          Chattanooga, Tennessee, USA
                          A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                          DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                          Do Not Chat For Help! NO_WAN_EGRESS(TM)

                          A 1 Reply Last reply Reply Quote 0
                          • A
                            AMD_infinium05 @Derelict
                            last edited by

                            @derelict I am having the same issue with iperf when Im initiating the connection from site B to any windows server on site A through the tunnel.

                            Im pretty sure QNAP has no problem as the workstations/servers on site A (same subnet as with qnap) can send data to qnap reaching 1gbps.

                            DerelictD 1 Reply Last reply Reply Quote 0
                            • DerelictD
                              Derelict LAYER 8 Netgate @AMD_infinium05
                              last edited by

                              @amd_infinium05 Right. Often connections from other subnets are trated differently.

                              I really cannot think of anything in the firewall that would cause what you are seeing unless you deliberately set a limiter. There is no checkbox to enable the issue you are seeing.

                              Packet capture an iperf session and see if there are retransmissions or something.

                              Set MSS Clamping in the advanced IPsec settings down to, say, 1300 and try again.

                              Try UDP iperf.

                              Chattanooga, Tennessee, USA
                              A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                              DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                              Do Not Chat For Help! NO_WAN_EGRESS(TM)

                              A 1 Reply Last reply Reply Quote 0
                              • A
                                AMD_infinium05 @Derelict
                                last edited by

                                @derelict thank you.

                                I will try udp on iperf when I get my hands on it.

                                I have observed that any traffic that is initiated from site a to site b gets full speed up and down through the tunnel regardless of what type of task I throw at it (rdp/samba/iperf).

                                Connections initiated from site B (iperf and file transfer via cifs/smb to qnap it is slow). This is really weird from my point of view.

                                Also I have observed that if the connection is initiated from site B, it is actually hitting the ipsec firewall rule on site A. If the connection is initiated from site A it is hitting the ipsec firewall rule in Site B. --- this is normal yes?

                                1 Reply Last reply Reply Quote 0
                                • DerelictD
                                  Derelict LAYER 8 Netgate
                                  last edited by

                                  Yes. The firewall rules on IPsec are the same as any other interface. They govern connections coming INTO that firewall on that interface.

                                  Chattanooga, Tennessee, USA
                                  A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                                  DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                                  Do Not Chat For Help! NO_WAN_EGRESS(TM)

                                  1 Reply Last reply Reply Quote 0
                                  • A
                                    AMD_infinium05
                                    last edited by

                                    So here are my iperf tests

                                    Site B to Site A (left window is Site B, right window is Site A)
                                    0_1550448666703_80e4182a-55ac-4e7b-801c-81705b73e3b3-image.png

                                    Site B to Site A
                                    0_1550448833656_d52cd3b2-0467-4044-8b47-b70f0a5b0779-image.png

                                    1 Reply Last reply Reply Quote 0
                                    • DerelictD
                                      Derelict LAYER 8 Netgate
                                      last edited by

                                      Still doesn't point at anything on the firewalls themselves.

                                      (You have to specify a -b bandwidth flag when using UDP or it tries to send 1Mbit/sec as you saw)

                                      Chattanooga, Tennessee, USA
                                      A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                                      DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                                      Do Not Chat For Help! NO_WAN_EGRESS(TM)

                                      B 1 Reply Last reply Reply Quote 0
                                      • B
                                        bbrendon @Derelict
                                        last edited by

                                        How do you know its not the ISP? I swear I've seen Comcast Residential throttle all kinds of things.

                                        A 1 Reply Last reply Reply Quote 0
                                        • A
                                          AMD_infinium05 @bbrendon
                                          last edited by

                                          @bbrendon I dont know sir. I do not know where else to look at.

                                          1 Reply Last reply Reply Quote 0
                                          • A
                                            AMD_infinium05
                                            last edited by

                                            Here are my speedtest using UDP from SiteB to SiteA

                                            They are showing two different information.

                                            Left: Site B (client)
                                            Right: Site A (Server)

                                            0_1550841573559_edb439b1-4f9a-4980-9afb-ba9cb0cc0859-image.png

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.