Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    SG-3100 Slow Throughput

    Scheduled Pinned Locked Moved Official Netgate® Hardware
    47 Posts 8 Posters 11.2k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • T
      torred
      last edited by torred

      I've had the SG-3100 for about a year now and it has been pretty solid, until the last month or so. I have AT&T 1GB service, but when connecting through the pfsense, I cannot get more than ~40Mbit/s down and ~70Mbit/s up. I've been mucking with settings for the last week and I've finally hit a wall. I'm hoping someone may see something I've over looked. (I don't recall making any changes that would affect this, other than updating to the latest versions. I was using remote logging for a while, but have that disabled now for testing.)

      Setup
      Internet<-->ATT Residential Gateway<--->(WAN port)SG 3100(LAN port)<-->LAN
      Version 2.4.4-RELEASE (arm)
      built on Thu Sep 20 09:33:19 EDT 2018
      FreeBSD 11.2-RELEASE-p3

      Tests
      Directly connected to the RG - 650+ down / 400.0+ up
      Directly connected to the SG-3100 - 43.2 / 70.0

      Interfaces:
      0_1541012772522_20ec1b47-d594-410d-9da7-7506f522c761-image.png

      0_1541013726582_799913b8-0316-4838-a299-9010011e89b1-image.png

      "Relevant" Settings:
      MBuf clusters
      kern.ipc.nmbclusters=65536
      Note: I had it set to 1M, just lowered it to 65K to see if anything changed

      TSO
      net.inet.tcp.tso=0
      Note: This keeps turning back on... I have it set to 0 in /boot/loader.conf, but upon boot it's back to 1

      I have openvpn (server) running, but nothing is connected.
      I don't have anything else running (no squid, no suricata).

      Any help what-so-ever would be greatly appreciated.

      Edit:
      LAN Speed seems to be fine, iperf3 between the pfsense and a computer on the LAN is 705 Mbit/s

      G 1 Reply Last reply Reply Quote 0
      • G
        gsmornot @torred
        last edited by

        @torred
        I don’t know. I have ATT gig service and SG-3100 and see 910 down, 940 up. I see the same using the provided gateway or bypassing the gateway. I have no idea what would cause you to see such low speeds. Your speeds through the gateway are off a bit too. Good luck.

        1 Reply Last reply Reply Quote 0
        • johnpozJ
          johnpoz LAYER 8 Global Moderator
          last edited by

          The 2 sg3100 I have in production see their full internet speed as well.. Its not gig.. But way above the 40/70 your seeing... That screams something really wrong - maybe duplex mismatch?

          So directly connect you only see about 1/2 of what your paying for - and you don't think that is a problem? If your paying for gig - ou should see high 800's if not low 900's at min.. If not I would be looking to what is wrong there.

          You should not need to do any tweaking like your doing.. It sure and the hell is not going to make 40mbps jump to 600..

          An intelligent man is sometimes forced to be drunk to spend time with his fools
          If you get confused: Listen to the Music Play
          Please don't Chat/PM me for help, unless mod related
          SG-4860 24.11 | Lab VMs 2.8, 24.11

          1 Reply Last reply Reply Quote 0
          • T
            torred
            last edited by

            @johnpoz said in SG-3100 Slow Throughput:

            So directly connect you only see about 1/2 of what your paying for - and you don't think that is a problem?

            I chatted with AT&T about this, it would seem that 600Mbit is "within tolerance" for their 1Gb service, smh.

            I agree that these tweaks shouldn't make such a huge difference, so should I be contacting Netgate about a possible hardware issue?

            1 Reply Last reply Reply Quote 0
            • johnpozJ
              johnpoz LAYER 8 Global Moderator
              last edited by johnpoz

              what about the crappy 400 up? Is that in their tolerance range as well... They should advertise it as you "MIGHT" get close to gig with our service.. But prob not ;) If you get 400 then we say its good! So Piss off and send us your money!!

              How exactly are you testing this? Take your isp device out of the equation... run say iperf..

              iperf server --- wan pfsense lan --- iperf client..

              What do you see then? This should be damn close to this

              ifperf server ---- iperf client

              If no pfsense and iperf serv and client are good, and with pfsense and tested same cables its BAD... like what your seeing then yeah I would be contacting support abut bad hardware..

              What speeds were you seeing with it before you switched to ATT?

              An intelligent man is sometimes forced to be drunk to spend time with his fools
              If you get confused: Listen to the Music Play
              Please don't Chat/PM me for help, unless mod related
              SG-4860 24.11 | Lab VMs 2.8, 24.11

              1 Reply Last reply Reply Quote 0
              • T
                torred
                last edited by

                So good idea removing the RG from the equation, here's what I came up with...

                In summary, TCP looked "good" (~700Mbit - ~900Mbit), regardless of pfsense WAN or LAN ports used (also good if pfsense was not in the path, e.g. just across my switches). Sooo....hardware on the SG-3100 is good...hardware on RG is "good" (suspect)...but as soon as I connect the two they hate me.

                I'm going to do some packet captures and see if I can gleam anything from those. If I find something interesting I'll reply back. If I don't reply in 3 days, I've thrown all of my equipment away and moved to the middle of Montana to start my life as a hermit.

                Same cables used for all tests (Cat 5E)

                --------------------- WAN-to-LAN tests (No ATT RG) -------------------

                Laptop(iperf client)->(WAN port)pfsense (iperf server)
                TCP: 737 Mbit/s
                Reverse TCP: 809 MBit/s

                Laptop(iperf client)->(WAN port-NAT rule)pfsense(LAN port 1)->[2x Netgear ProSafe Switches]->Internal server(iperf server)
                TCP: 863 Mbit/s
                Reverse TCP: 759 MBit/s

                --------------------- LAN-to-LAN tests (No ATT RG) -------------------

                Laptop(iperf client)->(LAN port 4)pfsense (iperf server)
                TCP: 679 Mbit/s
                Reverse TCP: 700 MBit/s

                Laptop(iperf client)->(LAN port 4)pfsense(LAN port 1)->[2x Netgear ProSafe Switches]->Internal server(iperf server)
                TCP: 909 Mbit/s
                Reverse TCP: 795 Mbit/s

                (Test without pfsense)
                Laptop(iperf client)->Netgear Switch->Internal Server (iperf server)
                TCP: 899 Mbit/s
                Reverse TCP: 949 Mbit/s

                --------------------- LAN-to-Internet tests -------------------

                (SG-3100 and ATT RG)
                Laptop(iperf client)->[2x Netgear ProSafe Switches]->(LAN port 1)pfsense(WAN)->(LAN Port 1)ATT RG->Internet VPS(iperf server)
                TCP: 14.4 Mbit/s (this would be me uploading) -- Likely an issue with my VPS, it throttles uploads
                Reverse TCP: 43.7 Mbit/s (this would be me downloading)

                Same thing, except to http://speedtest.att.com/speedtest/
                Upload: 47 MBit/s
                Download: 76.6 MBit/s

                (Just ATT RG, no SG-3100)
                (I used the same cable that was between the SG-3100 and the RG)
                Laptop(iperf client)->(LAN Port 1)ATT RG->Internet VPS(iperf server)
                TCP: 14.8 Mbit/s (this would be me uploading) -- Likely an issue with my VPS, it throttles uploads
                Reverse TCP: 21.6 - 455 Mbit/s (this would be me downloading) (Why the huge difference??? IDK.. sometimes low, sometimes 200s sometimes 400s... over 10 tests)

                Same thing, except to http://speedtest.att.com/speedtest/
                Upload: 124 - 459 MBit/s
                Download: 268 - 828 MBit/s (Again, all over the place)

                @johnpoz said in SG-3100 Slow Throughput:

                What speeds were you seeing with it before you switched to ATT?

                I've had AT&T for 3 years. When I got the SG-3100 back in October 2017 (as soon as it was released), my speed was good (600-700+). I've only noticed the slow down in the last month or two.

                1 Reply Last reply Reply Quote 0
                • johnpozJ
                  johnpoz LAYER 8 Global Moderator
                  last edited by

                  @torred said in SG-3100 Slow Throughput:

                  pfsense (iperf server)

                  That is going to show you low results compared to routing THRU pfsense.. You need 2 boxes.. Do not use pfsense as client or server in your iperf testing.

                  An intelligent man is sometimes forced to be drunk to spend time with his fools
                  If you get confused: Listen to the Music Play
                  Please don't Chat/PM me for help, unless mod related
                  SG-4860 24.11 | Lab VMs 2.8, 24.11

                  T 1 Reply Last reply Reply Quote 0
                  • T
                    torred @johnpoz
                    last edited by

                    @johnpoz I had only done that as one part of the test, as you can see, I did test through it in other tests.

                    I honestly do not know what is going on. I did a factory reset on the SG-3100 with the same results. Except now I'm experiencing a multitude of other failures.

                    I've removed my pfsense, and am just using the ATT RG and everything works perfectly.

                    Thanks for your help @johnpoz, I'll be contacting Netgate Support and see if they can help me out.

                    1 Reply Last reply Reply Quote 0
                    • T
                      torred
                      last edited by torred

                      Alright, if anyone happens to read all the way down here, I never figured out what the problem between the two was, but I ended up bypassing the AT&T RG by doing this:
                      https://github.com/aus/pfatt

                      It was fairly easy to compile the needed ng_etf.ko kernel module for armv6:

                      1. Get a FreeBSD 11.2 amd64 VM going (for pfSense 2.4.4) -- make sure to include src when installing
                      2. Get a shell, and do this:
                        $ cd /usr/src
                        $ make kernel-toolchain TARGET_ARCH=armv6
                        # Wait about an hour
                        $ make buildenv TARGET_ARCH=armv6 BUILDENV_SHELL=/bin/sh
                        $ cd /usr/src/sys/modules/netgraph/etf
                        $ make
                      3. You now have your own compile netgraph etf module, follow the rest of the guide.
                      4. I used the OPT1 (mvneta0) port for the RG, and the WAN (mvneta2) port for the ONT

                      Everything works fine now.

                      S 1 Reply Last reply Reply Quote 0
                      • S
                        sean.allen @torred
                        last edited by

                        @torred - you're one step (but light years) ahead of where I am. I simply need a trusted copy of ng_etf.ko for FreeBSD 11.2 to plop onto my SG-3100 and I'm done. Everything else in the pfatt project is ready for the reboot.

                        I don't have spare hardware lying around so have been trying to download the FreeBSD VMware image, but it has no source. And when I try to download the source, it fails. This simple step mocks me. Any thoughts?

                        Sean

                        T 1 Reply Last reply Reply Quote 0
                        • T
                          torred @sean.allen
                          last edited by torred

                          @sean-allen, try following this: https://www.freebsd.org/doc/handbook/makeworld.html#updating-src-obtaining-src

                          TL;DR: svn update /usr/src

                          There's quite a few guides on setting up FreeBSD for qemu, virtualbox, and VMWare. Once you get it running it's pretty easy.

                          Edit: Also, you could...

                          wget ftp://ftp.freebsd.org/pub/FreeBSD/releases/amd64/11.2-RELEASE/src.txz
                          tar -xz -C / -f src.txz

                          S 1 Reply Last reply Reply Quote 0
                          • S
                            sean.allen @torred
                            last edited by

                            @torred I'm very thankful for your help with the ng_etf.ko. I have all in place and am bypassing the AT&T RG.

                            However, my speeds have not improved. At all. Quite the let down as I was assuming the interaction between RG and SG-3100 was the issue

                            • Directly through the RG I was seeing 900+Mbit up/down
                            • SG-3100 through RG in IP Passthrough yielded ~100Mbit up/down
                            • Same setup using PIA as my VPN gave ~75Mbit up/down
                            • I bypassed the RG with great expectation and the ~100Mbit and ~75Mbit numbers remained. <sad trombone>

                            Those numbers varied, but not nearly as wildly as @torred results. The speed tests I'm doing are speedtest.net, dslreports.com and att.com. I'm not familiar with iperf. I loaded it on pfSense and the dizzying array of config options had me walk away from that.

                            Other than the PIA VPN, I have:

                            • pfBlockerNG DNSBL
                            • OpenVPN Server (though no clients, it's just there to hit my network from outside while traveling)
                            • ntopng

                            Turning off DNSBL and ntopng have no measurable effect on speed tests. I have the laptop I'm running speed tests on directly connected to one of the switched ethernet ports on the back of the SG-3100 removing other switches from the test.

                            Any other thoughts or suggestions here? I feel like the SG-3100 should be able to keep up with these, even with VPN, based on what I've read. It surely should be going faster than it is.

                            Thank you!
                            Sean

                            Side note: Anyone know why I can't access these forums through my PIA VPN? I have to bypass that before any page will load.

                            GrimsonG 1 Reply Last reply Reply Quote 0
                            • GrimsonG
                              Grimson Banned @sean.allen
                              last edited by

                              @sean-allen said in SG-3100 Slow Throughput:

                              Side note: Anyone know why I can't access these forums through my PIA VPN? I have to bypass that before any page will load.

                              https://forum.netgate.com/topic/136229/vpn-blocked

                              S 1 Reply Last reply Reply Quote 0
                              • S
                                sean.allen @Grimson
                                last edited by

                                @grimson thanks! I searched the forum, but I kept getting assorted posts about PIA/VPN/access/etc. - none having to do with the forum.

                                Sean

                                1 Reply Last reply Reply Quote 0
                                • S
                                  sean.allen
                                  last edited by

                                  It's clear that personal VPNs are a contentious issue here, so let's remove that from the equation for now.

                                  I can get 900Mbit speeds directly from the AT&T RG, but as soon as I introduce my SG-3100 into the path (either through or bypassing the RG) I start getting 100Mbit (not through VPN) - or a bit more than 10% of the available bandwidth.

                                  Any ideas on how I've messed up my configuration such that the SG-3100 is pouring molasses on my link? I'm going through my entire network to make sure I have "good" cables and switches to remove that from the equation - but even when I plug a new cable directly into the switched ports on the SG-3100, same result.

                                  Sean

                                  1 Reply Last reply Reply Quote 0
                                  • DerelictD
                                    Derelict LAYER 8 Netgate
                                    last edited by

                                    The AT&T RG is a beast, unfortunately.

                                    https://forum.netgate.com/topic/99190/att-uverse-rg-bypass-0-2-btc/1

                                    Chattanooga, Tennessee, USA
                                    A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                                    DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                                    Do Not Chat For Help! NO_WAN_EGRESS(TM)

                                    S 1 Reply Last reply Reply Quote 0
                                    • S
                                      sean.allen @Derelict
                                      last edited by

                                      @derelict Precisely why I went to such lengths to bypass it. Finally have that working (because @aus is a rockstar). Now the RG is not inline for standard (non EAP) traffic. It is SG-3100->ONT. That config yields less than 100Mbit, whereas if I do client->RG->ONT I get 900Mbit.

                                      1 Reply Last reply Reply Quote 1
                                      • DerelictD
                                        Derelict LAYER 8 Netgate
                                        last edited by

                                        It must still be something in the bypass configuration then. Perhaps something in the traffic that is not marked in some way that AT&T expects it.

                                        Or something that should be negotiating gigabit is negotiating at 100.

                                        If it were me - and I couldn't find someone else who has put all the pieces together - I would put a switch with a SPAN port between the RG and the ONT in this configuration client->RG->ONT and capture traffic on a mirror port.

                                        Then I would put the same switch between the SG-3100 and the ONT in this configuration SG-3100->ONT and capture traffic and see if there is a difference in QoS bits, VLAN priority, or something.

                                        Chattanooga, Tennessee, USA
                                        A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                                        DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                                        Do Not Chat For Help! NO_WAN_EGRESS(TM)

                                        1 Reply Last reply Reply Quote 1
                                        • johnpozJ
                                          johnpoz LAYER 8 Global Moderator
                                          last edited by

                                          Yup that is exactly the steps need to figure out what is going on

                                          An intelligent man is sometimes forced to be drunk to spend time with his fools
                                          If you get confused: Listen to the Music Play
                                          Please don't Chat/PM me for help, unless mod related
                                          SG-4860 24.11 | Lab VMs 2.8, 24.11

                                          1 Reply Last reply Reply Quote 0
                                          • S
                                            sean.allen
                                            last edited by

                                            @Derelict and @johnpoz - thanks for the feedback!

                                            Quick question, though: if client->RG->ONT is at 900Mbit and (SG-3100->RG->ONT or SG-3100->ONT) are both sub 100Mbit - doesn't that point to an issue with the config or hardware of the SG-3100? The RG running in IP Passthrough, or being bypassed, yields the same result when the SG-3100 originates the traffic. The bypass method would seem to not be adding or subtracting anything relevant here, but I defer to your expertise. The bypass, if you're curious, uses netgraph to set aside the EAP auth traffic such that it only goes between RG and ONT (which are plugged into the two routed eth ports of the 3100). All other traffic sent directly from the SG-3100 to the ONT via a new interface (ngeth0) defined by netgraph to tag outbound as VLAN 0 (some odd AT&T requirement). It would appear that the only thing the RG is used for by AT&T is to make sure AT&T equipment is present - so the hard-coded cert in the RG is required to authenticate the channel. Full details on the bypass, if interested, are here: https://github.com/aus/pfatt

                                            The reason I ask is because it will not be easy for me to mirror and capture traffic as you've suggested. Partly because of hardware, partly because of expertise.

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.