[SOLVED] Strange upload performance with Gigabit FTTH connection

bmeeks

@johnpoz said in Strange upload performance with Gigabit FTTH connection:

@bmeeks said in Strange upload performance with Gigabit FTTH connection:

I was totally surprised to have Gigabit fiber suddenly available to me

I keep checking - so the big question, do you have a public IPv4 or you behind a cgnat?

I could easy go gig down with my isp, or a couple of others - but I don't really care about down, 500 is fine for my needs.. But I really want more up.. 50 up is a bottle neck to my plex.. While I have not really run into any real issues as of yet.. I just wish is was 500/500

CGNAT for now, but their engineer promised they were working on direct public IPs. He did offer to configure a static port forward if I wanted, but I deferred. They are also getting ready to roll out an IPv6 BETA, and he will enroll me in that. This is a small town ISP, and I personally know the folks, so that is a positive that helps override the negative of CGNAT.

Like you, I wanted more upload. I was also limited to 50 meg with my cable ISP. A symmetrical path is nice -- no more bufferbloat shaping needed (well, at least for most of my machines) - this problem with my 8940 notwithstanding. Even still, with 150 - 180 megabits of upload, I score an A on the bufferbloat test. Haven't tested from one of my machines that gets full bore upload. Should get A+ there.

johnpoz

@bmeeks they could give me 10ge up and it wouldn't do me much good for plex ;) heheh if behind a cgnat.

bmeeks

@johnpoz said in Strange upload performance with Gigabit FTTH connection:

@bmeeks they could give me 10ge up and it wouldn't do me much good for plex ;) heheh if behind a cgnat.

I hear you. The only thing I did remotely was inbound VPN for rare remote access. But with secure cloud storage a bit more mature, I can now put the MS Word docs my wife and I might want to access remotely in the cloud. That cut down on the need for inbound VPN.

I get that it's hard for these small ISPs to afford enough IPv4 space to provide public routable IP addresses for all their customers.

johnpoz

@bmeeks well with that gig down you should have like 20 or 30 linux distros download and ready - have you tested it yet?

bmeeks

Working on it. Waiting on the bootable USB utility to finish creating a USB image. Decided to go that route instead of a DVD.

It's just finishing verifying, so going offline and going to boot into Ubuntu and test. Will report back when I'm done.

SteveITS

@bmeeks
I'd start with the "Receive-Side Scaling" and/or "Receive Segment Coalescing" settings. I seem to recall occasional posts in the past. Couldn't find any here quickly, but found:

This is a Dell "server" document: https://www.dell.com/support/kbdoc/en-us/000138508/how-to-resolve-performance-tuning-and-connectivity-issues

random person:
https://forums.cox.com/forum_home/internet_forum/f/internet-forum/24208/to-all-the-people-that-have-been-experiencing-low-upload-speeds-i-have-had-a-break-through

bmeeks

@johnpoz said in Strange upload performance with Gigabit FTTH connection:

@bmeeks well with that gig down you should have like 20 or 30 linux distros download and ready - have you tested it yet?

Well, the quest needs modification. Tested on Ubuntu LTS and got 964 megabits/sec down and 954 megabits/sec up. So, the issue is most definitely within the Windows 11 networking setup someplace.

I see @SteveITS has posted some possibilities while I was testing the Linux distro. Will check those out, although I previously have played around with the RSC settings.

At least I have eliminated all hardware possibilities. I really don't want to wipe and reload Windows, though. I've got too much other app configurations I hate to lose.

johnpoz

@bmeeks said in Strange upload performance with Gigabit FTTH connection:

Windows 11 networking setup someplace.

Oh yeah there you go - you found the problem ;) heheh

You got a spare disk around, some ssd you can just do a install of 10, and see if any issues with that..

edit: I browsed through that 2nd one - its mostly gibberish.. Bunch of settings that have shit to do with it.. Possible maybe the the large send offload, you could try disabling that maybe.. but disable RSS, that is nonsense..

bmeeks

@johnpoz said in Strange upload performance with Gigabit FTTH connection:

@bmeeks said in Strange upload performance with Gigabit FTTH connection:

Windows 11 networking setup someplace.

Oh yeah there you go - you found the problem ;) heheh

You got a spare disk around, some ssd you can just do a install of 10, and see if any issues with that..

Well, I know it's not Windows 11 because my wife's 8950 machine with 11 works fine. Also have Windows 10 on an older HP desktop with Realtek motherboard NIC and that one works fine. So, it's something in the network stack of my machine. I installed it new in the middle of March last year. All my Windows installs are Windows Pro edition and are domain-joined.

I could just order another Intel NIC and install that. Or I may could totally remove the driver for the current Killer NIC and reinstall the software with JUST the driver.

johnpoz

@bmeeks valid point, but your wifes 11 is using a different nic maybe? And you reset the stack already you mentioned.. I take your wifes is default on the stack, no tweaking you did..

edit: for a test could you pull your wifes nic? Or you have another nic? Or maybe even shoot a usb nic? But what is strange is no issues locally on the nic... You found a odd one for sure.

edit2: I take it your not running any "security" sort of software - you mentioned you disabled all that nonsense that dell installs with that killer nic..

bmeeks

@johnpoz said in Strange upload performance with Gigabit FTTH connection:

@bmeeks valid point, but your wifes 11 is using a different nic maybe? And you reset the stack already you mentioned.. I take your wifes is default on the stack, no tweaking you did..

No tweaking on hers. And hers still has the Killer Network Performance app installed.

In digging around old files in the hidden ProgramData directory I did see reference to Dell Optimizer back last March. That would have shipped on the machine back then I think. And according to the web, that app caused lots of issues. And one post even said it caused issues specifically with accessing external sites (not on the local LAN). I must have uninstalled the Optimizer app because there is no trace of it on the machine now that I see (but have not scanned the registry yet). But perhaps it left some hooks in place ??

Bob.Dig

@johnpoz I think the vmware installation is altering Windows.
Maybe hyper-v is an option for the future.

johnpoz

@bob-dig I think he uninstalled that..

Bob.Dig

@johnpoz No, I mean that moment you install vmware it will change your Windows installation (forever).

I know that third party firewall can have an impact too, but I guess it would be already mentioned, if @bmeeks would use one.

johnpoz

@bob-dig hmmm I have not heard of that.. While I have had vmware on this machine at one point - again I don't have the upload internet to see if I am having the same problem.

But perhaps it left some hooks in place ??

I always wipe and do a clean install of my machines - just not a fan of the bloatware they always install.. While I have been happy with the dell 8940.. It was a clean install..

bmeeks

I have uninstalled VMware Workstation, but @Bob-Dig has a valid point. It may have left something in place.

I have tried turning the LRO and TSO features off and on, but it had no effect. Same with hardware checksumming -- no change in behavior. Currently, I have all those back at the defaults (and they are matching the comparable settings on my wife's working PC that has a slightly newer version of the same NIC).

I'm leaning towards something being altered deep down in the networking stack, and VMware is a prime suspect. Time to go hunting ... lots of apps do a very crappy job of fully uninstalling themselves.

johnpoz

@bmeeks but you would think when you did a reset via netsh that shit would of been removed?

Now that you have vmware removed - maybe try another reset of the stack?

bmeeks

@johnpoz said in Strange upload performance with Gigabit FTTH connection:

@bob-dig hmmm I have not heard of that.. While I have had vmware on this machine at one point - again I don't have the upload internet to see if I am having the same problem.

But perhaps it left some hooks in place ??

I always wipe and do a clean install of my machines - just not a fan of the bloatware they always install.. While I have been happy with the dell 8940.. It was a clean install..

I do routinely remove all the bloatware, starting with the McAfee stuff and then all the Dell stuff. But I don't do a complete wipe and install.

bmeeks

@johnpoz said in Strange upload performance with Gigabit FTTH connection:

@bmeeks but you would think when you did a reset via netsh that shit would of been removed?

Now that you have vmware removed - maybe try another reset of the stack?

I think I did do the reset after VMware removal, but I will repeat just to be sure.

bmeeks

Well, I think I'm beaten. Just sitting here in the corner with my tail between my legs sulking ... . I have been unable to identify a cause for this weird external network throttling, and so therefore have not fixed it either. Speeds up and down to internal testing hosts run at full line rate, but speeds to external sites download at full Gigabit line rate, but upload only between 150 and 200 megabits/sec (usually more like 150ish).

What I've Tried
Note these steps are not necessarily in the order they were performed, they are simply in the order "remembered" by my now weary mind ... .

Tried removing VMware Workstation -- no change in behavior.
Reset the Windows network stack (twice) -- no change in behavior.
Totally uninstalled the Intel Killer E2600 Ethernet device (from Device Manager), rebooted, and let the default Microsoft driver install -- no change in behavior (although the Microsoft driver appeared a tad slower downloading).
Installed the newest Intel driver for the NIC (driver only, not the Killer Performance Center bloatware).
Uninstalled every possible piece of Dell bloatware -- no change in behavior.
Ran the Windows DISM tool to scan and repair the Windows 11 image -- no change in behavior (and the tool found nothing wrong).
Booted an Ubuntu USB live image on the machine and got the expected 950+ megabits/sec up and down speeds testing from Ubuntu to the external speed test server, so that rules out local hardware and cabling since it works fine on a Linux distro using the same NIC, cabling, and switch port.

There are only two things left to do. One is to replace the NIC with a completely different model. Likely will try that. That's a pretty easy next step with no real downside (except spending a little cash).

The other option is to wipe the PC and reload Windows 11, but I am loathe to do that as the PC has a lot of my daily work software installed and configured on it. While I could reinstall, it would be a real pain to reconfigure all that -- especially if I did all that and it still did not fix the upload speed problem!