Extremely Frustrating Outages

Stewart

What I was posting you can start seeing around packet 1190. But yes, lots of DNS. I don't know how normal it is. In fact, to look through the capture I used !(dns) as a filter.

Stewart

@jknott Have you had a chance to look anymore? Anything you see?

JKnott

@stewart

The one curious thing I see is about 35 duplicate acks that a device on Spectrum (I assume that's you) to Cloudfront. That would indicate a lost ack at the Cloudfront end. That's about all I see.

Stewart

@jknott @stephenw10

I went through all of the logs for all of our installs and identified a few other sites that appeared to be experiencing something similar. Going on the theory that maybe we are overloading the modem and the large number of DNS packets seen in the captures, I've switched Ubound from Resolver to Forwarder in a some of the units having issues. Since I have OpenDNS in the General tab I also disabled DNSSEC. After a couple of days I can see a trend where the issues have settled down tremendously. It's been enough that I don't think it's coincidence. Any thoughts?

stephenw10

Could be. There are some modems that are known to badly handle a lot of UDP traffic. (Intel PUMA based).

What modems are you using?

Steve

chpalmer

What brand and model modem are you using? Is it a "gateway" type modem that the ISP has in their style of "bridge mode"? It makes a difference.

Stewart

@chpalmer @stephenw10

At the site I was looking through logs with @JKnott, this is the info provided in the modem:

ARRIS DOCSIS 3.0 / PacketCable 2.0 Touchstone Residential Gateway
HW_REV: 8
VENDOR: ARRIS Group, Inc.
BOOTR: 4.2.0.45
SW_REV: 9.1.103J6TW2.SIP.PC20.TW
MODEL: TG1682G

I don't know about the other sites. Also, the initial site I was having problems with (that was fixed by removing the laptops from the wireless) went through several model of modems. I believe it is currently also using a TG1682G. In most cases the modems are configure as RIP w/o NAT. In some instances where there is no Static IP then they are in Bridge mode.

chpalmer

@stewart said in Extremely Frustrating Outages:

TG1682G

Thanks Stewart. Yes definitely a Puma model. https://badmodems.com/Forum/app.php/badmodems

If Spectrum has not updated firmware on these models.. then you can blame the modem.

Unless you can get Spectrum to provide a non Puma model you can continue to expect problems. Many issues have been fixed by the firmware fixes but many problems persist.

Stewart

@chpalmer said in Extremely Frustrating Outages:

@stewart said in Extremely Frustrating Outages:

TG1682G

Thanks Stewart. Yes definitely a Puma model. https://badmodems.com/Forum/app.php/badmodems

If Spectrum has not updated firmware on these models.. then you can blame the modem.

Unless you can get Spectrum to provide a non Puma model you can continue to expect problems. Many issues have been fixed by the firmware fixes but many problems persist.

That's really nice of you; thinking that Spectrum will update their modem's firmware! We once fought with them for months, even trying to install our own modems, because they had a firmware that broke SIP. (I think that was the TG1672 model) Happened all over across many of our clients. The only solution we found was to run SIP over SSL so it wouldn't know what the traffic was. Even had clients switch VOIP providers because their existing provider couldn't (or wouldn't) do it over SSL. That was with SIP ALG off, Firewall off, etc. Now they look back and laugh at all the headaches it caused but back then they wouldn't take any responsibility for it.

Can you explain what PUMA means?
Edit: I put in the IP of the modem and it does indeed cause the same high packet loss and latency I've been experiencing.

chpalmer

@stewart

If you haven't look over that Badmodems site and familiarize yourself with the issue.

Ive had to change out a bunch of modems for customers around my area due to similar issues.

Get a Broadcom based modem if you can.. Im doubtful though with the static IP's you are using but you have to ask. Im not familiar with Spectrum and how they hand out multiples..

Stewart

@chpalmer

Unfortunately, Spectrum only allows you to bring your own modem on a residential account. On a Business account you must use their modems. You now get the lottery of either the TB1682G or a new Spectrum branded 2 box solution.

chpalmer

@stewart

Thanks! Adding that information to my knowledge base.. Luckily I only have one customer on Spectrum in North Carolina and they are residential customers. Motorola MB8600 on their account.

Ill I can say is escalate escalate escalate! Enough input by customers and customers reps finally got Comcast to allow COE on their business accounts as long as static IP's are not involved.

Stewart

@chpalmer Escalating with Spectrum gives me "Call us again on this and we'll bill you for coming out." I've had multiple techs go out to both sites. The techs that go on site say they put the modem in their "SCOPE" system which puts them in Device Watch. That allows the techs to go back and look at history for the unit. BUT, when you call in and talk to a CSR every one of them says that they no longer use that system. Only the techs onsite can setup or see into the SCOPE system now but you can't get a tech onsite without the possibility of them billing for every visit. Even then they just troubleshoot the moment and don't even refer to it unless you make it a point to make them. It's crazy. But still better than every DSL provider and AT&T U-Verse in this area. You pick your poison.

stephenw10

Maybe try using DNSoverTLS so you're not sending UDP DNS packets at all.
If that improves things further at least it's another data point.

Steve

Stewart

@stephenw10 said in Extremely Frustrating Outages:

Maybe try using DNSoverTLS so you're not sending UDP DNS packets at all.
If that improves things further at least it's another data point.

Steve

It's only for forwarding that I can see. Is that accurate?

stephenw10

Yes, you can only use with forwarding mode.

As a test though it would be useful just eliminate all DNS UDP packets hitting the modem.

Steve

brians

https://forum.netgate.com/topic/137938/shaw-300-issue-with-arris-xb6-modem-severe-intermittent-wan-slowdowns/9?_=1614764271055

Sounds similar to issue I had couple years ago with Shaw cable.
Not sure if it was modem or moca filter that resolved as both were done at same time.