Puzzled: Wan latency is high when no RDP are opened
-
Hi everyone,
I must say, this is probably the most puzzling network issue I have ever seen in my life. Any kind of help would be much appreciated.
Long story short: All of sudden, without any recent known change occurring in our environment, the "internet" started lagging during the day. When pinging from within the network to any host on the internet, the pings will vary from 300 to 2000ms. The funny part here is that if we launch a terminal session using our sever BMC interface (baseboard management console, a java based remote control utility), or if we remote it using any remote access software such as splashtop or webassist or vnc, etc, then the latency almost disappear completely. The minute we drop the last active remote access session, the ping raise back up significantly
If you look at the attached picture, you will see that around noon yesterday, the latency dropped all of a sudden, which coincide exactly with the time I opened and willingly kept an RDP session opened.
https://imgur.com/a/weUFH7iWe used to see this issue even before swaping our old sonicwall tz100 with our new Netgate 3100. The switch connecting all the store network ports is also brand new. We need help finding the culprit and fixing this issue but we are fairly new to netgate. Any suggestion on how to troubleshoot this?
-
Hello!
ISP traffic shaping?
John
-
What is the WAN gateway monitor pinging against? By default it will be the gateway IP which might do odd things. Change it to something remote if that's the case, 8.8.8.8 is a popular target as long as it doesn't conflict with DNS server settings you might have.
Steve
-
Thanks Serbus and Stephew10 for your replies
Interesting developments here, just sharing, fyi;
1- A couple weeks ago, before swaping the Sonicwall for Netgate, we noticed that the sonicwall CPU was at 100%, which was quite unexpected for such an extremely small site with little bandwidth and subpar/light vpn tunnel encryption (should run under 10% cpu at all time). Either way, as soon as we rebooted the old 24ports HP Proliant lan switch, everything went back to normal on the sonicwall and the ping went from 90-2000ms range to a stable more stable 25-75ms (not perfect, but def better). We then replaced the switch along with the firewal, but the problem came back quickly, altho less severe, as we expected from throwing more hardware to palliate (and not fix) the issue at hand.
2- John asked about ISP traffic shaping. While we didn't get anything from the ISP support group to that regard, they did actually force a hard reboot on the modem yesterday after we noticed our Sonicwall (now only part of the lan as a reference before decomissioning it) was still in the >90% CPU, even when not handling any wan traffic (a quite sensitive device if you ask me), and surprisingly after the modem forced reset, the latency went away completely for the entire network the rest of the day and up until now. It is difficult for us to understand how an RDP session could have stabilized the latency from a cable modem into the lan switch passing by our firewall... but anyway, everything seems much better now.
3- Steve, I'm not sure what Netgate use to determine the quality of the link but we did ping from the local store to the gateway, 8.8.8.8 and other servers over our VPN connecting our 3 sites. Anything that was leveraging the wan link would have the exact same kind of latency, may it be initiate from within or outside the lan.
Anyhow, we still haven't found a proper way, with Netgate, to track and identify the culprit, may this be a broadcasting of some sort, or anything else. Yet, we seem to have a good lead with the reboot of this fairly old cable modem. Unfortunately, we might never know why on earth would an RDP session drive the latency of a wan connection has we have observed.
Any other thought or ideas?
I hope to get some real answer lately, so that I can update this post and possibly help some other people in the community. Time will tell!
Thanks again
-
pfSense uses the dpinger daemon to monitor connection quality. It pings something on the WAN twice a second, by default it uses the gateway IP as that;s what it always has but you can set any IP.
It's almost always better to use an external IP as that then actually monitors internet connectivity as opposed to just to the ISP.
https://docs.netgate.com/pfsense/en/latest/monitoring/using-an-alternate-monitor-ip-address-for-gateway-monitoring.htmlWhat you are describing though starts to sound like a possible modem issue. What is the modem they have there?
Steve
