Interpreting WAN quality RRD graph
-
We seem to be getting peak time congestion for the past while which I'm assuming is more subscribers or heavier usage in our area. What would be the best way of logging this effect if present? Informally checking things out when things slow down, pings increase and total bandwidth on the traffic graphs drops from a bit over 3Mbps to about 756Kbps (though the RRD Traffic graphs don't seem to go as high as speed tests). It seems to correspond to large pings on the WAN quality graphs, but would they be affected by our own traffic out which would presumably peak about the same time?
Attached is an example of the most recent WAN quality graph:
-
Well, the latency at peaks sucks, heavily… WiFi, presumably?
-
No, ADSL1. And it gets worse than that, can go over 10 seconds. I'm assuming there is a bonded 2x E1 backhaul so there would be up to 4Mbps available for all local users. Probably something like this.
-
Not acceptable on DSL. Would contact ISP for sure.
-
I'll try and arrange a night with no-one else using the connection to see what the spikes are like without any other traffic coming from our network. Saw this post when looking around which says PPPoE DSL can suffer packet loss when saturating the connection. Our connection is PPPoA but bridged to PPPoE via a Draytek Vigor 120. I'm assuming the congestion is at our exchange though as the total traffic speed is reduced at peak times.
-
Here's an example from peak times (but with only one computer being used within our network), it seems like even a single thread at peak dominates the connection to the exclusion of everything else, whereas when the connection is running as fast as our local exchange allows then multiple activities within our own network don't seem affected. I noticed this before that even if I limited a single threaded DownThemAll download to 10kBps it would basically lock up the connection for everyone else (and the computer running the download) but there'd be no problem even when running unlimited outside peak times. Does this seem feasible? Ping times are fine in both cases if that's all I'm running at the time.
Anyway, here's an example of just running a speedtest at the moment:
I ran an update to Calibre and while doing that here's an example of pings from the firewall to the gateway:
PING 222.153.64.1 (222.153.64.1): 56 data bytes
64 bytes from 222.153.64.1: icmp_seq=0 ttl=127 time=16357.440 ms
64 bytes from 222.153.64.1: icmp_seq=1 ttl=127 time=16757.918 ms
64 bytes from 222.153.64.1: icmp_seq=2 ttl=127 time=16978.610 ms
64 bytes from 222.153.64.1: icmp_seq=3 ttl=127 time=16900.429 ms–- 222.153.64.1 ping statistics ---
10 packets transmitted, 4 packets received, 60.0% packet loss
round-trip min/avg/max/stddev = 16357.440/16748.599/16978.610/239.296 msbut just running ping alone:
PING 222.153.64.1 (222.153.64.1): 56 data bytes
64 bytes from 222.153.64.1: icmp_seq=0 ttl=127 time=15.837 ms
64 bytes from 222.153.64.1: icmp_seq=1 ttl=127 time=15.738 ms
64 bytes from 222.153.64.1: icmp_seq=2 ttl=127 time=15.682 ms
64 bytes from 222.153.64.1: icmp_seq=3 ttl=127 time=18.351 ms
64 bytes from 222.153.64.1: icmp_seq=4 ttl=127 time=18.727 ms
64 bytes from 222.153.64.1: icmp_seq=5 ttl=127 time=15.443 ms
64 bytes from 222.153.64.1: icmp_seq=6 ttl=127 time=16.146 ms
64 bytes from 222.153.64.1: icmp_seq=7 ttl=127 time=26.404 ms
64 bytes from 222.153.64.1: icmp_seq=8 ttl=127 time=26.068 ms
64 bytes from 222.153.64.1: icmp_seq=9 ttl=127 time=26.754 ms--- 222.153.64.1 ping statistics ---
10 packets transmitted, 10 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 15.443/19.515/26.754/4.639 msIt doesn't seem to be pfSense getting overloaded in some way as pinging the modem itself still nets sub-second response times regardless.
I'll get back with some other results in a non-peak time.
-
Here's a non-peak speedtest:
Pings still go up during a download but don't get packet loss, the connection remains responsive for other tasks:
PING 222.153.64.1 (222.153.64.1): 56 data bytes
64 bytes from 222.153.64.1: icmp_seq=0 ttl=127 time=574.626 ms
64 bytes from 222.153.64.1: icmp_seq=1 ttl=127 time=684.047 ms
64 bytes from 222.153.64.1: icmp_seq=2 ttl=127 time=775.483 ms
64 bytes from 222.153.64.1: icmp_seq=3 ttl=127 time=1525.766 ms
64 bytes from 222.153.64.1: icmp_seq=4 ttl=127 time=1387.464 ms
64 bytes from 222.153.64.1: icmp_seq=5 ttl=127 time=2261.704 ms
64 bytes from 222.153.64.1: icmp_seq=6 ttl=127 time=3014.055 ms
64 bytes from 222.153.64.1: icmp_seq=7 ttl=127 time=3325.509 ms
64 bytes from 222.153.64.1: icmp_seq=8 ttl=127 time=3015.579 ms
64 bytes from 222.153.64.1: icmp_seq=9 ttl=127 time=3020.856 ms–- 222.153.64.1 ping statistics ---
10 packets transmitted, 10 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 574.626/1958.509/3325.509/1037.934 ms
-
Oh my… You need to change your ISP ASAP. In fact, yesterday was too late...
-
I agree - If your ISP were any slower, it would be Morse Code.
You might even consider switching to carrier pigeon over that service.
Latency is similar.
-
We're in a region that has no likelihood of improved fixed line service (the current service is meant to provide >1mbps speed, so most of the time it's better than we can expect). Changing ISP wouldn't achieve anything on the same exchange, and my ISPs bandwidth is generally considered one of the best in terms of consistency https://www.truenet.co.nz/articles/june-2013-broadband-report (Telecom). At some point in the future there'll be a greater than >5mbps wireless offering http://www.vodafone.co.nz/about/rural-broadband-initiative/
-
I wonder what latency is like on the BGAN I-4 Asia-Pacific? Can it be worse?
-
I wonder what latency is like on the BGAN I-4 Asia-Pacific? Can it be worse?
Round trip for satellites on geostationary orbit is something around 540 ms.
-
Clicking that link gave me some idea of what he is dealing with…
I clicked it and went and made a pot of coffee and a snack while the page loaded ;D
-
I clicked it and went and made a pot of coffee and a snack while the page loaded ;D
http://www.vodafone.co.nz/about/rural-broadband-initiative/Some parts of the page still not loaded after 2 minutes here… Edit: Wow, the "tower" image loaded just under 3 minutes...
P.S. Reading the page, I seriously don't think they did get it. The issue is absolutely not bandwidth. With similar latency, speed does not matter.
-
Their main server is running off this modem I think:
(I used to use this sort of rig for long haul encrypted digital comms. Just incase end of the world gear in my past jobs)
-
I think Squid and some sort of dynamic content caching system would be the most important thing for someone on a connection like this to have. I really do wish that for things like youtube, hulu, pandora and the whole plethora of semi-questionable content sites out there that pfsense had some sort of out of the box solution to cache them.
-
Their main server is running off this modem I think:
http://www.webpagetest.org/result/130721_PA_856/
Impressive. I think they'd better get out of business soon.
-
Beginnings are always an awkward place. I'm sure they will improve.
I think they should consider static high altitude "Blimp" like satellites for bandwidth.
It faster than towers or satellites and cheaper. Deploy overnight. Can carry enough payload for their entire country on a couple.
Its a no-brainer and yet - it rarely happens…
-
I wonder… who's the guy who happens to be in Nepal? Jim? Maybe he could compare that. Maybe moving to Nepal you'd be better off than at NZ, with a nice uncluttered view of Sagarmāthā as a bonus. 8) ;D
-
There are no good bars near an uncluttered view of Sagarmāthā. (and the internet sucks)
I prefer clutter.
-
Interesting to hear of how slow grabbing stuff from this part of the world is. Does anyone with the inclination want to try a speed test from either http://speedtest.telecom.co.nz/ or http://www.vodafone.co.nz/broadband/speedtest/ and see how it compares with your expectations?
-
On a sort of related note, you can check average speed for NZ or elsewhere at: http://www.akamai.com/stateoftheinternet/index.html#nui
-
I put an Alix pfSense in Mugu, Nepal on 192Kbps a few weeks ago. Their ping times to any real internet targets range from 900-1200ms, with some up to 3000ms. Packet loss (as reported on the dashboard, or on a typical ping for a few minutes) is nearly always 10-20%. The town telephone exchange is completely on a satellite link (no microwave towers on mountains yet) and is overloaded. They put in ADSL but they don't have enough satellite bandwidth at peak (or ordinary) times, so calls drop out, internet is slow…
But I have never seen 16000ms! I am rather surprised that a router anywhere would have a packet buffer that would hang on to packets for that long in a queue and finally transmit them on a slow link. In my case in Mugu, once the software has waited 5000ms for a reply it can give up - the packet has been dropped/lost somewhere in the sending and replying.
Good luck getting all those sheep online :)
-
http://www.vodafone.co.nz/broadband/speedtest/ and see how it compares with your expectations?
WiMax connection:
vs.
VDSL with two IPTV STBs (which causes the sucky ping)
vs.
-
I think Squid and some sort of dynamic content caching system would be the most important thing for someone on a connection like this to have. I really do wish that for things like youtube, hulu, pandora and the whole plethora of semi-questionable content sites out there that pfsense had some sort of out of the box solution to cache them.
Yeah, I find it frustrating that a lot of educational resources which would be of benefit in our local schools (which have the same poor bandwidth as us) are posted to Youtube. Which, firstly, seems to be a waste of bandwidth in terms of efficiency of imparting information - me, I prefer static text and diagrams, but most posters don't go to the trouble to make the content of their videos accessible by other means. Secondly, the reality is that they're on there and teachers expect that resource to be available but Youtube and their ilk aren't cache friendly. So then you have to decide whether to spend quite a bit on http://cachevideos.com/ to be able to utilise it easily.
-
No Java thanks. I'm trying to cut back.
Too much Java makes me jittery.
Yeah - that speed test sucks.
-
I put an Alix pfSense in Mugu, Nepal on 192Kbps a few weeks ago. Their ping times to any real internet targets range from 900-1200ms, with some up to 3000ms. Packet loss (as reported on the dashboard, or on a typical ping for a few minutes) is nearly always 10-20%. The town telephone exchange is completely on a satellite link (no microwave towers on mountains yet) and is overloaded. They put in ADSL but they don't have enough satellite bandwidth at peak (or ordinary) times, so calls drop out, internet is slow…
But I have never seen 16000ms! I am rather surprised that a router anywhere would have a packet buffer that would hang on to packets for that long in a queue and finally transmit them on a slow link. In my case in Mugu, once the software has waited 5000ms for a reply it can give up - the packet has been dropped/lost somewhere in the sending and replying.Thanks. Very interesting to hear from someone with hands-on experience from similar locations. Afraid VF NZ should be ashamed.
Good luck getting all those sheep online :)
ROTFLMAO! You owe me a new keyboard, mine if full of coffee now! ;D :D ;D
P.S. Yeah, squid even on a recycled old desktop machine would help significantly.
-
Good caching is such a universal need for most of the world, one would think that caching of dynamic content would be free (like free beer) and free as in freedom. You need it for sure. Most of the world does actually.
Squid is a nice tool and, contrary to many people's experiences, I've found that when configured as a transparent proxy it makes my browsing rock solid. Especially Hulu. God only knows why because its not caching hulu content, but its undeniable. No endless circles or pauses when I'm running squid. Very strange.But this guy - He needs alot better and inclusive cache than a standard squid proxy. He needs something that sucks in and and saves everything for later.
-
I just wrote that company a note on how they should give it away for free to schools in bandwidth deprived areas.
Who knows if they will reply…
-
Good luck getting all those sheep online :)
Put them nose to tail, a bit of no.8 wire, and I'm sure we could use some of that dark fibre
-
Just wondering if anyone else can post results from pinging their gateway from a pfSense shell while running a sustained download (e.g. pick a fairly large size from http://testmy.net/download ) to see if there's any noticeable difference to their normal results?
-
icmp_req=119 ttl=64 time=0.304 ms pinging my public IP
icmp_req=34 ttl=64 time=0.391 ms pinging my LAN ipDuring a 14.5 Mbps download.
Its pretty much the same downloading or not. No difference for me.
Following instructions this time, I pinged my gateway IP (-;
icmp_req=18 ttl=126 time=8.11 ms Not dowloading
icmp_req=28 ttl=126 time=88.5 ms Downloading (MOCA is a POS when its loaded down. I plan to connect directly to ONT at this location soon)
-
Just wondering if anyone else can post results from pinging their gateway from a pfSense shell while running a sustained download (e.g. pick a fairly large size from http://testmy.net/download ) to see if there's any noticeable difference to their normal results?
I guess not. 2.1RC0 on Alix:
PING www.google.com (173.194.35.84): 56 data bytes 64 bytes from 173.194.35.84: icmp_seq=0 ttl=58 time=3.096 ms 64 bytes from 173.194.35.84: icmp_seq=1 ttl=58 time=2.447 ms 64 bytes from 173.194.35.84: icmp_seq=2 ttl=58 time=2.657 ms 64 bytes from 173.194.35.84: icmp_seq=3 ttl=58 time=2.726 ms 64 bytes from 173.194.35.84: icmp_seq=4 ttl=58 time=2.544 ms 64 bytes from 173.194.35.84: icmp_seq=5 ttl=58 time=2.655 ms 64 bytes from 173.194.35.84: icmp_seq=6 ttl=58 time=2.330 ms 64 bytes from 173.194.35.84: icmp_seq=7 ttl=58 time=2.720 ms 64 bytes from 173.194.35.84: icmp_seq=8 ttl=58 time=2.245 ms 64 bytes from 173.194.35.84: icmp_seq=9 ttl=58 time=2.499 ms
-
Crap MOCA introduces a heap of latency. Whichever idiot at verizon got the brilliant idea of wedging coax MOCA between between the router and a gigabit ethernet port on an optical connection needs to be shot.
Maybe we can all try this to get faster internet
I'm getting out my scissors and tape now… haha (do not! its a joke)
-
And do you run with or without traffic shaping?
-
Here yes. Without. This is just home. I only have traffic shaping running on the boxes others use for VPN access.
I used to have a 150 plan here, and it was nice but totally a waste here since the only servers running out of here are chat, phone, private VPN and other small things. Nothing that ever touched my bandwidth. So, I just moved it back to cheap old reliable FIOS rather than keep paying for bandwidth I never touched. Mostly here what I need is less latency not more bandwidth. I'll eliminate MOCA and should be all good.