<![CDATA[MPLS to the extreme]]>The internet has been strangely stable for a while. Generally there is some sort of routing issues that pops up. I decided to start doing trace routes and looking glasses and here are my results from different looking glass services....

Cogent San Francisco
1 gi0-0-0-7.221.agr22.sfo01.atlas.cogentco.com (66.250.250.145) 0.580 ms 0.581 ms
2 be2905.ccr22.sfo01.atlas.cogentco.com (154.54.30.221) 0.718 ms 0.825 ms
3 be3670.ccr41.sjc03.atlas.cogentco.com (154.54.43.14) 1.975 ms 1.981 ms
4 4.68.110.137 (4.68.110.137) 1.720 ms 1.733 ms
5 * *
6 redacted.Chicago3.Level3.net (redacted) 62.492 ms 62.460 ms
7 ISP (redacted) 61.761 ms 61.732 ms

Cogent Frankfurt, Germany
1 gi0-7-1-9.6.agr22.fra03.atlas.cogentco.com (130.117.254.33) 0.719 ms 0.725 ms
2 be2534.ccr42.fra03.atlas.cogentco.com (130.117.48.210) 0.553 ms 0.573 ms
3 be2845.rcr22.fra06.atlas.cogentco.com (154.54.56.190) 1.074 ms be2846.rcr22.fra06.atlas.cogentco.com (154.54.37.30) 0.730 ms
4 * *
5 * *
6 redacted.Chicago3.Level3.net (redacted) 117.779 ms 117.075 ms
7 ISP (redacted) 117.287 ms 117.303 ms

Hurricane Electric San Jose, CA
1 14 ms 14 ms 14 ms 100ge15-1.core1.slc1.he.net (184.105.223.214)
2 27 ms 27 ms 27 ms 100ge9-1.core1.den1.he.net (72.52.92.41)
3 61 ms 43 ms 58 ms 100ge8-1.core1.msp1.he.net (184.105.222.94)
4 43 ms 43 ms 43 ms redacted.msp1.he.net (184.105.63.222)
5 50 ms 50 ms 51 ms mpls.ISP.net (redacted)

Hurricane Electric Dallas, TX
1 10 ms 27 ms 11 ms 100ge12-2.core1.mci3.he.net (184.105.81.205)
2 15 ms 14 ms 14 ms 100ge9-2.core1.oma1.he.net (184.105.65.166)
3 41 ms 24 ms 21 ms 100ge8-1.core1.blp1.he.net (184.105.65.98)
4 23 ms 33 ms 23 ms 100ge8-2.core1.msp1.he.net (184.105.64.97)
5 25 ms 23 ms 27 ms mpls.ISP.net (redacted)

Hurricane Electric Dublin, DE
1 72 ms 61 ms 59 ms 100ge5-2.core1.nyc5.he.net (184.105.65.245)
2 78 ms 71 ms 60 ms 100ge4-2.core1.nyc4.he.net (184.105.213.217)
3 79 ms 90 ms 98 ms 100ge9-1.core2.chi1.he.net (184.105.223.161)
4 100 ms 99 ms 100 ms 100ge15-2.core1.chi1.he.net (184.104.192.117)
5 99 ms 104 ms 95 ms 100ge8-1.core1.msn1.he.net (184.105.65.226)
6 99 ms 102 ms 98 ms mpls.ISP.net (redacted)

Hurricane Electric Tokyo, Japan
1 81 ms 81 ms 81 ms 100ge11-1.core1.sea1.he.net (184.105.213.117)
2 117 ms 115 ms 114 ms 100ge4-2.core1.msp1.he.net (184.105.223.194)
3 114 ms 117 ms 114 ms mpls.ISP.net (redacted)

Blizzard Battle.Net Auth USA
1 Blizzard Blizzard 0.300 ms 0.270 ms 1.122 ms
2 24.105.18.131 (24.105.18.131) 0.532 ms 0.669 ms 0.726 ms <-- Blizzard
3 137.221.105.16 (137.221.105.16) 0.598 ms 0.729 ms 0.835 ms <-- Blizzard
4 137.221.66.22 (137.221.66.22) 1.101 ms 1.102 ms 1.103 ms <-- Blizzard
5 137.221.83.68 (137.221.83.68) 12.921 ms 12.940 ms 12.942 ms <-- Blizzard
6 137.221.65.68 (137.221.65.68) 45.266 ms 44.073 ms 44.045 ms <-- Blizzard
7 137.221.65.1 (137.221.65.1) 62.689 ms 62.700 ms 62.680 ms <-- Blizzard
8 137.221.65.7 (137.221.65.7) 12.922 ms 12.935 ms 12.931 ms <-- Blizzard
9 137.221.70.34 (137.221.70.34) 13.023 ms 13.065 ms 13.038 ms <-- Blizzard
10 * * *
11 mpls.ISP.net (redacted) 64.848 ms 64.920 ms 65.184 ms

Blizzard Battle.Net Auth Europe
1 Blizzard Blizzard 0.356 ms 0.342 ms 0.342 ms
2 37.244.24.2 (37.244.24.2) 0.571 ms 0.632 ms 0.712 ms
3 Blizzard Blizzard 1.420 ms 1.632 ms 1.680 ms
4 137.221.66.40 (137.221.66.40) 1.148 ms 1.162 ms 1.160 ms
5 137.221.78.70 (137.221.78.70) 95.669 ms 95.690 ms 95.690 ms
6 137.221.65.90 (137.221.65.90) 110.372 ms 108.572 ms 108.524 ms
7 137.221.65.24 (137.221.65.24) 95.623 ms 95.660 ms 95.615 ms
8 137.221.65.8 (137.221.65.8) 95.603 ms 95.599 ms 95.596 ms
9 137.221.69.34 (137.221.69.34) 95.306 ms 95.388 ms 95.480 ms
10 * * *
11 mpls.ISP.net (redacted) 101.388 ms 101.365 ms 101.416 ms

CentryLink Frankfurt, DE
1 0.0.0.0 (0.0.0.0) * * *
2 redacted.Chicago3.Level3.net (redacted) 115ms 115ms 115ms
3 mpls.ISP.net (redacted) 114ms 115ms 115ms

CentryLink Capetown, South Africa
1 0.0.0.0 (0.0.0.0) * * *
2 redacted.Chicago3.Level3.net (redacted) 240ms 240ms 240ms
3 mpls.ISP.net (redacted) 240ms 240ms 240ms

Turns out my ISP decided to start announcing itself around the world at major IXs and special routing back over several transit providers. Pretty much all of my trace routes are like this

1 <1 ms <1 ms <1 ms pfsense.localdomain [10.255.42.1]
2 <1 ms <1 ms <1 ms 192.168.1.1
3 2 ms 1 ms 2 ms redacted
4 2 ms 2 ms 2 ms redacted [redacted]
5 3 ms 2 ms 2 ms ISP [redacted]
6 13 ms 13 ms 13 ms 4.71.102.197
7 * * * Request timed out.
8 139 ms 139 ms 139 ms 212.73.243.150 <-- Level3 Paris
9 173 ms 147 ms 148 ms 52.95.60.20 <-- AWS Ireland

It all looks whack to me. But performance has been really really good. Seems like even when there is a major outage in my region, I'm mostly unaffected.

]]>https://forum.netgate.com/topic/144537/mpls-to-the-extremeRSS for NodeSat, 13 Jun 2026 19:12:05 GMTSat, 29 Jun 2019 02:36:09 GMT60<![CDATA[Reply to MPLS to the extreme on Wed, 03 Jul 2019 20:27:14 GMT]]>@stephenw10 Seemingly peering at major IXPs around the USA, but it's a small local ISP. And it's not just a gimmick. The bandwidth is very stable and these routes seem to continue to function even when there's regional internet outages.

Of course with enough money, you can get your own fiber, but we're talking about each IX being thousands of miles away and the podunk ISP has ridiculously low prices. Private MPLS comes to mind, but I have always assumed that to be very expensive and reserved for high tech companies like Google or Amazon where latency matters.

I do enjoy when situations like "Battle.Net is down for the entire Midwest due to routing issues" and I'm unaffected. And peering disputes seem to be a thing of the past. It seemed like once every few months, some congestion issue between transit providers would occur in some of my routes. But now that most everything is just direct peering and all of the traffic is effectively "tunneled" over from private route, it's been an issue of the past.

Even without IX peering. Pick a game server on the west coast

1 <1 ms <1 ms <1 ms pfsense.localdomain [10.255.42.1]
2 <1 ms <1 ms <1 ms 192.168.1.1
3 2 ms 2 ms 2 ms redacted
4 2 ms 2 ms 2 ms redacted
5 3 ms 2 ms 2 ms ISP redacted
6 13 ms 13 ms 13 ms 4.71.102.197
7 62 ms 62 ms 62 ms 4.69.202.241
8 62 ms 62 ms 62 ms 4.28.172.102
9 62 ms 62 ms 61 ms 159.153.68.252

Routes look a lot different to me from just a year ago.

]]>https://forum.netgate.com/post/851510https://forum.netgate.com/post/851510Wed, 03 Jul 2019 20:27:14 GMT<![CDATA[Reply to MPLS to the extreme on Sun, 30 Jun 2019 18:49:33 GMT]]>What pattern exactly are we looking at here?

]]>https://forum.netgate.com/post/851017https://forum.netgate.com/post/851017Sun, 30 Jun 2019 18:49:33 GMT<![CDATA[Reply to MPLS to the extreme on Sat, 29 Jun 2019 16:59:50 GMT]]>Is there a term for this kind of pattern?

]]>https://forum.netgate.com/post/850907https://forum.netgate.com/post/850907Sat, 29 Jun 2019 16:59:50 GMT