Routing Loop

Hello,
Guyz please try to reach my network 194.9.82.0/24 from your networks.
Am seeeing routing loops from several looking glasses.
above.net, Alameda.net. but from traceroute.eu. the block comes down ok.
Kindly anyone assist.

Uh, still seems to be unhappy. Have you tried poking the above.net NOC?

Adrian

adrian@kandy:~$ traceroute 194.9.82.1
traceroute to 194.9.82.1 (194.9.82.1), 30 hops max, 40 byte packets
1 192.168.1.1 (192.168.1.1) 1.379 ms 1.332 ms 1.339 ms
2 gw.cacheboy.net (203.56.15.73) 3.883 ms 4.199 ms 3.798 ms
3 green.westnet.net.au (203.56.14.7) 39.615 ms 36.898 ms 36.533 ms
4 rtr-v1-blue-1.westnet.net.au (203.56.14.1) 36.139 ms 44.072 ms 39.037 ms
5 bdr1.perth.westnet.net.au (203.56.14.46) 35.067 ms 36.074 ms 37.672 ms
6 CORE-203-190-192-249.auto.conceptual.net.au (203.190.192.249) 38.465 ms 52 .764 ms 39.663 ms
7 59.154.14.81 (59.154.14.81) 57.525 ms 48.933 ms 40.739 ms
8 203.208.192.241 (203.208.192.241) 257.458 ms 257.153 ms 259.887 ms
9 ge-0-0-0-0.plapx-dr1.ix.singtel.com (203.208.149.1) 263.012 ms 264.251 ms ge-4-0-0-0.plapx-dr1.ix.singtel.com (203.208.149.5) 258.881 ms
10 ge-5-2-1.mpr4.pao1.us.above.net (64.125.13.5) 260.073 ms 261.251 ms 265.6 42 ms
11 so-1-2-2.mpr2.sjc2.us.above.net (64.125.29.126) 260.772 ms 260.820 ms 263 .196 ms
12 so-0-0-0.mpr1.sjc2.us.above.net (64.125.27.245) 267.666 ms 261.984 ms 279 .690 ms
13 so-1-0-0.mpr1.lga5.above.net (64.125.26.230) 337.744 ms 343.318 ms 337.64 6 ms
14 so-1-0-0.mpr1.ams1.nl.above.net (64.125.27.186) 426.055 ms 424.355 ms 440 .676 ms
15 ten-gige-1-1.mpr1.ams2.nl.above.net (64.125.26.73) 427.159 ms 424.206 ms 431.439 ms
16 ten-gige-2-2.mpr2.ams2.nl.above.net (64.125.26.70) 423.779 ms 425.797 ms 428.981 ms
17 pni-verio.ams2.nl.above.net (82.98.247.10) 416.403 ms 415.042 ms 417.738 ms
18 pni-verio.ams2.nl.above.net (82.98.247.9) 428.718 ms 430.827 ms 428.914 m s
19 pni-verio.ams2.nl.above.net (82.98.247.10) 418.211 ms 419.692 ms 417.855 ms
20 pni-verio.ams2.nl.above.net (82.98.247.9) 428.804 ms 432.612 ms 424.800 m s
21 pni-verio.ams2.nl.above.net (82.98.247.10) 418.601 ms 417.696 ms 449.603 ms
22 pni-verio.ams2.nl.above.net (82.98.247.9) 428.716 ms 428.140 ms 428.684 m s
23 pni-verio.ams2.nl.above.net (82.98.247.10) 418.273 ms 418.931 ms 473.879 ms
24 pni-verio.ams2.nl.above.net (82.98.247.9) 554.731 ms 591.014 ms 467.694 m s
25 pni-verio.ams2.nl.above.net (82.98.247.10) 431.515 ms 419.261 ms 431.965 ms
26 pni-verio.ams2.nl.above.net (82.98.247.9) 429.947 ms 433.778 ms 426.749 m s
27 pni-verio.ams2.nl.above.net (82.98.247.10) 419.974 ms 418.109 ms 421.930 ms
28 pni-verio.ams2.nl.above.net (82.98.247.9) 427.184 ms 435.642 ms 430.471 ms
29 pni-verio.ams2.nl.above.net (82.98.247.10) 422.111 ms 420.085 ms 423.038 ms
30 pni-verio.ams2.nl.above.net (82.98.247.9) 426.140 ms 429.449 ms 429.056 ms
adrian@kandy:~$

Tracing route to uu-194-009-082-001.uunet.co.ke [194.9.82.1] over a maximum
of 30 hops:

  1 8 ms 8 ms 7 ms ssrb230a-vpn-3080-1-e2.ns.ufl.edu
[128.227.166.116]
  2 8 ms 7 ms 8 ms 128.227.252.109
  3 7 ms 7 ms 7 ms ssrb230a-ewan-msfc-1-v704-1.ns.ufl.edu
[128.227.252.50]
  4 34 ms 37 ms 50 ms jax-flrcore-7609-1-te31-1806.net.flrnet.org
[198.32.155.93]
  5 33 ms 33 ms 33 ms te2-1--583.tr01-asbnva01.transitrail.net
[137.164.131.165]
  6 32 ms 33 ms 32 ms abovenet-peer.asbnva01.transitrail.net
[137.164.130.50]
  7 33 ms 33 ms 33 ms ge-3-0-0.mpr2.dca2.above.net [64.125.26.241]
  8 35 ms 35 ms 35 ms so-0-0-0.mpr1.dca2.us.above.net
[64.125.26.1]
  9 107 ms 105 ms 105 ms so-6-0-0.cr1.lhr3.uk.above.net
[64.125.31.185]
10 120 ms 119 ms 119 ms so-1-0-0.mpr2.ams5.nl.above.net
[64.125.27.178]
11 142 ms 141 ms 141 ms ten-gige-1-1.mpr2.ams2.nl.above.net
[64.125.26.77]
12 129 ms 127 ms 125 ms pni-verio.ams2.nl.above.net [82.98.247.10]
13 142 ms 141 ms 141 ms pni-verio.ams2.nl.above.net [82.98.247.9]
14 126 ms 125 ms 126 ms pni-verio.ams2.nl.above.net [82.98.247.10]
15 142 ms 141 ms 141 ms pni-verio.ams2.nl.above.net [82.98.247.9]
16 127 ms 125 ms 126 ms pni-verio.ams2.nl.above.net [82.98.247.10]
17 142 ms 141 ms 141 ms pni-verio.ams2.nl.above.net [82.98.247.9]
18 126 ms 127 ms 126 ms pni-verio.ams2.nl.above.net [82.98.247.10]
19 141 ms 142 ms 141 ms pni-verio.ams2.nl.above.net [82.98.247.9]
20 127 ms 129 ms 126 ms pni-verio.ams2.nl.above.net [82.98.247.10]
21 142 ms 142 ms 143 ms pni-verio.ams2.nl.above.net [82.98.247.9]
22 128 ms ^C

Still happening 9:00 AM EDST -4

Robert D. Scott Robert@ufl.edu
Senior Network Engineer 352-273-0113 Phone
CNS - Network Services 352-392-2061 CNS Receptionist
University of Florida 352-392-9440 FAX
Florida Lambda Rail 352-294-3571 FLR NOC
Gainesville, FL 32611

Guyz,
Does anyone know what i can do to expediate Above.Net to fix this issue quickly.
There Noc they said they were checking now for more than 4 hrs

Regards
Felix

Robert D. Scott wrote:

Are you talking to them, or your upstream provider? If your provider is not
yet engaged do so.

Above net is doing something, the loop is bigger now. 11:30 EDST -4

11 156 ms 208 ms 224 ms 64.125.26.77
12 117 ms 116 ms 117 ms 64.125.26.69
13 119 ms 121 ms 125 ms 64.125.26.74
14 146 ms 141 ms 141 ms 64.125.26.73
15 148 ms 159 ms 160 ms 64.125.26.70
16 119 ms 117 ms 117 ms 64.125.26.69
17 121 ms 119 ms 118 ms 64.125.26.74
18 142 ms 142 ms 142 ms 64.125.26.73
19 281 ms 215 ms 222 ms 64.125.26.70

Robert

Guyz,
Does anyone know what i can do to expediate Above.Net to fix this issue quickly.
There Noc they said they were checking now for more than 4 hrs

I see it differently from here:

traceroute to 194.9.82.1 (194.9.82.1), 30 hops max, 40 byte packets
  1 72.1.130.1 (72.1.130.1) 15.050 ms 14.517 ms 15.313 ms
  2 69.17.82.237 (69.17.82.237) 12.836 ms 12.252 ms 12.533 ms
  3 * * *
  4 217.194.157.226 (217.194.157.226) 716.452 ms 716.280 ms 715.706 ms
  5 217.194.157.225 (217.194.157.225) 717.818 ms 716.473 ms 716.547 ms
  6 41.207.64.245 (41.207.64.245) 708.166 ms 706.566 ms 732.847 ms
  7 41.207.64.254 (41.207.64.254) 731.986 ms 728.693 ms 725.945 ms
  8 41.207.64.253 (41.207.64.253) 722.842 ms 718.985 ms 716.708 ms
  9 41.207.64.254 (41.207.64.254) 717.871 ms 714.942 ms 714.001 ms
10 41.207.64.253 (41.207.64.253) 712.438 ms 710.607 ms 713.404 ms
...
28 41.207.64.253 (41.207.64.253) 798.511 ms 796.038 ms 792.762 ms
29 41.207.64.254 (41.207.64.254) 794.451 ms 795.007 ms 892.323 ms
30 41.207.64.253 (41.207.64.253) 889.490 ms 885.750 ms 881.745 ms

morannon:~>host 41.207.64.253
253.64.207.41.in-addr.arpa domain name pointer ge0-1-br.nbo.infinet.co.ke.
morannon:~>host 41.207.64.254
254.64.207.41.in-addr.arpa domain name pointer ge0-2-pdsn.nbo.infinet.co.ke.

  - d.

> Does anyone know what i can do to expediate Above.Net to fix this issue
> quickly.
> There Noc they said they were checking now for more than 4 hrs

I see it differently from here:

From here (Oslo, Norway, Level3 as one of our transit providers) it

works fine - I can even ping 194.9.82.137.

traceroute to 194.9.82.137 (194.9.82.137), 64 hops max, 40 byte packets
1 nethelp-gw (195.1.209.46) 1.037 ms 1.136 ms 1.129 ms
2 ge-0-0-0-10.ar1.skoey.no.catchbone.net (81.0.130.190) 41.408 ms 99.281 ms 18.974 ms
3 ge-0-0-2.cr2.osls.no.catchbone.net (217.8.128.125) 14.076 ms 17.252 ms 16.724 ms
4 c10G-ge-6-1-0.br1.osls.no.catchbone.net (81.0.128.82) 16.408 ms 11.944 ms 17.357 ms
5 213.242.110.25 (213.242.110.25) 20.627 ms 24.517 ms 22.185 ms
6 ae-4-4.ebr2.Dusseldorf1.Level3.net (4.69.135.22) 57.604 ms 53.915 ms 53.915 ms
7 ae-2.ebr1.Amsterdam1.Level3.net (4.69.133.89) 49.335 ms 51.415 ms 54.128 ms
8 ae-1-100.ebr2.Amsterdam1.Level3.net (4.69.133.86) 50.000 ms 53.604 ms 54.647 ms
9 ae-2.ebr2.London1.Level3.net (4.69.132.133) 60.138 ms 57.248 ms 55.036 ms
10 ae-24-56.car3.London1.Level3.net (4.68.116.177) 55.117 ms
    ae-24-52.car3.London1.Level3.net (4.68.116.49) 60.006 ms
    ae-24-54.car3.London1.Level3.net (4.68.116.113) 62.998 ms
11 195.50.113.18 (195.50.113.18) 116.916 ms 118.000 ms 116.558 ms
12 217.194.157.226 (217.194.157.226) 630.275 ms 658.066 ms 631.207 ms
13 217.194.157.225 (217.194.157.225) 635.765 ms 654.525 ms 648.362 ms
14 fe2-0-br.nbo.infinet.co.ke (41.207.64.245) 622.878 ms 617.356 ms 643.694 ms
15 ge0-2-pdsn.nbo.infinet.co.ke (41.207.64.254) 623.575 ms 620.682 ms 755.128 ms
16 uu-194-009-082-137.uunet.co.ke (194.9.82.137) 703.257 ms 696.020 ms 709.526 ms

% ping 194.9.82.137
PING 194.9.82.137 (194.9.82.137): 56 data bytes
64 bytes from 194.9.82.137: icmp_seq=0 ttl=107 time=683.480 ms
64 bytes from 194.9.82.137: icmp_seq=1 ttl=107 time=681.910 ms
64 bytes from 194.9.82.137: icmp_seq=2 ttl=107 time=684.992 ms
64 bytes from 194.9.82.137: icmp_seq=3 ttl=107 time=703.734 ms
64 bytes from 194.9.82.137: icmp_seq=4 ttl=107 time=676.809 ms

Steinar Haug, Nethelp consulting, sthaug@nethelp.no

I'm seeing an inconsistent origin AS for this /24 from Above.net and PCCW.

# show bgp 194.9.82.0/24
[snip]
  3491 25228 25228 25228 25228 25228 36915
    63.218.31.1 from 67.59.145.6 (192.168.254.6)
      Origin IGP, metric 0, localpref 100, valid, internal
      Community: 3491:300 25228:2203 25228:2221 25228:2249 25228:2253 25228:2263 25228:2310
      Last update: Sat Mar 15 15:00:08 2008
[snip]
  6461
    64.124.78.26 from 67.59.145.7 (192.168.254.7)
      Origin IGP, metric 0, localpref 100, valid, internal, best
      Community: 6461:5999
      Last update: Sat Mar 15 05:53:40 2008

In general, Above.net has been really awesome, but their BGP has been
screwing pooch on and off this month.

Ross

Seeing slightly different results from here. Aside from the fact the
network I'm on is a little slow,

Tracing route to uu-194-009-082-001.uunet.co.ke [194.9.82.1]
over a maximum of 30 hops:

  1 11 ms 2 ms 1 ms 192.168.15.1
  2 * * * Request timed out.
  3 21 ms * 16 ms
ge-1-19-ur01.burlington.wa.seattle.comcast.net [68.86.99.161]
  4 22 ms * 13 ms
te-5-1-ur01.ferndale.wa.seattle.comcast.net [68.86.96.113]
  5 * * 17 ms
te-5-3-ur02.ferndale.wa.seattle.comcast.net [68.86.96.110]
  6 24 ms 15 ms 14 ms te [68.86.96.105]
  7 22 ms 16 ms 16 ms COMCAST-IP.car1.Seattle1.Level3.net
[4.79.104.110]
  8 419 ms 15 ms 23 ms te-3-3.car1.Seattle1.Level3.net
[4.79.104.109]
  9 33 ms 18 ms 15 ms ae-31-53.ebr1.Seattle1.Level3.net
[4.68.105.94]
10 24 ms 17 ms 19 ms ae-1-100.ebr2.Seattle1.Level3.net
[4.69.132.18]
11 93 ms 88 ms 114 ms ae-2.ebr2.Denver1.Level3.net
[4.69.132.54]
12 111 ms 204 ms 101 ms ae-3.ebr3.Chicago1.Level3.net
[4.69.132.62]
13 194 ms 106 ms 204 ms ae-68.ebr1.Chicago1.Level3.net
[4.69.134.57]
14 201 ms 203 ms 101 ms ae [4.69.132.66]
15 240 ms 203 ms 101 ms ae-72-72.csw2.NewYork1.Level3.net
[4.69.134.86]
16 122 ms 204 ms 101 ms ae-71-71.ebr1.NewYork1.Level3.net
[4.69.134.69]
17 429 ms 203 ms 408 ms ae-42.ebr2.London1.Level3.net
[4.69.137.69]
18 430 ms 203 ms 306 ms ae-24-52.car3.London1.Level3.net
[4.68.116.49]
19 477 ms 362 ms 306 ms 195.50.113.18
20 948 ms 817 ms 990 ms 217.194.157.226
21 1016 ms 920 ms 772 ms 217.194.157.225
22 803 ms 1023 ms 920 ms fe2-0-br.nbo.infinet.co.ke
[41.207.64.245]
23 845 ms 778 ms 942 ms ge0-2-pdsn.nbo.infinet.co.ke
[41.207.64.254]
24 773 ms 866 ms 770 ms ge0-1-br.nbo.infinet.co.ke
[41.207.64.253]
25 997 ms 921 ms 920 ms ge0-2-pdsn.nbo.infinet.co.ke
[41.207.64.254]
26 839 ms 920 ms 925 ms ge0-1-br.nbo.infinet.co.ke
[41.207.64.253]
27 840 ms 920 ms 921 ms ge0-2-pdsn.nbo.infinet.co.ke
[41.207.64.254]
28 840 ms 921 ms 920 ms ge0-1-br.nbo.infinet.co.ke
[41.207.64.253]
29 839 ms 920 ms 920 ms ge0-2-pdsn.nbo.infinet.co.ke
[41.207.64.254]
30 839 ms 920 ms 920 ms ge0-1-br.nbo.infinet.co.ke
[41.207.64.253]

Trace complete.

Felix:

There's still a routing look at above.net, as documented by others and the
other listserv you posted this on (cisco-nsp?).

....
12 76 ms 73 ms 79 ms chp-brdr-01.inet.qwest.net [205.171.139.150]
13 78 ms 77 ms 75 ms so-4-1-0.mpr2.ord7.us.above.net
[64.125.12.149]
14 82 ms 83 ms 78 ms so-0-1-0.mpr1.ord2.us.above.net
[64.125.30.146]
15 102 ms 102 ms 101 ms so-1-1-0.mpr1.lga5.us.above.net
[64.125.27.170]
16 185 ms 189 ms 190 ms so-1-0-0.mpr1.ams1.nl.above.net
[64.125.27.186]
17 187 ms 187 ms 191 ms ten-gige-1-1.mpr1.ams2.nl.above.net
[64.125.26.73]
18 199 ms 214 ms 223 ms ten-gige-2-2.mpr2.ams2.nl.above.net
[64.125.26.70]
19 190 ms 187 ms 188 ms ten-gige-2-2.mpr1.ams2.nl.above.net
[64.125.26.69]
20 189 ms 191 ms 191 ms ge-1-2-0.mpr1.ams1.nl.above.net
[64.125.26.74]
21 185 ms 189 ms 186 ms ten-gige-1-1.mpr1.ams2.nl.above.net
[64.125.26.73]
22 188 ms 187 ms 183 ms ten-gige-2-2.mpr2.ams2.nl.above.net
[64.125.26.70]
23 186 ms 188 ms 186 ms ten-gige-2-2.mpr1.ams2.nl.above.net
[64.125.26.69]
24 186 ms 187 ms 188 ms ge-1-2-0.mpr1.ams1.nl.above.net
[64.125.26.74]
25 191 ms 194 ms 190 ms ten-gige-1-1.mpr1.ams2.nl.above.net
[64.125.26.73]
26 185 ms 184 ms 189 ms ten-gige-2-2.mpr2.ams2.nl.above.net
[64.125.26.70]
27 188 ms 191 ms 187 ms ten-gige-2-2.mpr1.ams2.nl.above.net
[64.125.26.69]
28 188 ms 191 ms 187 ms ge-1-2-0.mpr1.ams1.nl.above.net
[64.125.26.74]
29 186 ms 192 ms 196 ms ten-gige-1-1.mpr1.ams2.nl.above.net
[64.125.26.73]
30 188 ms 187 ms 188 ms ten-gige-2-2.mpr2.ams2.nl.above.net
[64.125.26.70]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu] On Behalf Of
Felix Bako
Sent: Saturday, March 15, 2008 2:11 AM
To: nanog@merit.edu
Subject: Routing Loop

Guyz please try to reach my network 194.9.82.0/24 from your networks.
Am seeeing routing loops from several looking glasses.
above.net, Alameda.net. but from traceroute.eu. the block comes down ok.
Kindly anyone assist.

Here's what I'm seeing going via my Cogent connection in Denver, CO, US:

HOST: smtp1.spameater.net Loss% Snt Last Avg Best Wrst StDev
  1. 38.116.133.1 0.0% 10 0.6 0.6 0.4 0.9 0.1
  2. 38.112.6.145 0.0% 10 1.1 0.9 0.7 1.1 0.2
  3. 66.250.10.217 0.0% 10 1.4 28.9 0.6 133.9 49.1
  4. 154.54.5.142 0.0% 10 0.9 1.6 0.7 6.2 1.6
  5. 154.54.24.82 0.0% 10 12.1 13.2 11.8 23.4 3.6
  6. 154.54.7.186 0.0% 10 24.2 26.7 23.7 51.7 8.8
  7. 154.54.5.18 0.0% 10 24.1 24.2 24.1 24.6 0.2
  8. 154.54.11.58 0.0% 10 23.7 26.3 23.7 42.6 5.8
  9. 64.125.30.142 0.0% 10 30.0 26.2 24.6 30.0 1.9
10. 64.125.27.34 0.0% 10 57.1 63.3 55.7 104.4 15.9
11. 64.125.27.237 0.0% 10 56.6 60.8 56.3 85.2 8.8
12. 64.125.27.186 0.0% 10 135.5 139.3 134.4 158.3 8.2
13. 64.125.26.73 0.0% 10 134.9 135.6 134.9 136.3 0.5
14. 64.125.26.70 0.0% 10 134.7 134.7 134.6 135.1 0.1
15. 64.125.26.69 0.0% 10 136.4 135.9 134.8 136.4 0.5
16. 64.125.26.74 0.0% 10 139.6 142.3 134.8 198.3 19.8
17. 64.125.26.73 0.0% 10 138.5 136.5 135.5 138.5 0.8
18. 64.125.26.70 0.0% 10 238.3 145.7 135.2 238.3 32.5
19. 64.125.26.69 0.0% 10 136.4 136.4 135.3 137.0 0.6
20. 64.125.26.74 0.0% 10 135.6 136.1 135.2 141.8 2.0
21. 64.125.26.73 0.0% 10 136.9 136.7 135.9 137.5 0.6
22. 64.125.26.70 0.0% 10 136.3 136.0 135.7 136.3 0.2
23. 64.125.26.69 0.0% 10 137.8 137.5 135.9 139.9 1.1
24. 64.125.26.74 0.0% 10 135.9 138.0 135.7 143.9 3.5
25. 64.125.26.73 0.0% 10 137.9 137.7 137.3 138.3 0.3
26. 64.125.26.70 0.0% 10 136.7 136.7 136.2 139.0 0.8
27. 64.125.26.69 10.0% 10 137.9 138.7 136.9 147.2 3.2
28. 64.125.26.74 10.0% 10 136.6 137.6 136.3 142.9 2.3
29. 64.125.26.73 10.0% 10 138.2 138.1 137.7 138.6 0.3
30. 64.125.26.70 10.0% 10 137.1 137.1 136.8 137.3 0.2

There's also somewhat odd data in RADB (look at the changed: line):

route: 194.9.64.0/19
descr: SES-Newskies Customer Prefix
origin: AS16422
remarks: SES-Newskies Customer Prefix
notify: noc@ses-newskies.com
mnt-by: MNT-NWSK
changed: noc@ses-newskies.com 20080314
source: ARIN

This is in the middle of RIPE-managed swamp space, a /19 definitely
doesn't belong there.

Yeah, I saw that a bit earlier and it did seem incredibly suspicious
given the timing. Had I seen anything in the routing system itself for
that explicit /19 related to this I would mentioned it, but nothing there.

Amazingly, a query to the NOC at SES Newskies yielded a near-
immediate response, which said they added it a few days back because
they were updating some policies and noticed it existed in a prefix list for
one of their upstreams, that it appeared to be legacy, and that they'd get
it removed.

All the more reason RIR allocation authentication used to seed IRR
information would be of value for routing policy specification, let alone
informational purposes.

-danny

7018 is still seeing announcements from 6461,
and the Oregon Routeviews server route-views.routeviews.org also sees
many announcements
from different ISPs seeing it announced from 6461.

The whois entry for Above.net lists the NOC as
   RTechHandle: NOC41-ORG-ARIN
   RTechName: AboveNet NOC
   RTechPhone: +1-877-479-7378
   RTechEmail: noc@above.net

wrote to NANOG

I see this at 9:10pm PST:

traceroute uu-194-009-082-001.uunet.co.ke
traceroute to uu-194-009-082-001.uunet.co.ke (194.9.82.1), 64 hops max, 40 byte packets
  1 64-192-2-193.static.unwiredbb.com (64.192.2.193) 14 ms 15 ms 14 ms
  2 64-192-0-253.static.unwiredbb.com (64.192.0.253) 15 ms 14 ms 14 ms
  3 so-5-3.ipcolo1.SanFrancisco1.Level3.net (4.78.240.37) 19 ms 28 ms 25 ms
  4 so-6-1-0.mp2.SanFrancisco1.Level3.net (4.68.96.145) 25 ms 20 ms 21 ms
  5 as-0-0.bbr1.London1.Level3.net (4.68.128.109) 157 ms 157 ms ae-1-0.bbr2.London1.Level3.net (212.187.128.57) 158 ms
  6 * ae-14-55.car3.London1.Level3.net (4.68.116.145) 448 ms 289 ms
  7 195.50.113.18 (195.50.113.18) 231 ms 318 ms 231 ms
  8 217.194.157.226 (217.194.157.226) 754 ms 753 ms 746 ms
  9 217.194.157.225 (217.194.157.225) 755 ms 735 ms 730 ms
10 fe2-0-br.nbo.infinet.co.ke (41.207.64.245) 758 ms 744 ms 792 ms
11 ge0-2-pdsn.nbo.infinet.co.ke (41.207.64.254) 772 ms 764 ms 753 ms
12 ge0-1-br.nbo.infinet.co.ke (41.207.64.253) 736 ms 742 ms 776 ms
13 ge0-2-pdsn.nbo.infinet.co.ke (41.207.64.254) 745 ms 752 ms 747 ms
14 ge0-1-br.nbo.infinet.co.ke (41.207.64.253) 746 ms 735 ms 758 ms
15 ge0-2-pdsn.nbo.infinet.co.ke (41.207.64.254) 749 ms 740 ms 753 ms
16 ge0-1-br.nbo.infinet.co.ke (41.207.64.253) 746 ms 737 ms 757 ms
17 ge0-2-pdsn.nbo.infinet.co.ke (41.207.64.254) 740 ms 741 ms 738 ms
18 ge0-1-br.nbo.infinet.co.ke (41.207.64.253) 746 ms 742 ms 737 ms
19 ge0-2-pdsn.nbo.infinet.co.ke (41.207.64.254) 753 ms 771 ms 744 ms
20 ge0-1-br.nbo.infinet.co.ke (41.207.64.253) 735 ms 763 ms 738 ms
21 ge0-2-pdsn.nbo.infinet.co.ke (41.207.64.254) 779 ms 742 ms 819 ms
22 ge0-1-br.nbo.infinet.co.ke (41.207.64.253) 753 ms 768 ms 738 ms
23 ge0-2-pdsn.nbo.infinet.co.ke (41.207.64.254) 764 ms 749 ms 745 ms
24 ge0-1-br.nbo.infinet.co.ke (41.207.64.253) 766 ms 760 ms 761 ms
25 ge0-2-pdsn.nbo.infinet.co.ke (41.207.64.254) 746 ms 743 ms 808 ms
26

Bill Stewart wroteth on 3/15/2008 9:01 PM:

Wow! Its close to 20 hours now, and folks have still not fixed the problem!

I hope they're not many IPs NATed behind that unfortunate /24 that has
been cherry picked by AboveNet.