i reported a problem with packet duplication to the gridnet
(worldcom) noc, and they checked it and they said everything
is fine, some hosts give duplicate echo replies to pings. i
get either one or three duplicates. are there any hosts,
routers, firewalls, load balancers, etc. that do this? i
couldn't find anything in the archives on this subject.
when i tried pinging from locations on other backbones i
didn't receive dups so i feel the problem is within
gridnet. i am reporting here as i would like to know if any
others are suffering this problem. also, many hosts and
networks may be receiving unnecessary traffic from duplicate
acks as there is duplication of tcp packets also.
most of the problems are through connections from east to
west coast on atm circuits--see traces below.
other unfriendly behavior--their default route appears to be
pb-nap.eni.net. most unroutable packets, including reserved
and rfc1918, are routed there--traces below. this morning,
when g.root-servers.net was unreachable for a few minutes
(via gridnet, did anyone else see it disappear?) i watched
those packets go to eni. (sorry, didn't capture the
trace.) are they selling /dev/rj0 service?
this (everything above) has been happening for at least
several months.
matt sommer
sysadmin
webkorner.com
PING route-server.cerf.net (192.215.254.5): 56 data bytes
64 bytes from 192.215.254.5: icmp_seq=0 ttl=248 time=150.6
ms
64 bytes from 192.215.254.5: icmp_seq=0 ttl=248 time=151.1
ms (DUP!)
64 bytes from 192.215.254.5: icmp_seq=0 ttl=248 time=151.9
ms (DUP!)
64 bytes from 192.215.254.5: icmp_seq=0 ttl=248 time=152.4
ms (DUP!)
64 bytes from 192.215.254.5: icmp_seq=1 ttl=248 time=129.1
ms
--- route-server.cerf.net ping statistics ---
2 packets transmitted, 2 packets received, +3 duplicates, 0%
packet loss
round-trip min/avg/max = 129.1/147.0/152.4 ms
traceroute to route-server.cerf.net (192.215.254.5), 30 hops
max, 40 byte packets
1 nemo.webkorner.com (207.53.77.1) 3.039 ms 2.013 ms
1.728 ms
2 core-atln1-hssi4/0.9.grid.net (207.205.2.1) 16.749 ms
19.756 ms 17.061 ms
3 core-snfx1-atm9-0.2.grid.net (206.80.170.70) 79.949 ms
81.523 ms 81.564 ms
4 snfc1.grid.net (198.32.136.48) 79.668 ms 93.924 ms
79.515 ms
5 atm8-0-155M.sjc-bb3.cerf.net (134.24.29.38) 97.64 ms
90.751 ms
f1-0-0.sjc-bb1.cerf.net (134.24.88.55) 106.49 ms
6 atm0-0-155M.sfo-bb2.cerf.net (134.24.29.21) 114.862 ms
atm8-0-155M.sjc-bb3.cerf.net (134.24.29.38) 101.439 ms
atm0-0-155M.sfo-bb2.cerf.net (134.24.29.21) 105.387 ms
7 pos0-2-155M.sfo-bb3.cerf.net (134.24.29.197) 93.883 ms
97.39 ms 97.007 ms
8 atm10-0-155M.lax-bb1.cerf.net (134.24.29.41) 110.238
ms 113.954 ms 107.834
ms
9 atm1-0-1-622M.san-bb6.cerf.net (134.24.29.141) 106.82
ms
atm10-0-155M.lax-bb1.cerf.net (134.24.29.41) 190.6 ms
atm1-0-1-622M.san-bb6.cerf.net (134.24.29.141) 135.938 ms
10 pos10-0-0-155M.san-bb2.cerf.net (134.24.29.105) 114.09
ms 100.856 ms
103.303 ms
11 pos1-0-155M.san-hq5.cerf.net (134.24.29.54) 112.1 ms
118.083 ms 104.277 ms
12 pos1-0-155M.san-hq5.cerf.net (134.24.29.54) 105.645 ms
106.927 ms 113.921
ms
13 * route-server.cerf.net (192.215.254.5) 122.29 ms *
PING cerf.net (192.102.249.3): 56 data bytes
64 bytes from 192.102.249.3: icmp_seq=0 ttl=247 time=178.2
ms
64 bytes from 192.102.249.3: icmp_seq=0 ttl=247 time=178.9
ms (DUP!)
64 bytes from 192.102.249.3: icmp_seq=0 ttl=247 time=179.2
ms (DUP!)
64 bytes from 192.102.249.3: icmp_seq=0 ttl=247 time=179.7
ms (DUP!)
64 bytes from 192.102.249.3: icmp_seq=1 ttl=247 time=126.5
ms
--- cerf.net ping statistics ---
2 packets transmitted, 2 packets received, +3 duplicates, 0%
packet loss
round-trip min/avg/max = 126.5/168.5/179.7 ms
traceroute to cerf.net (192.102.249.3), 30 hops max, 40 byte
packets
1 nemo.webkorner.com (207.53.77.1) 1.978 ms 1.959 ms
1.724 ms
2 core-atln1-hssi4/0.9.grid.net (207.205.2.1) 29.475 ms
17.064 ms 18.406 ms
3 core-snfx1-atm9-0.2.grid.net (206.80.170.70) 80.843 ms
89.331 ms 82.987 ms
4 snfc1.grid.net (198.32.136.48) 85.938 ms 78.285 ms
82.579 ms
5 f0-0-0.sjc-bb2.cerf.net (134.24.88.5) 85.739 ms 88.796
ms 90.437 ms
6 pos0-0-155M.san-bb6.cerf.net (134.24.29.130) 104.53 ms
102.469 ms 102.327
ms
7 pos0-0-155M.san-hq5.cerf.net (134.24.29.113) 102.044 ms
pos0-0-155M.san-bb6.cerf.net (134.24.29.130) 100.354 ms
pos0-0-155M.san-hq5.cerf.net (134.24.29.113) 101.231 ms
8 pos0-0-155M.san-hq5.cerf.net (134.24.29.113) 103.14 ms
98.93 ms 100.854 ms
9 pos9-0-0-155M.san-hq1.cerf.net (134.24.29.126) 108.44
ms 104.816 ms 104.518
ms
10 nic.cerf.net (192.102.249.3) 102.09 ms 110.655 ms
103.239 ms
PING cisco.com (192.31.7.130): 56 data bytes
64 bytes from 192.31.7.130: icmp_seq=0 ttl=246 time=89.0 ms
64 bytes from 192.31.7.130: icmp_seq=0 ttl=246 time=89.5 ms
(DUP!)
64 bytes from 192.31.7.130: icmp_seq=0 ttl=246 time=90.0 ms
(DUP!)
64 bytes from 192.31.7.130: icmp_seq=0 ttl=246 time=90.5 ms
(DUP!)
64 bytes from 192.31.7.130: icmp_seq=1 ttl=246 time=104.7 ms
--- cisco.com ping statistics ---
2 packets transmitted, 2 packets received, +3 duplicates, 0%
packet loss
round-trip min/avg/max = 89.0/92.7/104.7 ms
traceroute to cisco.com (192.31.7.130), 30 hops max, 40 byte
packets
1 nemo.webkorner.com (207.53.77.1) 1.946 ms 5.717 ms
2.183 ms
2 core-atln1-hssi4/0.9.grid.net (207.205.2.1) 16.692 ms
25.742 ms 40.845 ms
3 core-snfx1-atm5-0.4.grid.net (206.80.187.29) 98.354 ms
85.901 ms 82.039 ms
4 snfc1.grid.net (198.32.136.48) 79.189 ms 107.082 ms
79.548 ms
5 p2-0.sanjose1-nbr1.bbnplanet.net (4.0.3.193) 85.506 ms
100.916 ms 105.923
ms
6 p2-0.sanjose1-nbr1.bbnplanet.net (4.0.3.193) 92.522 ms
96.826 ms 100.667 ms
7 p4-0.paloalto-nbr2.bbnplanet.net (4.0.1.1) 91.881 ms
110.78 ms 106.679 ms
8 h1-0.cisco.bbnplanet.net (131.119.26.10) 95.844 ms
89.91 ms 89.474 ms
9 h1-0.cisco.bbnplanet.net (131.119.26.10) 92.13 ms
pigpen.cisco.com
(192.31.7.40) 98.463 ms h1-0.cisco.bbnplanet.net
(131.119.26.10) 114.123 ms
10 cio-sys.cisco.com (192.31.7.130) 97.249 ms
pigpen.cisco.com (192.31.7.40)
99.075 ms 95.722 ms
PING enmu.edu (192.94.216.250): 56 data bytes
64 bytes from 192.94.216.250: icmp_seq=0 ttl=51 time=322.1
ms
64 bytes from 192.94.216.250: icmp_seq=0 ttl=51 time=322.7
ms (DUP!)
64 bytes from 192.94.216.250: icmp_seq=1 ttl=51 time=239.1
ms
--- enmu.edu ping statistics ---
2 packets transmitted, 2 packets received, +1 duplicates, 0%
packet loss
round-trip min/avg/max = 239.1/294.6/322.7 ms
traceroute to enmu.edu (192.94.216.250), 30 hops max, 40
byte packets
1 nemo.webkorner.com (207.53.77.1) 2.014 ms 1.927 ms
2.617 ms
2 core-atln1-hssi4/0.9.grid.net (207.205.2.1) 19.161 ms
27.238 ms 16.684 ms
3 core-snfx1-atm5-0.4.grid.net (206.80.187.29) 81.792 ms
82.091 ms 80.942 ms
4 t8-0.San-Francisco.t3.ans.net (198.32.128.10) 95.598
ms 83.259 ms 83.83 ms
5 pacbell-nap.ans.net (198.32.128.67) 110.493 ms 97.621
ms 85.529 ms
6 h13-1.t16-0.Los-Angeles.t3.ans.net (140.223.9.14)
106.263 ms 119.991 ms
99.344 ms
7 h14-1.t112-0.Albuquerque.t3.ans.net (140.223.17.10)
136.081 ms 120.616 ms
119.887 ms
8 f0-0.cnss116.Albuquerque.t3.ans.net (140.222.112.196)
138.032 ms 122.725 ms
125.71 ms
9 h1-0.enss191.t3.ans.net (192.103.74.42) 126.823 ms
131.757 ms 130.943 ms
10 198.83.5.4 (198.83.5.4) 158.435 ms 144.432 ms 154.82
ms
11 192.65.78.82 (192.65.78.82) 175.607 ms 160.103 ms
168.386 ms
12 EM0X.ENMU.EDU (192.94.216.250) 244.882 ms 172.999 ms
183.95 ms
PING e.root-servers.net (192.203.230.10): 56 data bytes
64 bytes from 192.203.230.10: icmp_seq=0 ttl=251 time=112.5
ms
64 bytes from 192.203.230.10: icmp_seq=0 ttl=251 time=114.3
ms (DUP!)
64 bytes from 192.203.230.10: icmp_seq=1 ttl=251 time=117.3
ms
--- e.root-servers.net ping statistics ---
2 packets transmitted, 2 packets received, +1 duplicates, 0%
packet loss
round-trip min/avg/max = 112.5/114.7/117.3 ms
traceroute to e.root-servers.net (192.203.230.10), 30 hops
max, 40 byte packets
1 nemo.webkorner.com (207.53.77.1) 2.072 ms 2.964 ms
1.746 ms
2 core-atln1-hssi4/0.9.grid.net (207.205.2.1) 17.125 ms
17.129 ms 16.715 ms
3 core-wash1-atm4/0.4.grid.net (206.80.188.33) 81.316 ms
69.578 ms 81.366 ms
4 mae-east.nsn.nasa.gov (192.41.177.125) 98.22 ms 87.618
ms 91.117 ms
5 s-ARC1-ATM.NSN.NASA.GOV (128.161.10.1) 116.303 ms
112.699 ms 113.355 ms
6 E.ROOT-SERVERS.NET (192.203.230.10) 123.275 ms *
111.486 ms
PING l.root-servers.net (198.32.64.12): 56 data bytes
64 bytes from 198.32.64.12: icmp_seq=0 ttl=244 time=174.9 ms
64 bytes from 198.32.64.12: icmp_seq=0 ttl=244 time=175.1 ms
(DUP!)
64 bytes from 198.32.64.12: icmp_seq=1 ttl=244 time=157.6 ms
--- l.root-servers.net ping statistics ---
2 packets transmitted, 2 packets received, +1 duplicates, 0%
packet loss
round-trip min/avg/max = 157.6/169.2/175.1 ms
traceroute to l.root-servers.net (198.32.64.12), 30 hops
max, 40 byte packets
1 nemo.webkorner.com (207.53.77.1) 3.558 ms 3.419 ms
1.765 ms
2 core-atln1-hssi4/0.9.grid.net (207.205.2.1) 18.912 ms
25.015 ms 17.186 ms
3 core-wash1-atm8-0.3.grid.net (206.80.180.34) 94.471 ms
87.77 ms 95.625 ms
4 * f6.peer1.wdc1.genuity.net (192.41.177.147) 109.531
ms 140.935 ms
5 h5-0-1.core1.wdc1.genuity.net (207.240.1.233) 136.672
ms * 154.67 ms
6 * core1.lax1.genuity.net (207.240.0.5) 169.329 ms
181.559 ms
7 fe-5-0.peer1.lax1.genuity.net (207.240.1.142) 244.746
ms 164.322 ms 184.965
ms
8 sandbox.ep.net (198.32.146.11) 168.058 ms 159.353 ms *
9 * l.root-servers.net (198.32.64.12) 159.825 ms 170.084
ms
PING altavista.com (204.123.2.66): 56 data bytes
64 bytes from 204.123.2.66: icmp_seq=0 ttl=54 time=96.8 ms
64 bytes from 204.123.2.66: icmp_seq=0 ttl=54 time=97.3 ms
(DUP!)
64 bytes from 204.123.2.66: icmp_seq=1 ttl=54 time=99.8 ms
--- altavista.com ping statistics ---
2 packets transmitted, 2 packets received, +1 duplicates, 0%
packet loss
round-trip min/avg/max = 96.8/97.9/99.8 ms
traceroute to altavista.com (204.123.2.69), 30 hops max, 40
byte packets
1 nemo.webkorner.com (207.53.77.1) 1.96 ms 3.553 ms
1.782 ms
2 core-atln1-hssi4/0.9.grid.net (207.205.2.1) 16.567 ms
19.488 ms 21.282 ms
3 peer-atl-uunet-fddi.grid.net (206.80.190.201) 25.164
ms 17.09 ms 61.332 ms
4 104.ATM2-0.XR1.ATL1.ALTER.NET (146.188.232.50) 18.564
ms 17.982 ms 17.676
ms
5 295.ATM2-0.TR1.ATL1.ALTER.NET (146.188.232.90) 17.736
ms 86.312 ms 36.625
ms
6 109.ATM6-0.TR1.SCL1.ALTER.NET (146.188.136.66) 74.736
ms 80.162 ms 83.007
ms
7 199.ATM7-0.XR1.PAO1.ALTER.NET (146.188.147.117) 75.24
ms 107.525 ms 86.392
ms
8 189.ATM10-0-0.GW3.PAO1.ALTER.NET (146.188.147.249)
78.844 ms 81.42 ms 85.98
ms
9 208.195.253.18 (208.195.253.18) 94.185 ms 81.601 ms
84.882 ms
10 core-gw1.pa-x.dec.com (204.123.1.1) 81.878 ms 81.915
ms 82.955 ms
11 altavista.com (204.123.2.69) 90.244 ms 86.355 ms
84.804 ms
*** default route stuff ***
traceroute to 192.168.0.1 (192.168.0.1), 7 hops max, 40 byte
packets
1 nemo.webkorner.com (207.53.77.1) 2.109 ms 1.987 ms
1.725 ms
2 core-atln1-hssi4/0.9.grid.net (207.205.2.1) 16.876 ms
17.136 ms 16.879 ms
3 core-wash1-atm4/0.4.grid.net (206.80.188.33) 80.955 ms
69.07 ms 84.384 ms
4 206.80.168.178 (206.80.168.178) 108.509 ms 108.005 ms
108.684 ms
5 pb-nap.eni.net (198.32.128.39) 116.592 ms 108.99 ms
108.275 ms
6 pb-nap.eni.net (198.32.128.39) 108.28 ms !H * *
7 * * *
[they blackholed this one, oddly enough]
traceroute to 172.16.0.1 (172.16.0.1), 7 hops max, 40 byte
packets
1 nemo.webkorner.com (207.53.77.1) 1.956 ms 1.996 ms
1.695 ms
2 core-atln1-hssi4/0.9.grid.net (207.205.2.1) 16.623 ms
17.005 ms 20.315 ms
3 core-snfc1-hssi11-0.1.grid.net (206.80.180.214) 78.943
ms 76.954 ms 78.453
ms
4 * * *
5 * * *
6 * * *
7 * * *
traceroute to 172.17.0.1 (172.17.0.1), 7 hops max, 40 byte
packets
1 nemo.webkorner.com (207.53.77.1) 2.184 ms 2.169 ms
1.69 ms
2 core-atln1-hssi4/0.9.grid.net (207.205.2.1) 17.204 ms
17.49 ms 96.326 ms
3 core-wash1-atm4/0.4.grid.net (206.80.188.33) 73.894 ms
80.646 ms 72.661 ms
4 206.80.168.178 (206.80.168.178) 114.916 ms 111.088 ms
107.569 ms
5 pb-nap.eni.net (198.32.128.39) 110.934 ms 109.213 ms
109.857 ms
6 * * *
7 * * *
traceroute to 172.32.0.1 (172.32.0.1), 7 hops max, 40 byte
packets
1 nemo.webkorner.com (207.53.77.1) 2.002 ms 3.351 ms
1.696 ms
2 core-atln1-hssi4/0.9.grid.net (207.205.2.1) 17.148 ms
16.834 ms 18.726 ms
3 core-wash1-atm8-0.3.grid.net (206.80.180.34) 94.444 ms
94.4 ms 99.834 ms
4 206.80.168.178 (206.80.168.178) 140.235 ms 140.247 ms
137.033 ms
5 pb-nap.eni.net (198.32.128.39) 127.318 ms 133.545 ms
124.439 ms
6 * * *
7 * * *
traceroute to 10.0.0.1 (10.0.0.1), 7 hops max, 40 byte
packets
1 nemo.webkorner.com (207.53.77.1) 2.998 ms 1.933 ms
2.636 ms
2 core-atln1-hssi4/0.9.grid.net (207.205.2.1) 16.79 ms
17.42 ms 17.519 ms
3 core-wash1-atm4/0.4.grid.net (206.80.188.33) 83.64 ms
80.487 ms 98.335 ms
4 206.80.168.178 (206.80.168.178) 126.873 ms 111.404 ms
110.231 ms
5 pb-nap.eni.net (198.32.128.39) 127.533 ms 109.746 ms
111.201 ms
6 pb-nap.eni.net (198.32.128.39) 114.215 ms !H * *
7 * * *
traceroute to 1.0.0.1 (1.0.0.1), 7 hops max, 40 byte packets
1 nemo.webkorner.com (207.53.77.1) 2.053 ms 2.003 ms
1.685 ms
2 core-atln1-hssi4/0.9.grid.net (207.205.2.1) 30.512 ms
17.147 ms 18.741 ms
3 core-wash1-atm4/0.4.grid.net (206.80.188.33) 76.457 ms
89.586 ms 69.123 ms
4 206.80.168.178 (206.80.168.178) 110.383 ms 117.309 ms
112.722 ms
5 pb-nap.eni.net (198.32.128.39) 112.76 ms 112.733 ms
111.573 ms
6 * * *
7 * * *
traceroute to 0.0.0.1 (0.0.0.1), 7 hops max, 40 byte packets
1 nemo.webkorner.com (207.53.77.1) 2.025 ms 2.825 ms
1.753 ms
2 core-atln1-hssi4/0.9.grid.net (207.205.2.1) 20.68 ms
17.356 ms 20.616 ms
3 core-wash1-atm4/0.4.grid.net (206.80.188.33) 73.731 ms
core-wash1-atm8-0.3.grid.net (206.80.180.34) 94.608 ms
core-wash1-atm4/0.4.grid.net (206.80.188.33) 68.392 ms
4 206.80.168.178 (206.80.168.178) 121.929 ms 107.539 ms
120.781 ms
5 pb-nap.eni.net (198.32.128.39) 112.065 ms 122.983 ms
109.885 ms
6 * * *
7 pb-nap.eni.net (198.32.128.39) 111.42 ms !H * *