Here's some background:
AGIS's router is not colocated at the MAE parking garage, but is in fact
colocated at WorldCom in downtown Washington DC. Our bits get from there to
the MAE via a DS3, and that DS3 is terminated at each end with a device
called a NetEdge, which does the FDDI to DS3 ATM conversion.
These NetEdges seem to have three different possible operating states:
completely working (which doesn't happen often enough); broken (often, right
out of the box); and kind of working (which happens all too often). This
third operating state results in some very interesting, possibly misleading,
and sometimes damaging behavior. It looks quite similar to the kind of
behavior you get when you change the MAC layer device but keep the same ip
address at either of the MAE's: ARP caches get inconsistent, and BGP
sessions with other routers flop around, leading to routes getting flap
dampened by those running the appropriate code.
Here's what happened:
AGIS's connection to MAE-East experienced one of these kind-of-working
problems which resulted in the erratic behavior above. Digex customers
wishing to reach AGIS customers called the Digex NOC, and the posting which
started this all was made to the Digex internal news group. Similarly, AGIS
customers had problems, and we worked with MFS to get the problem resolved
(they must have a warehouse full of swapped-out NetEdges at this point).
In the interval, a short-on-facts bozo spit into the wind and got us and
Digex wet. I'm in private correspondence with Ed Kern to postmortem the
situation.
Peter