Since ANS seems to be passing our interface address as the
next-hop directly to some nets (e.g., Digex and Advantis), the failure
as I described above did lead to a loss of connectivity between AGIS
and at least Digex and Advantis. Pending the solution of the MFS
problem, it would have been possible to work around the issue if the
affected nets had routed _through_ their transit provider.
Let me reiterate the point that propagating third-party
next-hops in the absence of guaranteed fate-sharing is EVIL,
or at least very very risky.
My opinion is probably at one pole of the spectrum of ideas
about NAPs and MAEs, however it's essentially this: do not
propagate other people's next-hops at all to your NAP/MAE peers,
either using next-hop-self (or the equivalent) or announcing only
those prefixes for which you have yourself as a next-hop. Moreover,
one should be very cagey about accepting third-party next-hops
from one's peers, and either refuse routes with such next-hops,
or (with permission only), rewrite the next-hops in question,
unless there is a very good reason to do otherwise.
On that point, and with respect to "neighbor foo next-hop-self"
at an exchange-point, one has to remember that one will be doing
multiple hops across the same medium, which is an inefficient
way of making use of both one's (or maybe everyone's, depending
on the particular fabric) exchange-point bandwidth, and also
one's router capacity. One may, given careful thinking, decide
to accept and/or propagate third-party next-hops, if this is
an issue. However, this should not be the default behaviour
at any exchange-point, because the AGIS/DIGEX disconnectivity
is a well-known and formerly oft-seen problem.
Finally, there is wording in some bilaterals floating out
there with respect to third-party transit at exchange points,
which may push towards one pole or the other of opinions
on third-party next-hops on non-fate-shared (and often *weird*)
On another front, *weird* MAE and NAP setups have caused
so much trouble that I sometimes wonder when the next time
I get to say "I told you so" about multi-fabric-bridging-from-hell
will be, and how bad it will hurt.