IP provider performance measurement BOF

In message <199503082144.QAA01126@curtis.ansremote.com>, Curtis Villamizar writ

IMO the IETF Operations Area has not been unfair or hostile toward
providers. I'm not sure what the objections are.

I didn't mean to suggest that they are.

My objections are more philosophical and, perhaps, pragmatic.

I would like to see Internet standards-describing bodies
stay well away from trying to standardize operational
metrics and operational procedures.

This is to avoid three things:

  -- operators who say, "our NIC is RFC 9000 compliant"

  -- attempting to define what an operational
     organization should do as an Internet standard:
     "An ISP that is compliant with this standard
     will..." which is particularly amusing if it
     doesn't make business sense in a particular case,
     or when it is drafted by people who mispredict
     real operational issues (due to lack of foresight
     or lack of experience) that obviate the standard.

     This is essentially to avoid, "our competitor's
     organization is not RFC 9000 compliant"

  -- Operators and standards-folks are aliens from
     another planet. Operators typically will bend
     standards as they see fit for their business,
     while standards-folks attempt to make it easy for
     anyone -- say, someone new to the field -- to
     interoperate with the current operators.

     In extreme cases, operators completely ignore or
     break standards willy-nilly, while standards-
     folks get quite noisy when standards aren't
     followed to the letter. I have run into both,
     and, arguably, am an example of the former type.

     There is a tension between the two mindsets which
     is a good one for the Internet in general, but
     which essentialy necessitates, in my opinion, a
     division into two separate clubs when it comes to
     working out issues that are more or less fully
     in the domain of operators vs. standards folks.

     That's not to say that standards-folks should
     be discouraged from commenting on how we run our
     parts of the Internet, nor that operators should
     be told to shut up in working groups -- in fact,
     I like to see operators in working groups and
     standards-folks at NANOGs or IEPGs. However,
     any kind of consesus-building about operations
     issues or discussions about metrics showing good
     or bad things about operators' networks should
     be done in operators' forums, and not standards

     Conversely, protocol standards shouldn't be done
     at operators' forums.

An example of how things work differently among high
concentrations of operators vs high concentrations of
standards-folks is the CIDR issue. CIDRD at IETF and
the CIDR talks at NANOG are quite different, involve
different issues, and come up with very different forms
of consensus.

I believe this is a good thing, and should be encouraged.

So, wrt the provider measurement BOF, I am quite happy to
see the standards-folks describe standard tools which
operators can use to work out operational issues in
operators' forums.

I would not like to see the Operations Area turn into an
olympics for providers any more than for vendors in other