requesting hard data sources on ramifications of verisign wildcard

as already mentioned, fascinating public policy theatre
is going on in DC on the verisign wildcard issue, see
  http://secsac.icann.org/
[all video and even transcripts of both meetings online. go icann.]
you are encouraged to read through all of it before making public comments
on this issue at nanog. (or, hope springs eternal, to this list.)

caida has the following request on behalf of icann's secsac committee
[an exceptionally competent group who is approaching this issue
with impressive speed, equaniminity, and integrity.
  http://www.icann.org/committees/security/
i believe we're in good hands here. let's give the process a chance
and constructively contribute where we can.]

a common theme over the last week is an admitted lack of hard data
[rather than lists of theoretical breakages, and anecdotal evidence,
and predictions] from the operational community on actual loss of
stability in Internet performance or functionality.
david from XO gave an outstanding talk on 7 oct,
  http://www.icann.org/presentations/shairer-secsac-dc-07oct03.ppt
but, as with many other providers, he deployed the bind patch
within 24 hours so he didn't really have useful hard data
to put on the table. i get similar comments from others.
ben from harvard also gave some hard alexa data, fwiw
  http://www.icann.org/presentations/edelman-secsac-dc-15oct03.ppt
but from a specific vantage point. we need more of these.

icann's secsac committee is in a much stronger position to
provide technically sound and equitable guidance if we can
provide them with as specific, concrete examples (*hard data*)
that indicate extent of various types of breakage.

please save the arguments regarding the legitimacy and short notice
of this request until you've read the hours of discussion about
it that has already occurred among many qualified folks in DC
(and in any case the meta-issue still stretches AUP of nanog list).

the inconvenient reality is that the secsac committee needs concrete
data (imagine), in addition to bulleted lists of things that break,
for the policy process to work most effectively here. and nanog is
in a position to make a difference. you are hereby encouraged to do so.

please send any hard data reflecting observed ramifications on
security and stability of Internet infrastructure to

  secsac-comment@icann.org

no hard data will be refused service
k

VeriSign's response to the IAB confirms actual breakage, but dismisses
this acknowledgement by claiming that the breakages are insignificantly
minor to the Internet community as a whole.

This is fodder for the canonical political issue -- the original generic
TLDs are public resources, and have operational and management
requirements which are significantly more stringent than private TLDs.

k claffy wrote:

...
please send any hard data reflecting observed ramifications on
security and stability of Internet infrastructure to

        secsac-comment@icann.org

no hard data will be refused service

Here's a glimpse of some data for a small ISP (bcc'd to secsac).

This mail server was clogging with spam that couldn't be rejected with
bad .com and .net incoming addresses, and with bad .com and .net
outgoing undeliverable addresses. The server failed (stopped responding
to new SMTP requests, and/or crashed) again and again:

Sun, Sep 21, 2003 11:52 PM mail.WaterValley.Net 2 minutes, 35 seconds
Mon, Sep 22, 2003 00:01 AM mail.WaterValley.Net 4 minutes, 7 seconds
Mon, Sep 22, 2003 00:12 AM mail.WaterValley.Net 5 minutes, 48 seconds
Mon, Sep 22, 2003 01:18 AM mail.WaterValley.Net 1 minute, 1 second
Mon, Sep 22, 2003 04:07 AM mail.WaterValley.Net 5 minutes, 16 seconds
Mon, Sep 22, 2003 04:23 AM mail.WaterValley.Net 3 minutes, 3 seconds
Mon, Sep 22, 2003 04:33 AM mail.WaterValley.Net 1 minute, 19 seconds
Mon, Sep 22, 2003 04:37 AM mail.WaterValley.Net 9 minutes, 4 seconds
Mon, Sep 22, 2003 06:47 AM mail.WaterValley.Net 22 minutes, 58 seconds
Mon, Sep 22, 2003 07:15 AM mail.WaterValley.Net 6 minutes, 59 seconds
...
Mon, Sep 22, 2003 09:53 PM mail.WaterValley.Net 3 minutes, 0 seconds
Mon, Sep 22, 2003 10:01 PM mail.WaterValley.Net 5 minutes, 0 seconds
Mon, Sep 22, 2003 10:13 PM mail.WaterValley.Net 3 minutes, 1 second
Mon, Sep 22, 2003 10:21 PM mail.WaterValley.Net 3 minutes, 1 second
Mon, Sep 22, 2003 10:31 PM mail.WaterValley.Net 3 minutes, 1 second
Mon, Sep 22, 2003 10:39 PM mail.WaterValley.Net 3 minutes, 1 second
Mon, Sep 22, 2003 10:49 PM mail.WaterValley.Net 3 minutes, 1 second
Mon, Sep 22, 2003 10:59 PM mail.WaterValley.Net 3 minutes, 1 second
Mon, Sep 22, 2003 11:07 PM mail.WaterValley.Net 3 minutes, 2 seconds
Mon, Sep 22, 2003 11:17 PM mail.WaterValley.Net 1 minute, 3 seconds

Then, A MIRACLE OCCURRED! The problems STOPPED!

That miracle was BIND 9.2.3rc3, for which we are eternally grateful.
As I posted to NANOG on Tue, 23 Sep 2003 02:35:48 -0400:

William Allen Simpson wrote:
# Thought I'd mention that I helped setup BIND 9.2.3rc3 on a yellowdog
# linux powercomputing machine tonight. It worked. And the mail queues
# began clearing out. ...

The next downtime (for restoring saved mail queues) was:
Wed, Sep 24, 2003 06:39 PM mail.WaterValley.Net 21 minutes, 0 seconds

Note the dramatic difference -- from failures several times per hour,
to stability for days!

I don't know how many others were devastated by the VeriSign wildcards,
or whether the differences were as dramatic elsewhere. Hopefully,
other ISPs worldwide will step forward.

I expect we can come up with more data, but I'll save most of it for
the expected future affidavits....