Level 3 was down in KC, Chi, and San Jose (at least) for us between
about 8:10 and 8:40, plus or minus. Brought down SureWest in KC too.
-Steve
From: nanog-request@nanog.org [mailto:nanog-request@nanog.org]
Sent: Monday, November 07, 2011 10:05 AM
To: nanog@nanog.org
Subject: NANOG Digest, Vol 46, Issue 15Send NANOG mailing list submissions to
nanog@nanog.orgTo subscribe or unsubscribe via the World Wide Web, visit
https://mailman.nanog.org/mailman/listinfo/nanog
or, via email, send a message with subject or body 'help' to
nanog-request@nanog.orgYou can reach the person managing the list at
nanog-owner@nanog.orgWhen replying, please edit your Subject line so it is more specific
than "Re: Contents of NANOG digest..."Today's Topics:
1. Re: Time Warner Telecom problems (Jon Lewis)
2. Re: Performance Issues - PTR Records (Leigh Porter)
3. Re: Time Warner Telecom problems (Ray Van Dolson)
4. General Internet Instability (Jared Mauch)
5. Re: TATA problems? (Pierre-Yves Maunier)
6. Re: TATA problems? (Leigh Porter)
7. Re: Time Warner Telecom problems (Joe Greco)
8. Re: TATA problems? (Kelly Kane)
9. Re: Time Warner Telecom problems (Blake Hudson)
10. RE: Time Warner Telecom problems (Thomas York)----------------------------------------------------------------------
Message: 1
Date: Mon, 7 Nov 2011 10:12:30 -0500 (EST)
From: Jon Lewis <jlewis@lewis.org>
To: Peter Pauly <ppauly@gmail.com>
Cc: nanog@nanog.org
Subject: Re: Time Warner Telecom problems
Message-ID: <Pine.LNX.4.61.1111071005540.24418@soloth.lewis.org>
Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed> Gizmodo is reporting problems at Time Warner Telecom .... we're
suffering
> from it too and calls to the NOC have not been answered so far...
does
> anyone have any further information?
>
> Massive Time Warner Outage Hits the USI noticed just a little while ago that we're having a lot of DNS fail.
Initial findings were that several of the root-servers we were trying
to
reach via our TWTelecom link were unreachable after 2 hops into TWT.
4 64-128-130-233.static.twtelecom.NET (64.128.130.233) 2.399 ms
2.298
ms 2.338 ms
5 mia2-pr1-xe-1-3-0-0.us.twtelecom.net (66.192.253.18) 11.571 ms
11.552 ms 9.467 ms
6 * * *
7 * * *
8 * * *For instance, a.root-servers.net is pingable from a rackspace server,
but
not from our network (unless I shut off TWT, at which point it is, but
it's apparently not the same a.root-servers.net instance rackspace
sees).
I assume this is one of the root-servers being anycast.
Shutting off our BGP with TWT didn't appear to help (though the
root-servers became reachable)...so I assume there's more going on
than
just TWT routing fail.
----------------------------------------------------------------------
Jon Lewis, MCP| I route
Senior Network Engineer | therefore you are
Atlantic Net |
_________ http://www.lewis.org/~jlewis/pgp for PGP public key_________------------------------------
Message: 2
Date: Mon, 7 Nov 2011 15:29:30 +0000
From: Leigh Porter <leigh.porter@ukbroadband.com>
To: Bj?rn Mork <bjorn@mork.no>
Cc: "nanog@nanog.org" <nanog@nanog.org>
Subject: Re: Performance Issues - PTR Records
Message-ID: <53A4963F-4969-4A60-BF06-E690C7324863@ukbroadband.com>
Content-Type: text/plain; charset="iso-8859-1"> Leigh Porter <leigh.porter@ukbroadband.com> writes:
>
>> Indeed, there is no way I would allow that either. But really,
>> providing a reverse zone and forward zone to match is a case of
five
>> minutes and a shell script or a DNS that as Steinar said, will
>> synthesise results.
>>
>> It's really not all that difficult..
>
> No, not at all. It's just totally pointless. Any IPv6 address is
just
> as pretty as a synthesized name. Maybe even prettier. Do you prefer
> "2001:db8:1::2" or
"20010db8000100000000000000000002.rev.example.com"?
>
> If we're going to provide any reverse DNS for end users, then it is
> because we can create names which actually improves something.
>
>
> Bj?rn
>
>Yup it is pointless.. Mine are all ipadrress.domain which is of
course,
pointless.. I suppose at least somebody would glean that perhaps its a
home user rather than a business or server on that address but that's
all.
With IPv6 arguably even more pointless as you say.
______________________________________________________________________
This email has been scanned by the MessageLabs Email Security System.
For more information please visit Email Security
______________________________________________________________________------------------------------
Message: 3
Date: Mon, 7 Nov 2011 07:28:18 -0800
From: Ray Van Dolson <rvandolson@esri.com>
To: nanog@nanog.org
Subject: Re: Time Warner Telecom problems
Message-ID: <20111107152817.GA29715@esri.com>
Content-Type: text/plain; charset=us-ascii> Gizmodo is reporting problems at Time Warner Telecom .... we're
suffering
> from it too and calls to the NOC have not been answered so far...
does
> anyone have any further information?
>
> Massive Time Warner Outage Hits the USFWIW, my home TWC connection dropped this morning for about 15 minutes
(Southern California around 6:30AM'ish). Still could ping the default
gateway, but packets weren't traversing much beyond that.Didn't investigate further, just headed into work.
Ray
------------------------------
Message: 4
Date: Mon, 7 Nov 2011 10:31:31 -0500
From: Jared Mauch <jared@puck.nether.net>
To: Tom Hill <tom@ninjabadger.net>
Cc: nanog@nanog.org
Subject: General Internet Instability
Message-ID: <B6567BC5-75E9-4E58-AFA9-5ADC0C2A7BDD@puck.nether.net>
Content-Type: text/plain; charset=us-ascii>> We seem to be having some problems with our tata links - first seen
in
EU
>> about 45 minutes ago, now we're seeing problems in NA. I'm focused
on
DNS,
>> so I'm seeing a lot of timeouts/servfails, but our networking folks
are
>> talking about links dropping.
>>
>> Anyone else seeing oddness on the NA Internet right now?
>>
>> http://downrightnow.com/ confirms - something is up.
>
> There are widespread issues across the Internet; certain versions of
> Juniper firmware have core dumped after seeing a particular BGP
'UPDATE'
> message.
>
> (That's the running theory at least).
>
> It's affected multiple service providers, globally, not just those
> connected to TATA.Pretty much any major BGP event will impact multiple providers.
A threshold you should use to view the general instability (which I
find
valuable, you may as well) is route views data.
If you look at the BGP UPDATES archive sizes, you can see when
something
happens, e.g.:
Index of /bgpdata/2011.11/UPDATES
Take a look at the size of the updates.20111107.1400.bz2 file and the
1415
file. They are abnormally large compared to a normal period of time.
This shows there were a lot of updates out there being processed and a
reference to levels of instability.If you are not feeding route views or similar community projects,
please
consider doing so. It helps paint the view for those doing analysis.
- Jared
------------------------------
Message: 5
Date: Mon, 7 Nov 2011 16:33:15 +0100
From: Pierre-Yves Maunier <nanog@maunier.org>
To: Tom Hill <tom@ninjabadger.net>
Cc: nanog@nanog.org
Subject: Re: TATA problems?
Message-ID:
<CAEGdXrM+2rFXG2=d80kO_1ObtAv=24npNqZsLCTXUxeKS9q9yA@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1
> > We seem to be having some problems with our tata links - first
seen in
EU
> > about 45 minutes ago, now we're seeing problems in NA. I'm
focused on
> DNS,
> > so I'm seeing a lot of timeouts/servfails, but our networking
folks
are
> > talking about links dropping.
> >
> > Anyone else seeing oddness on the NA Internet right now?
> >
> > http://downrightnow.com/ confirms - something is up.
>
> There are widespread issues across the Internet; certain versions of
> Juniper firmware have core dumped after seeing a particular BGP
'UPDATE'
> message.
>
> (That's the running theory at least).
>
> It's affected multiple service providers, globally, not just those
> connected to TATA.
>
> Tom
>
>
>
On our side all our 10.3R2.11 core dumped which made all our
interfaces
flapped.
I've been told 10.4R1.9 is affected too.--
Pierre-Yves Maunier------------------------------
Message: 6
Date: Mon, 7 Nov 2011 15:45:18 +0000
From: Leigh Porter <leigh.porter@ukbroadband.com>
To: Pierre-Yves Maunier <nanog@maunier.org>
Cc: "nanog@nanog.org" <nanog@nanog.org>
Subject: Re: TATA problems?
Message-ID: <7994AF08-0622-434F-974F-FC9269469176@ukbroadband.com>
Content-Type: text/plain; charset="us-ascii"My 10.4r1.9 boxes died also but I saw interfaces go down whilst bgpd
seemed stable.--
Leigh>
>>> We seem to be having some problems with our tata links - first
seen in
EU
>>> about 45 minutes ago, now we're seeing problems in NA. I'm
focused on
>> DNS,
>>> so I'm seeing a lot of timeouts/servfails, but our networking
folks
are
>>> talking about links dropping.
>>>
>>> Anyone else seeing oddness on the NA Internet right now?
>>>
>>> http://downrightnow.com/ confirms - something is up.
>>
>> There are widespread issues across the Internet; certain versions
of
>> Juniper firmware have core dumped after seeing a particular BGP
'UPDATE'
>> message.
>>
>> (That's the running theory at least).
>>
>> It's affected multiple service providers, globally, not just those
>> connected to TATA.
>>
>> Tom
>>
>>
>>
> On our side all our 10.3R2.11 core dumped which made all our
interfaces