Well, AT&T couldn't fix the problem, but at least their customer
service people knew there was a problem and were willing to attempt
contact with someone at Microsoft.
Most Microsoft sites (Microsoft.com, MSN.com, Hotmail.com, etc.)
are affected. The Microsoft technical team is troubleshooting this
issue.
Including passport.com, so even though msn.co.uk appears to be unaffected
(Pipex nameservers, not msft), you can't do anything that requires authentication
So, does anyone know what really is going on (from a technical POV) ?
Info also available on CNN as of 2:19 pm EST.
http://www.cnn.com/2001/TECH/computing/01/24/microsoft.blackout.idg/index.html
Reading it now...
Chuck
> Most Microsoft sites (Microsoft.com, MSN.com, Hotmail.com, etc.)
> are affected. The Microsoft technical team is troubleshooting this
> issue.Hotmail has had no impact regarding this issue.
That's *not* what our clients have reported or what I observe.
Some clients have hotmail set as their start page and are calling to
report "The internet is DOWN!"
To make matters worse, IE's default 'auto.search.msn.com' which is used
when a site is unreachable is itself unreachable. So client's aren't
even getting the marginally useful error page normally displayed if a
site can't be contacted.
I observed the same behaviors from a dialup outside our network ~9am
MST, and no connections to www.hotmail.com were possible.
It doesn't appear to be a DNS resolution issue for 'hotmail.com', as I
can telnet www.hotmail.com 80 and GET / HTTP1.1 to retrieve the page.
But Internet Explorer does *not* load the page, appearing to hang while
attempting to access 'passport.com' which does not have resolution
currently.
Mike
Microsoft's ITG is investigating this issue. I haven't been clued in as
of yet as to what is the main issue. Hotmail's graphs and logins are
currently following the same trends as normal, they seem unaffected,
however this is not the case in all locations. DNS seems to be the
obvious choice for the blame. This is not the case in all areas, however.
At this point Microsoft is not willing to put the blame on anyone, or
any protocol for that matter. (Unless they already released a public
statement saying so, then who knows?) Anyway, the issues are being worked
on and service will be restored as soon as possible. I apolozise for not
being able to disclose more information.
Well, whether Microsoft gets their DNS records fixed or not in the root
zones (assuming that's even the problem), they STILL won't get much
traffic from some avenues. Packet loss rates to many destinations off
ATT's network range from 50% to 100%
Looking at traces to Microsoft's DNS servers (which are all hosted on
the same ISP backbone, bad idea) shows 50% packet loss right now from
ATT Broadband. It's possible the servers will actually answer DNS
queries, but the protocols don't survive all that well in the face of
50% or greater packet loss.
Looks like several problems in the wild today...
Hotmail NS:
ns3.hotmail.com internet address = 209.185.130.68
ns1.hotmail.com internet address = 216.200.206.140
Whereas Microsoft NS:
DNS6.CP.MSFT.NET internet address = 207.46.138.20
DNS4.CP.MSFT.NET internet address = 207.46.138.11
DNS5.CP.MSFT.NET internet address = 207.46.138.12
DNS7.CP.MSFT.NET internet address = 207.46.138.21
A simple nslookup to the above servers says:
nslookup -q=ns microsoft.com dns6.cp.msft.net
*** Can't find server name for address 207.46.138.20: No response from server
*** Default servers are not available
Just to check, the 207.46.138 dns servers belong to:
route: 207.46.128.0/18
descr: MS-CP
origin: AS8070
mnt-by: MICROSOFT-MAINT-CW
changed: judithsh@microsoft.com 20001024
source: CW
And so without doubt, it is MS themselves who are down not icann, root dns
or anything else...
Steve
http://www.wirednews.com/news/business/0,1367,41387,00.html
Looks like they're still denying the DDOS possibility but are doing so
with less enthusiasm. Packet loss appears to be constrained to the last
hop, so it's feasible that this is the result of a targetted attack.
"we are horribly sorry, please put yourself in our shoes and have a little
patience."
which is why i ensure my DNS is RFC2182 compliant.
ns.reptiles.org Toronto, Canada
ns2.reptiles.org Karachi, Pakistan
Oh, come on....
If you query direct to Microsoft's DNS servers, they return MX records
happily, but they time out rather then return an A record.
This seems more obviously like a software problem, rather then DDoS, or
ICANN, who apparently controls Microsoft's DNS servers (laugh out loud)
"We don't manage the DNS ourselves, it is a system controlled by the Internet
Corporation for Assigned Names and Numbers (ICANN) with worldwide replicas."
I'll just point out a few lines I wrote. I have tried to be polite about
all of this.
Imagine what happens when all cached RRs expire worldwide. With
people hammering for microsoft.com DNS resolution. This can easily
translate to a "DDoS" to the four listed nameservers :->
A kinda self-brewn DDoS, because they don't manage to get backup DNS
servers up before all the cached RRs expire. :-]
Someone please send them a copy of DNS&Bind?
Ok, I wasn't going to say this, but Microsoft has been absolutely
ridiculous about blaming other people, so I'm going to say it; it's not
directed personally at you, it's directed at the morons who wrote your DNS
software, and the even bigger morons who insist that
It's-Not-A-Microsoft-Problem-Because-Microsoft-Is-Perfect:
topaz.nstc.com used to run Microsoft DNS. After a while, having tried
repeatedly to get MSDNS to actually *serve* the zone files I had set up,
and having not been successful, I switched to Bind 8 and have had no
problems since.
You might want to try Bind 8.