New Outage Hits Comcast Subscribers

BetaNews:

New Outage Hits Comcast Subscribers
http://www.betanews.com/article/New_Outage_Hits_Comcast_Subscribers/1113367699

- ferg

During the first outage this week I used Bluetooth DUN via my Treo to
dial-up from home and check Comcast's customer support web page. There was
a note on the "network health" page stating that Internet access was down
for all cable modem subscribers.

Uh no, it wasn't down - just their DNS was down (which I suppose to most is
the same as the Internet being down). Of course, none of Comcast's
subscribers could actually get to that page unless they had an alternative
service or could manually modify their DNS settings to point elsewhere. (I'm
reminded of Less Nessmen's famous broadcast announcing that WKRP was off the
air).

Irwin

Dear Comcast,

Let me inform you of an exciting new concept... Anycast DNS... It is not difficult... Get with the freaking program...

Peter

I attempted to get DNS deployed under anycast when I worked there. As you can see, I don't work there any more. Draw your own conclusions.

Let me inform you of an exciting new concept... Anycast DNS... It is
not difficult... Get with the freaking program...

Are you suggesting that network operators should supply
their customers with recursive DNS services by operating
DNS servers on their network which share the same anycast
addresses as the recursive DNS servers on other networks?

Or are you suggesting that a network operator should set
up anycast internal to their network so that all of their
recursive DNS servers share the same IP address?

I'd like to hear some more detail on this.

--Michael Dillon

Let me inform you of an exciting new concept... Anycast DNS... It is
not difficult... Get with the freaking program...

* Michael.Dillon@radianz.com (Michael.Dillon@radianz.com) [Thu 14 Apr 2005, 12:03 CEST]:

Are you suggesting that network operators should supply
their customers with recursive DNS services by operating
DNS servers on their network which share the same anycast
addresses as the recursive DNS servers on other networks?

Or are you suggesting that a network operator should set
up anycast internal to their network so that all of their
recursive DNS servers share the same IP address?

I'd like to hear some more detail on this.

Michael, put down the crackpipe already, will you? *Of course* the
previous (unattributed) poster was not talking about hijacking other
people's address space, but suggested that it's a good idea to not make
your entire customer base reliant on two puny servers somewhere.

I find this obvious.

  -- Niels.

> Are you suggesting that network operators should supply
> their customers with recursive DNS services by operating
> DNS servers on their network which share the same anycast
> addresses as the recursive DNS servers on other networks?
>
> Or are you suggesting that a network operator should set
> up anycast internal to their network so that all of their
> recursive DNS servers share the same IP address?
>
> I'd like to hear some more detail on this.

Michael, put down the crackpipe already, will you? *Of course* the
previous (unattributed) poster was not talking about hijacking other
people's address space, but suggested that it's a good idea to not make
your entire customer base reliant on two puny servers somewhere.

Maybe you should reread the two messages.

The poster suggested that anycast was the way to make
sure that your userbase does not have two rely on
two puny servers somewhere for recursive DNS services.

So, the question remains, does the poster think that
network operators should band together and operate
shared anycast recursive DNS services? Or does the poster
think that network operators should operate many
recursive DNS servers throughout their infrastructure
and tie them together using anycast?

Or was it something else?

If anycast is a good idea for recursive DNS service, then
there is a 3rd party business opportunity here to operate
global recursive DNS services so that network operators can
focus on running the network, not on providing services
like DNS resolution.

--Michael Dillon

So, the question remains, does the poster think that
network operators should band together and operate
shared anycast recursive DNS services? Or does the poster
think that network operators should operate many
recursive DNS servers throughout their infrastructure
and tie them together using anycast?

I don't know what the other poster(s) were referring to, but I was not suggesting that network operators try to run some unified DNS infrastructure. It is an intriguing idea, however.

If anycast is a good idea for recursive DNS service, then
there is a 3rd party business opportunity here to operate
global recursive DNS services so that network operators can
focus on running the network, not on providing services
like DNS resolution.

Perhaps even more interesting is being able to sell anycasted reverse DNS service directly to users that are connected to incompetent providers. :wink: Seriously, though, some benefits can be imagined, like being able to use the same DNS server on my laptop no matter where in the world I plug in.

people do that today.... unforunately they do it with 198.6.1.1 :frowning:

Its called DHCP/PPP, both will auto-magically configure the correct DNS
for your current network connection. If your laptop changes IP addresses,
it should get new network configuration details for the current network.

I have completely given up on relying on Comcast for dns service... For now I will continue to use them for "transit"

If they are unwilling to implement anycast dns then I cannot trust them... On my mac...

sudo vi /etc/hostconfig
DNSSERVER=-YES-
:wq

No wonder entrenched broadband ISPs are so against metro wifi...

Peter

It's unclear why anycast would be required. Most or all of their customers use DHCP to obtain address information, including DNS information. It would be just as reasonable for them to install a few small DNS servers along-side the router at the cable head-end at every town. Now it might be simpler for them to manage if they placed those same servers but used Anycast, but the effect should be the same.

The point is, anycast is not the issue. Reliable service is the issue. DNS isn't their only issue, of course (that they're single-homed to AT&T adds to their unreliability, not that they can fix that at present).

Dan

The deeper issue is that most Comcast customers (and I am one) don't
have an SLA, don't have a pressing need, and don't really care to pay
more for a resilient infrastructure at Comcast (or any other +90% home
provider). If I wanted to run a business out of my home I would bring
in some SLA backed bandwidth. I don't think that is unreasonable.

When Comcast goes down at home I hop in the car or walk a few blocks to
one of many wifi outlets (some even free). Yes, that does make it
difficult to check email or pay bills at 6am in my bath robe, but it
works.

Would I *like* to see Comcast 100% bulletproof? Sure. Do I *need* to
see Comcast 100% bulletproof? No.

Just my $.02

-Jim P.

Do you understand anycast? Do you understand how different operating systems react to failures of configured dns servers?

You really need to look into anycast and see why it is used. Perhaps the comcast people are as naive as you about dns... Check out:
http://www.net.cmu.edu/pres/anycast/

or my favorite: http://www.net.cmu.edu/pres/lisa03/

This excellent presentation will help you with your understanding:

"In configuring multiple hosts to respond to the same address, stateless protocols such as DNS can be easily scaled. Servers can be located in closer proximity to clients, providing faster responses to queries. In the event of a single host failure, routes can quickly be withdrawn and servers in other locations handle the request traffic, all without any changes to client configurations.

  Recursive DNS clients built into many of today's operating systems deal rather poorly with a failure of their primary recursive server. Of eight operating systems evaluated in a recent survey, seven kept no history of failed servers, trying each DNS query against the first server and waiting for a response before moving to secondary servers. Using anycast, service is maintained even in the face of a single or multiple host failure. This substantially reduces resolution delays due to server failure."

Peter Hill

I just don't want my wife to complain to me that she could not check her email because "the Internet was broken"

:slight_smile:

is attacking people necessary? I think Daniel's point was: "Perhaps
anycast isn't te only answer"

Perhaps it's not even appropriate for comcast's
network/design/pop/hub/area/infrastructure... Don't get me, or I think
Daniel, wrong, anycast is fun, but it's not for everyone. The main goal
for comcast should be stability, regardless of how they implement that,
eh?

No, they want to maximize the pricepoint/stability function (which has
multiple maxima/minima corresponding to various market niches). They want
to pay as little as possible for sufficient stability to not torque off the
customer base into migrating to another provider.

In fact, I'd not even be surprised if the answer is different for different
market regions - the needed stability in areas they're a near-monopoly at
their price range is probably a lot lower than what they need in areas where
there's lots of competitors.

5 nines impresses everybody on this list. 4 nines will probably cover most
business customers. 3 nines for most consumers, and I bet you can make a living
selling 2 nines to Joe Sixpack for $9.95/mo....

But you can't make a living selling to the 2-nines crowd at the 2-nines price
with a 5-nines infrastructure (Anybody who can, drop me a note if you've got
a POP near me :wink:

But you can't make a living selling to the 2-nines crowd at the 2-nines
price
with a 5-nines infrastructure (Anybody who can, drop me a note if you've
got
a POP near me :wink:

But Comcast sure makes a great profit by charging a 2 or 3-nine's price for
a 1.5-nine service :wink:

/Alex K.

oops, I wasn't clear what I meant and didn't type properly was: "the goal
for comcast is stability, you don't care how they achieve that" which
still isn't 100% correct, but basically let them do their operations and
lets not pot-shot something that 'we' don't understand. I'm assuming that
peter hill is not a comcast employee, his email headers suggest a CMU grad
and a current Amazon employee.

go tartans.

Do you? Relying 100% on anycast is MUCH worse than not deploying anycast at all. Spend some time thinking about various failure modes. (*sigh* just read NANOG archives if you want the short cut)

There is more than one solution to every problem. Don't fixate on anycast purely because your university hosts a couple of web pages on it.