Cringley has a theory and it involves Google, video, and oversubscribed
backbones:
http://www.pbs.org/cringely/pulpit/2007/pulpit_20070119_001510.html
The following comment has to be one of the most important comments in
the entire article and its a bit disturbing.
"Right now somewhat more than half of all Internet bandwidth is being
used for BitTorrent traffic, which is mainly video. Yet if you
surveyed your neighbors you'd find that few of them are BitTorrent
users. Less than 5 percent of all Internet users are presently
consuming more than 50 percent of all bandwidth."
I'm not sure why you find that disturbing. I can think of two reasons, and,
they depend almost entirely on your perspective:
If you are disturbed because you know that these users are early adopters
and that eventually, a much wider audience will adopt this technology
driving a need for much more bandwidth than is available today, then,
the solution is obvious. As in the past, bandwidth will have to increase to
meet increased demand.
If you are disturbed by the inequity of it, then, little can be done. There
will always be classes of consumers who use more than other classes
of consumers of any resource. Frankly, looking from my corner of the
internet, I don't think that statistic is entirely accurate. From my perspective,
SPAM uses more bandwidth than BitTorrent.
OTOH, another thing to consider is that if all those video downloads
being handled by BitTorrent were migrated to HTTP connections
instead the required amount of bandwidth would be substantially
higher.
Owen
Rodrick Brown wrote:
Cringley has a theory and it involves Google, video, and oversubscribed
backbones:http://www.pbs.org/cringely/pulpit/2007/pulpit_20070119_001510.html
The following comment has to be one of the most important comments in
the entire article and its a bit disturbing."Right now somewhat more than half of all Internet bandwidth is being
used for BitTorrent traffic, which is mainly video. Yet if you
surveyed your neighbors you'd find that few of them are BitTorrent
users. Less than 5 percent of all Internet users are presently
consuming more than 50 percent of all bandwidth."
Moreover, those of you who were at NANOG in June will remember some of the numbers Colin gave about Youtube using >20gbps outbound.
That number was still early in the exponential growth phase the site is (*still*) having. The 20gbps number would likely seem laughable now.
-david
The Internet: the world’s only industry that complains that people want its product.
The following comment has to be one of the most important comments in
the entire article and its a bit disturbing."Right now somewhat more than half of all Internet bandwidth is being
used for BitTorrent traffic, which is mainly video. Yet if you
surveyed your neighbors you'd find that few of them are BitTorrent
users. Less than 5 percent of all Internet users are presently
consuming more than 50 percent of all bandwidth."
the heavy hitters are long known. get over it.
i won't bother to cite cho et al. and similar actual measurement
studies, as doing so seems not to cause people to read them, only to say
they already did or say how unlike japan north america is. the
phenomonon is part protocol and part social.
the question to me is whether isps and end user borders (universities,
large enterprises, ...) will learn to embrace this as opposed to
fighting it; i.e. find a business model that embraces delivering what
the customer wants as opposed to winging and warring against it.
if we do, then the authors of the 2p2 protocols will feel safe in
improving their customers' experience by taking advantage of
localization and proximity, as opposed to focusing on subverting
perceived fierce opposition by isps and end user border fascists. and
then, guess what; the traffic will distribute more reasonably and not
all sum up on the longer glass.
randy
randy
* Rodrick Brown:
"Right now somewhat more than half of all Internet bandwidth is being
used for BitTorrent traffic, which is mainly video. Yet if you
surveyed your neighbors you'd find that few of them are BitTorrent
users. Less than 5 percent of all Internet users are presently
consuming more than 50 percent of all bandwidth."
s/BitTtorrent/porn, and we've been there all along.
I think the real issue here is that Google's video traffic does *not*
clog the network, but would be distributed through private networks
(sometimes Google's own, or through another company's CDN) and
injected into the Internet very close to the consumer. No one is able
to charge for that traffic because if they did, Google would simply
inject it someplace else. At best your, one of your peerings would go
out of balance, or at worst, *you* would have to pay for Google's
traffic.
Alexander Harrowell wrote:
The Internet: the world's only industry that complains that people want its product.
The quote sounds good, but nobody in this thread is complaining.
There have always been top-talkers on networks and there always will be. The current top-talkers are the joe and jane users of tomorrow. That is what is important. BitTorrent-like technology might start showing up in your media center, your access point, etc. The Venice Project (Joost) and a number of other new startups are also built around this model of distribution.
Maybe a more symmetric load on the network (at least on the edge) will improve economic models or maybe we'll see "eyeball" networks start to peer with each other as they start sourcing more and more of the bits. Maybe that's already happening.
-david
Hello;
Cringley has a theory and it involves Google, video, and oversubscribed
backbones:http://www.pbs.org/cringely/pulpit/2007/pulpit_20070119_001510.html
The following comment has to be one of the most important comments in
the entire article and its a bit disturbing."Right now somewhat more than half of all Internet bandwidth is being
used for BitTorrent traffic, which is mainly video. Yet if you
surveyed your neighbors you'd find that few of them are BitTorrent
users. Less than 5 percent of all Internet users are presently
consuming more than 50 percent of all bandwidth."
Those sorts of percentages are common in Pareto distributions (AKA Zipf's law AKA "the 80-20 rule").
With the Zipf's exponent typical of web usage and video watching, I would predict something closer to
10% of the users consuming 50% of the usage, but this estimate is not that unrealistic.
I would predict that these sorts of distributions will continue as long as humans are the primary consumers of
bandwidth.
Regards
Marshall
Marshall wrote:
Those sorts of percentages are common in Pareto distributions (AKA
Zipf’s law AKA “the 80-20 rule”).
With the Zipf’s exponent typical of web usage and video watching, I
would predict something closer to
10% of the users consuming 50% of the usage, but this estimate is not
that unrealistic.I would predict that these sorts of distributions will continue as
long as humans are the primary consumers of
bandwidth.Regards
Marshall
That’s until the spambots inherit the world, right?
Aren't there some Telco laws wrt cross-state, but still interlata, calls
not being able to be charged as interstate? Perhaps Google wants to
avoid any future federal/state regulations by providing in-state (i.e.
"local") access. Additionally, it makes it easier to do state and local
govt business when the data is in the same state (it's not out-sourcing
if it's just nextdoor...). And then there is the "lobbying" issue, what
better way to lobby multiple states than do do significant business
their in? Or perhaps I'm just daydreaming too much today.... 
-Jim P.
That is if you see a distinction, metaphorical or physical, between
spambots and real users.
It has been a long time since I bowed before Mr. Bush's wisdom, but
indeed, I bow now in a very humble fashion.
Thing is though, it is quivalent to one or all of the following:
-. EFF-like thinking (moral high-ground or impractical at times, yet
correct and to live by).
-. (very) Forward thinking (yet not possible for people to get behind - by
people I mean those who do this daily), likely to encounter much
resistence until it becomes mainstream a few years down the road.
-. Not connected with what can currently happen to affect change, but
rather how things really are which people can not yet accept.
As Randy is obviously not much affected when people disagree with him, nor
should he, I am sure he will preach this until it becomes real. With that
in mind, if many of us believe this is a philosophical as well as a
technological truth -- what can be done today to affect this change?
Some examples may be:
-. Working with network gear vendors to create better equipment built to
handle this and lighten the load.
-. Working on establishing new standards and topologies to enable both
vendors and providers to adopt them.
-. Presenting case studies after putting our money where our mouth is, and
showing how we made it work in a live network.
Staying in the philosophical realm is more than respectable, but waiting
for FUSSP-like wide-addoption or for sheep to fly is not going to change
the world, much.
For now, the P2P folks who are not in most cases eveel "Internet
Pirates" are mostly allied, whether in name or in practice with
illegal activities. The technology isn't illegal and can be quite good for
all of us to save quite a bit of bandwidth rather than waste it (quite a
bit of redudndancy there!).
So, instead of fighting it and seeing it left in the hands of the
"pirates" and the privacy folks trying to bypass the Firewall of [insert
evil regime here], why not utilize it?
How can service providers make use of all this redudndancy among their top
talkers and remove the privacy advocates and warez freaks from the
picture, leaving that front with less technology and legitimacy while
helping themselves?
This is a pure example of a problem from the operational front which can
be floated to research and the industry, with smarter solutions than port
blocking and QoS.
Gadi.
the question to me is whether isps and end user borders (universities,
large enterprises, ...) will learn to embrace this as opposed to
fighting it; i.e. find a business model that embraces delivering what
the customer wants as opposed to winging and warring against it.
interesting.. i was about to say..
I am involved in London, in building an ISP that encourages users of
p2p with respect from major and independent record labels. it makes
sense that the film industry will (and is?) moving towards some kind of
acceptance as well.
Thing is though, it is quivalent to one or all of the following:
-. EFF-like thinking (moral high-ground or impractical at times, yet
correct and to live by).
-. (very) Forward thinking (yet not possible for people to get behind - by
people I mean those who do this daily), likely to encounter much
resistence until it becomes mainstream a few years down the road.
-. Not connected with what can currently happen to affect change, but
rather how things really are which people can not yet accept.
well, a little dash of all thinking makes for a healthy environment
doesn't it?
This is a pure example of a problem from the operational front which can
be floated to research and the industry, with smarter solutions than port
blocking and QoS.
This is what I am interested/scared by.
C.
I am involved in London, in building an ISP that encourages users of
Cool!
p2p with respect from major and independent record labels. it makes
sense that the film industry will (and is?) moving towards some kind of
acceptance as well.
Erm.. as in to help them sue users? 
well, a little dash of all thinking makes for a healthy environment
doesn't it?
Not on NANOG. :o)
> This is a pure example of a problem from the operational front which can
> be floated to research and the industry, with smarter solutions than port
> blocking and QoS.This is what I am interested/scared by.
Can you please elaborate on this point?
C.
Gadi.
Its not that hard a problem to get on top of. Caching, unfortunately, continues
to be viewed as anaethma by ISP network operators in the US. Strangely enough
the caching technologies aren't a problem with the content -delivery- people.
I've had a few ISPs out here in Australia indicate interest in a cache that
could do the normal stuff (http, rtsp, wma) and some of the p2p stuff (bittorrent
especially) with a smattering of QoS/shaping/control - but not cost upwards of
USD$100,000 a box. Lots of interest, no commitment.
It doesn't help (at least in Australia) where the wholesale model of ADSL isn't
content-replication-friendly: we have to buy ATM or ethernet pipes to upstreams
and then receive each session via L2TP. Fine from an aggregation point of view,
but missing the true usefuless of content replication and caching - right at
the point where your customers connect in.
(Disclaimer: I'm one of the Squid developers. I'm getting an increasing amount
of interest from CDN/content origination players but none from ISPs. I'd love
to know why ISPs don't view caching as a viable option in today's world and
what we could to do make it easier for y'all.)
Adrian
p2p with respect from major and independent record labels. it makes
sense that the film industry will (and is?) moving towards some kind of
acceptance as well.Erm.. as in to help them sue users?
as in - a DMZ for the RIAA vs. the User
well, a little dash of all thinking makes for a healthy environment
doesn't it?Not on NANOG. :o)
hahah.
This is a pure example of a problem from the operational front which can
be floated to research and the industry, with smarter solutions than port
blocking and QoS.This is what I am interested/scared by.
Can you please elaborate on this point?
well all I mean is that backbone networks and the technology that is
used is still a bit of a mystery to me, having never fiddled with (and
broken) it, myself.
One thing I was amazed by this week, was, looking into my ADSL that we
are providing, and seeing how low-level, and, well - tailored for a
market that could only be perceived 15 years ago.
I welcome our new masters, GOOG. 
C.
I tend to take the long view.
Some examples may be:
-. Working on establishing new standards and topologies to enable both
vendors and providers to adopt them.
Keep this point in mind while reading my below comment.
For now, the P2P folks who are not in most cases eveel "Internet
Pirates" are mostly allied, whether in name or in practice with
illegal activities. The technology isn't illegal and can be quite good for
all of us to save quite a bit of bandwidth rather than waste it (quite a
bit of redudndancy there!).
A paper put together by the authors of a download-only "free riding"
BitTorrent client, called BitThief. The paper is worth reading:
http://dcg.ethz.ch/publications/hotnets06.pdf
http://dcg.ethz.ch/projects/bitthief/ (client is here)
The part that saddens me the most about this project isn't the
complete disregard for the "give back what you take" moral (though
that part does sadden me personally) , but what this is going to
do to the protocol and the clients.
Chances are that other torrent client authors are going to see the
project as "major defiance" and start implementing things like
filtering what client can connect to who based on the client name/ID
string (ex. uTorrent, Azureus, MainLine), which as we all know, is
going to last maybe 3 weeks.
This in turn will solicit the BitThief authors implementing a feature
that allows the client to either spoof its client name or use randomly-
generated ones. Rinse lather repeat, until everyone is fighting rather
than cooperating.
Will the BT protocol be reformed to address this? 50/50 chance.
So, instead of fighting it and seeing it left in the hands of the
"pirates" and the privacy folks trying to bypass the Firewall of [insert
evil regime here], why not utilize it?
I think Adrian Chadd's mail addresses this indirectly: it's not
being utilised because of the bandwidth requirements.
ISPs probably don't have an interest in BT caching because of 1)
cost of ownership, 2) legal concerns (if an ISP cached a publicly
distributed copy of some pirated software, who's then responsible?),
and most of all, 3) it's easier to buy a content-sniffing device that
rate-limits, or just start hard-limiting users who use "too much
bandwidth" (a phrase ISPs use as justification for shutting off
customers' connections, but never provide numbers of just what's "too
much").
The result of these items already been shown: BT encryption. I
personally know of 3 individuals who have their client to use en-
cryption only (disabling non-encrypted connection support). For
security? Nope -- solely because their ISP uses a rate limiting
device.
Bram Cohen's official statement is that using encryption to get
around this "is silly" because "not many ISPs are implementing
such devices" (maybe not *right now*, Bram, but in the next year
or two, they likely will):
ISPs will go with implementing the above device *before* implementing
something like a BT caching box. Adrian probably knows this too,
and chances are it's probably because of the 3 above items I listed.
So my question is this: how exactly do we (as administrators of
systems or networks) get companies, managers, and even other
administrators, to think differently about solving this?
<snip>
ISPs probably don't have an interest in BT caching because of 1)
cost of ownership, 2) legal concerns (if an ISP cached a publicly
distributed copy of some pirated software, who's then responsible?),
They cache the web, which has the same chance of being illegal content.
<snip>
The result of these items already been shown: BT encryption. I
personally know of 3 individuals who have their client to use en-
cryption only (disabling non-encrypted connection support). For
security? Nope -- solely because their ISP uses a rate limiting
device.
Yep. Users will find a way to maintain functionality.
Bram Cohen's official statement is that using encryption to get
around this "is silly" because "not many ISPs are implementing
such devices" (maybe not *right now*, Bram, but in the next year
or two, they likely will):
I don't know of many user ISPs which don't implement them, you kidding?
<snip>