Overall Netflix bandwidth usage numbers on a network?

Mailman List Mirror (Read Only)
1

Been lurking for a while and posed a question to a few folks without much
response, figured someone here might've done something like this already.

So, before I go about building wheels that already exist:

I'm interested in doing a bit of a passive survey of bandwidth usage on
my network (smallish isp, a few thousand DSL/FTTx customers) to understand
the percentage of average/overall traffic generated by Netflix streaming.

What I have available is a few gigabit transport switches providing me with
mirror ports, a juniper MX series router running 10.4 code, plenty of BSD
machines and libpcap-fu.

What I'm looking for is either a timed-average or moments-glance number
of the traffic. For instance, on an interface moving 150mbit/sec total,
50mbit/sec of it is attributed to Netflix right now. I'm pretty handy with
RRDtool, so that isn't out of the question, either.

I've really only spent dinnertime considering this, but have come up with
two potential approaches so far, and haven't actively investigated either
of them:

* firewall terms and counters on the MX router + snmp
* writing a quick libpcap application to filter and count in a completely
  out-of-band way on one of my monitoring hosts

Some challenges I can see:

* Nailing down the streaming source for Netflix, that is, IP ranges etc.
* Making assumptions about CDN source IPs that could be used for something
  else, and further, should I care?

Happy to hear thoughts about this, helpful or not! I know Netflix themselves
have probably done plenty of studies like this, but pretty likely not limited
to my customer base. Not aiming for anything creepy or crazy, just some
vague understanding of what's going on, and the ability to do some trending
for future planning.

-- Jonathan Towne

2

Surely this is what Netflow is for.

no need to re-invent the wheel.

Andrew

3

Wow.. not sure how I missed that option. Exactly why I posted before dumping
a bunch of time into a bottomless bucket!

Thanks.. :slight_smile:

-- Jonathan Towne

On Sat, Dec 03, 2011 at 12:56:34AM +0000, Andrew Mulholland scribbled:
# Surely this is what Netflow is for.

4

Also checkout Adrian Cockcroft presentations on their architecture which
describes how they use aws and CDns etc

Martin

Wow.. not sure how I missed that option. Exactly why I posted before

dumping

a bunch of time into a bottomless bucket!

Thanks.. :slight_smile:

-- Jonathan Towne

On Sat, Dec 03, 2011 at 12:56:34AM +0000, Andrew Mulholland scribbled:
# Surely this is what Netflow is for.
#
#
# no need to re-invent the wheel.
#
#
# Andrew
#
#
#
# > Been lurking for a while and posed a question to a few folks without

much

# > response, figured someone here might've done something like this

already.

# >
# > So, before I go about building wheels that already exist:
# >
# > I'm interested in doing a bit of a passive survey of bandwidth usage

on

# > my network (smallish isp, a few thousand DSL/FTTx customers) to

understand

# > the percentage of average/overall traffic generated by Netflix

streaming.

# >
# > What I have available is a few gigabit transport switches providing

me with

# > mirror ports, a juniper MX series router running 10.4 code, plenty of

BSD

# > machines and libpcap-fu.
# >
# > What I'm looking for is either a timed-average or moments-glance

number

# > of the traffic. For instance, on an interface moving 150mbit/sec

total,

# > 50mbit/sec of it is attributed to Netflix right now. I'm pretty

handy with

# > RRDtool, so that isn't out of the question, either.
# >
# > I've really only spent dinnertime considering this, but have come up

with

# > two potential approaches so far, and haven't actively investigated

either

# > of them:
# >
# > * firewall terms and counters on the MX router + snmp
# > * writing a quick libpcap application to filter and count in a

completely

# > out-of-band way on one of my monitoring hosts
# >
# > Some challenges I can see:
# >
# > * Nailing down the streaming source for Netflix, that is, IP ranges

etc.

# > * Making assumptions about CDN source IPs that could be used for

something

# > else, and further, should I care?
# >
# > Happy to hear thoughts about this, helpful or not! I know Netflix
# > themselves
# > have probably done plenty of studies like this, but pretty likely not
# > limited
# > to my customer base. Not aiming for anything creepy or crazy, just

some

# > vague understanding of what's going on, and the ability to do some

trending

5

Yeah, that's an interesting one. We currently utilize netflow for this, but you also need to consider that netflix streaming is just port 80 www traffic. Because netflix uses CDNs, its difficult to pin down the traffic to specific hosts in the CDN and say that this traffic was netflix, while this traffic was the latest windows update (remember this is often a shared hosting platform). We've done our own testing and have come to a good solution which uses a combination of nbar, packet marking, and netflow to come to a conclusion. On a ~160Mbps link, netflix peaks out between 30-50Mbps around 8-10PM each evening. The rest of the traffic is predominantly other forms of HTTP traffic (including other video streaming services).

Martin Hepworth wrote the following on 12/3/2011 2:36 AM:

6

Feel free to contact peering@netflix<dot>com - we're happy to provide you with delivery statistics for traffic terminating on your network.

Regards,
-Dave Temkin
Netflix

7

Which leads to a question to be asked...

Is netflix willing to peer directly with ISP / NSP's ?

Regards.

Faisal Imtiaz
Snappy Internet& Telecom

8

Netflix uses CDNs for content delivery and the platform runs in EC2. What would peering with them achieve?

9

I suspect Faisal's *real* question is "Who at Netflix do I talk to in order to discuss
mutually beneficial traffic engineering?"

10

Simple, keep traffic off paid ip transit circuits....

Faisal

11

Simple, keep traffic off paid ip transit circuits....

(I think joel's point was: "peer with amazon, done-and-done")

12

Thanks for the explanation...did not consider that before...will investigate.., any tips that can be shared will be welcome.
:slight_smile:

Faisal

13

Simple, keep traffic off paid ip transit circuits....

(I think joel's point was: "peer with amazon, done-and-done")

also probably your relationships to akamai and level3

14

Probably want to add Limelight to that list as well (do Netflix even use Akamai these days?)

-Shaun

15

DirectConnect seems to be a good way to get a dedicated 1G or 10G link
  with AWS:

  http://aws.amazon.com/directconnect/

  It's not settlement-free peering, but it's an option if you can't
  negotiate something. Maybe it will reduce costs in some use cases.

Beckman

16

Netflix's EC2 instances do not speak to end users AFAIK. I believe Akamai, LLNW, & L3 are the only companies that stream movies for Netflix. Peer with the CDNs to save your transit.

Happy to be educated otherwise if someone knows more than I do.

Netflix's client is also _very_ intelligent. If a user cannot get high enough quality from CDN_1, it will switch to CDN_2 without interrupting the stream. Which is nice if you have good connectivity to one but not the other CDN. (Note I spoke of "good", not "inexpensive" connectivity. The NF client doesn't know how much it costs you to show a video, only whether there is packet loss.)

17

That would be good if more than one of those CDNs peered openly.

Simon

18

Hi!

I believe Akamai,
LLNW, & L3 are the only companies that stream movies for Netflix. Peer with
the CDNs to save your transit.

That would be good if more than one of those CDNs peered openly.

So what one doesnt?

Akamai will peer with you anywhere and i doubt LLNW will give you trouble.

L3, well, they run a superb network and even more superb pricing, so why would they peer with anyone :wink:

I still think, old fashioned me, that a CDN doesnt do go without peering but some feel otherwise.

Bye,
Raymond.

19

Akamai will peer with you anywhere and i doubt LLNW will give you trouble.

LLNW are restictive on peering.

L3, well, they run a superb network and even more superb pricing, so why
would they peer with anyone :wink:

And as I use L3 for transit, they'd never peer with me. Not that they peer
with anyone outside the cabal anyway.

I still think, old fashioned me, that a CDN doesnt do go without peering
but some feel otherwise.

Quite so. I don't get why CDNs don't all peer openly. I guess most (i.e. those
which aren't Akamai) are more concerned with making money than with delivering
a good service to the end user.

Simon

20

Really? I always thought that higher profits and buying transit were mutually exclusive relative to higher profits and openly peering.

So what you are saying is that one stands to make more by paying upstreams for bit swapping? How's that work?

If the argument is that the opex required for maintaining peering relationships is too expensive relative to the direct and indirect cost of buying bandwidth, I love to be edumacated on how that math actually works because it makes absolutely no sense to me.