YouTube IP Hijacking

Mailman List Mirror (Read Only)
1

As you guys probably know Youtube's IP's are being hijacked. Trace:
~ $ host youtube.com
youtube.com has address 208.65.153.253
youtube.com has address 208.65.153.238
youtube.com has address 208.65.153.251
[Same /24]

701 3491 17557
    64.74.137.253 (metric 1) from 66.151.144.148 (66.151.144.148)
      Origin IGP, metric 100, localpref 100, valid, external
      Community: 65010:300
      Last update: Sun Feb 24 11:33:05 2008 [PST8PDT]
3491 17557
    216.218.135.205 from 216.218.135.205 (216.218.252.164)
      Origin IGP, metric 100, localpref 100, valid, external, best
      Last update: Sun Feb 24 10:47:57 2008 [PST8PDT]

So, it seems that youtube's ip block has been hijacked by a more
specific prefix being advertised. This is a case of IP hijacking, not
case of DNS poisoning, youtube engineers doing something stupid, etc.
For people that don't know. The router will try to get the most specific
prefix. This is by design, not by accident. This is a case of censorship
on the internet. Anyways, I hope this doesn't get into a political
situation, and someone stops this.

What action are you going to take? Are you going to filter
announcements from AS17557, or just filter that specific announcement?
Considering youtube is a fairly high-traffic website I think that other
operators are just going to start filtering that AS. This is a great
example of global politics getting in the way of honest corporatism.
This is also an example of how vulnerable the internet is, and how lax
providers are in their filtering policies. I don't know how large
Pakistani Telecom is, but it I bet its not large enough that PCCW should
be allowing it to advertise anything.

2

Sargun Dhillon wrote:

So, it seems that youtube's ip block has been hijacked by a more
specific prefix being advertised. This is a case of IP hijacking, not
case of DNS poisoning, youtube engineers doing something stupid, etc.
For people that don't know. The router will try to get the most specific
prefix. This is by design, not by accident.

You are making the assumption of malice when the more likely cause is one of accident on the part of probably stressed NOC staff at 17557.

They probably have that /24 going to a gateway walled garden box which replies with a site saying 'we have banned this', and that /24 route is leaking outside of their AS via PCCW due to dodgy filters/communities.

Will

3

Pakistan is deliberately blocking Youtube.

http://politics.slashdot.org/article.pl?sid=08/02/24/1628213

Maybe we should all block Pakistan.

4

Sounds more like a typo on a filter over at AS17557
than anything else.

http://ca.news.yahoo.com/s/afp/080224/world/denmark_media_islam_pakistan_internet_youtube

-r

5

While they are deliberately blocking Youtube nationally, I suspect the wider issue has no malice, and is a case of poorly constructed/implemented outbound policies on their part, and poorly constructed/implemented inbound polices on their upstreams part.

6

Looks like it just went back to normal:

cr1-sea-A>show ip bgp 208.65.153.253
BGP routing table entry for 208.65.153.0/24, version 41150187
Paths: (3 available, best #3)
Flag: 0x8E0
  Advertised to update-groups:
     1 3 4 6 13 14
16
  3356 3549 36561, (Received from a RR-client)
    208.76.153.126 (metric 110) from 208.76.153.126 (208.76.153.126)
      Origin IGP, metric 0, localpref 50, valid, internal
      Community: 3356:3 3356:22 3356:86 3356:575 3356:666 3356:2011
3549:4142 3549:30840 11404:1000 11404:1030
  2914 3549 36561, (Received from a RR-client)
    208.76.153.125 (metric 310) from 208.76.153.125 (208.76.153.125)
      Origin IGP, metric 0, localpref 49, valid, internal
      Community: 2914:420 2914:2000 2914:3000 11404:1000 11404:1010
  3491 3549 36561
    63.216.14.137 from 63.216.14.137 (63.216.14.9)
      Origin IGP, localpref 51, valid, external, best
      Community: 3491:2000 3491:2003 3491:3549 11404:1000 11404:1020
cr1-sea-A>

Probably worth noting that the performace at least from our perspective
(via PCCW) is abysmal. As a side note, I know PCCW allows unfiltered
route-announcement capability to a large number of their customers, our
feed appears to be that way (or they apply RADB filters instantly which
would be a bit impressive).

John van Oppen
Spectrum Networks LLC
206.973.8302 (Direct)
206.973.8300 (main office)

7

Clearly, they are incensed by youtube content, so what makes anyone
think that they would not be trying to engage in a case of Cyber-Jihad?

I hosted the site that was rated #1 on Google for the Jyllands Posten
(di2.nu) cartoons when it was a current issue, and I STILL get lots of
script kiddie DOS from the Islamic world.

I generally don't assume malice when mere incompetence will suffice, but
in the case of the Islamic world, they've proved themselves malicious
towards the non-Islamic world often, and violently, enough, that I don't
believe they deserve that presumption of innocence any more.

In either case, the correct COA is to filter all advertisements with AS
17557 in the path, until they fix the routes they are advertising, and
let us know how they plan on making sure this doesn't happen again.

8

Tomas L. Byrnes wrote:

Clearly, they are incensed by youtube content, so what makes anyone
think that they would not be trying to engage in a case of Cyber-Jihad?

Because this usually doesn't work very well, is very evident, and easily fixed? Even on a sleepy Sunday, it took 3491 about two hours to filter/turn down 17557 and remove the problem. I bet most of their peers say that's too slow, however :slight_smile:

I generally don't assume malice when mere incompetence will suffice, but
in the case of the Islamic world, they've proved themselves malicious
towards the non-Islamic world often, and violently, enough, that I don't
believe they deserve that presumption of innocence any more.

I think your perspective is a little off.

9

Let's avoid speculation as to the why and reserve this thread for
global restoration activity.

-M<

10

So, from the tit-bits I've picked up from IRC and first-hand knowledge,
it would appear that 17557 leaked an announcement of 208.65.153.0/24 to
3491 (PCCW/BTN). After several calls to PCCW NOC, including from Youtube
themselves, PCCW claimed to be shutting down the links to 17557. Initially
I saw the announcement change from "3491 17557" to "3491 17557 17557", so
I speculate that they shut down the primary link (or filtered the announcement
on that link), and the prefix was still coming in over a secondary link
(hence the prepend). After more prodding, that route vanished too.

Various mitigations were talked about and tried, including Youtube announcing
the /24 as 2*/25, but these announcements did not seem to make it out to the
world at large.

Currently Youtube are announcing the /24 themselves - I assume this will drop
at some time once it's safe.

It was noticed that all the youtube.com DNS servers were in the affected /24.
Youtube have subsequently added a DNS server in another prefix.

Simon

11

I think it was NOT a typo. This was a test, much more important test for this world than last american anti-satellite missile.

And if they do it again with more mind, site will became down for a weeks at least... More of that, if big national telecom operator did it and have neighbors to filter them out - it can lead to global split of the network.

Of course, it should be happened early or late with THIS design of the Network.

Ravi Pina wrote:

12

http://www.google.com/reader/m/view/?source=mobilepack&v=2.1.4&rlz=1H2GGLE_en&i=-3701578819353178822&c=CMOjuszq3ZEC&n=1

13

First the operational portion:

For all the affected network owners, please read and start using/implement one of the following excellent ideas:

* Pretty Good BGP and the Internet Alert Registry
   http://www.nanog.org/mtg-0606/pdf/josh-karlin.pdf

* PHAS: A Prefix Hijack Alert System
   http://irl.cs.ucla.edu/papers/originChange.pdf
   (A live/direct BGP-feed version of this would be neat)

* Routing Registry checking, as per the above two
   rr.arin.net & whois.ripe.net contains all the data you need
   Networks who are not in there are simply not important enough to
   exist on the internet as clearly those ops folks don't care about
   their network...

Of course there is also (S-)BGP(-S), but that will apparently never happen, and actually, with the a system like PGBGP or PHAS one already covers quite a bit of the issue, until a real hijacker just uses the original ASN. IRR data helps there partially though as it tends to have upstream/downstream information, but it doesn't cover all cases.

For the rest google(bgp monitor hijack) for a list of other things.

Now for the sillynesss....

<non-ops political blabla FUD>

Max Tulyev wrote:

I think it was NOT a typo. This was a test, much more important test for this world than last american anti-satellite missile.

And if they do it again with more mind, site will became down for a weeks at least... More of that, if big national telecom operator did it and have neighbors to filter them out - it can lead to global split of the network.

Of course, it should be happened early or late with THIS design of the Network.

Oh boy oh boy, I just have to comment on this :slight_smile:

Wow, somebody with an email address like yours, especially the president and the .su bit are amusing, is commenting on another country doing 'tests'!? You might actually try keeping your bombers closer to the shores instead of trying to play chicken with the USS Nimitz :slight_smile:

http://www.upi.com/NewsTrack/Top_News/2008/02/11/russian_bomber_buzzes_nimitz/5914/

In Soviet Russia the Internet hijacks you?

Please folks, keep the posts operational :slight_smile:

</non-ops political blabla FUD>

Greets,
  Jeroen

14

For us who actually have customers we care about, we probably find it better for business to try to make sure our own customers can't announce prefixes they don't own, but accept basically anything from the world that isn't ours.

Using pure RR based filtering just isn't cost efficient today, as these borks (unintentional mostly) we see sometimes are few and fairly far between, but problems due to wrong or missing information in the RRs is plentyful and constant.

15

Jeroen Massar wrote:

* PHAS: A Prefix Hijack Alert System
  http://irl.cs.ucla.edu/papers/originChange.pdf
  (A live/direct BGP-feed version of this would be neat)

Does PHAS still work? I tried to submit a request to subscribe a few weeks ago and never heard back from their automated system. I figured the project was terminated but the site was still up.

Justin

16

You are a distinct minority. My experience has shown that most ISPs don't give a sh*t about filtering what their customers can announce so what has happened, will continue to happen.

-Hank

17

having built an ISP or two in pakistan, PTCL (Pakistan Telecom) is not the
sole provider of bandwidth to the country, although it likely carries the
bulk of traffic to the country.

operationally, there are a number of jurisdictions which filter content
and connectivity on a variety of basis.

adjusting the BGP announcements is a fairly quick and sure way to hobble
connectivity to specific content. although, it is quickly bypassed by
shifting the content to other addresses and domain names.

i'm sure that this was an accidental leakage, and that appropriate corrections
were/are taken in due course.

18

Interesting that (according to Renesys) BT reconnected about 500 networks in Pakistan after the big fibre cut. I wonder if there’s any data around that would tell us who filters and who doesn’t?

19

Interesting that (according to Renesys) BT reconnected about 500 networks in
Pakistan after the big fibre cut. I wonder if there's any data around that
would tell us who filters and who doesn't?

based on my experience of routing (and de-routing) my own legacy space as
well as some RIPE space through PTCL, i know they have procedures in place to
restrict what their customers can send to them, so it makes sense that they
have a clue as to how to control what they send out.

probably fat fingers, and probably fat wobbly fingers in a rush to comply with
a government directive.

20

I've only dealt with a handful of the bigger networks, but every transit BGP session I've ever been the customer role on has been filtered by the provider. From memory and in no particular order, that's UUNet, Level3, Digex, Intermedia, Global Crossing, Genuity, Sprint, Above.net, Time Warner, C&W, MCI, XO, Broadwing, and a few smaller ones nobody's likely to have heard of.

As an ISP providing transit, all of our customers get prefix-filtered.