Announcing long AS-sets tomorrow

Mailman List Mirror (Read Only)
1

Hi,

as part of our AS-set stuffing experiments (announced, including links to in-depth information, in [1]), we will be announcing unusually large AS-sets tomorrow, Thursday 16 June.

The prefixes involved will be 84.205.73.0/24 and 84.205.89.0/24, both orignating in AS12654. The AS-sets will consist of AS12654 repeated n times, thus the paths will look like 12654 {12654, 12654, ..., 12654}. No other AS numbers will be used. The values of n we will use are 25, 50, 75 and 100.

The announcements are designed to discover how far large AS-sets are propagated, and thus how effective our techniques can be, in today's Internet with today's IPv4 operational practices. This is *not* a test to see if routers will fall over: we have successfully tested longer AS-sets in the lab on both Cisco and Juniper, and longer AS-paths and AS-sets have been observed in the wild [2,3]. See [1] for more information on the safety of these announcements.

The proposed schedule is as follows:

               84.205.73.0/24 84.205.89.0/24
14:00 UTC: 25-element AS-set 50-element AS-set
14:30 UTC: withdrawal withdrawal

16:00 UTC: 75-element AS-set 100-element AS-set
16:30 UTC: withdrawal withdrawal

If anyone should see a problem during the announcements, please contact me and I will take immediate action.

Regards,
Lorenzo

[1] http://www.merit.edu/mail.archives/nanog/2005-06/msg00210.html
[2] http://www.ripe.net/projects/ris/Talks/0101_RIPE38_AA/sld003.html
[3] http://www.ripe.net/maillists/ncc-archives/ris-users/2002/msg00044.html

2

Hi,

as part of our AS-set stuffing experiments (announced, including links
to in-depth information, in [1]), we will be announcing unusually large
AS-sets tomorrow, Thursday 16 June.

It would be *very* nice if you could announce this at least a week in
advance instead of "tomorrow we are going to abuse the internet for some
weird testing".

Then folks could add some filtering to discard your announcements.

This is *not* a test
to see if routers will fall over: we have successfully tested longer
AS-sets in the lab on both Cisco and Juniper, and longer AS-paths and
AS-sets have been observed in the wild [2,3].

Unfortunately, especially in the case of Cisco's, there are a lot of
varying versions, with each and every one of them their own special
little bugs. Also don't forget that there are actually people using
other setups than C and J boxes. Not that it would be bad when they
would fall over, it would be bad if they would keep on transmitting
these paths, like what Cisco's tend to do when not forwarding to other
peers that the prefix was actually retracted.

PS: Don't forget to notify our asian colleagues.

Greets,
Jeroen

3

Lorenzo Colitti wrote:

as part of our AS-set stuffing experiments (announced, including links to in-depth information, in [1]), we will be announcing unusually large AS-sets tomorrow, Thursday 16 June.

Hi,

due to unforeseen technical difficulties, we have been forced to postpone these experiments. We plan to make the announcements at the same times on Monday 20 June.

The prefixes will be the same (84.205.73.0/24 and 84.205.89.0/24) and will be originated by AS12654 as before, but the AS-set will consist of AS2121 repeated n times, so the paths will look like 12654 {2121, 2121, ..., 2121}. AS2121 is the RIPE meeting AS, which is reserved for RIPE meetings and does not currently appear in the global routing table.

As before, should anyone encounter a problem with these experiments, please let me know and I will take immediate action.

Regards,
Lorenzo

4

Yeah, add more monday morning trouble, people will love to get to work
then :wink:

Btw, if you postponed the 'experiment', how come I did pick up this one:

84.205.73.0/24 12654 12654
{1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,
1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,
1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,
1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,
1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,
1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,
1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,
1221,1221,1221,1221,1221}

Greets,
Jeroen

PS: Is the 'technical difficulty' your own router falling over? :slight_smile:

5

June 15th: Lorenzo gives us 24 hours notice that he is going to be using
our (a very general our here, meaning all Internet operators) network for
performing his experiments on. (oh, and points out that hes been doing the
same with IPv6 since last year, just unannounced, but thats okay because
noone noticed)

June 15th: Those that check their email frequently enough to spot this ask
Lorenzo for at least a week notice before doing this in future.

June 20th: Lorenzo gives us same day notice that he will be using our
network as his plaything again. Anyone sufficiently behind Lorenzo in the
grand scheme of things (either they have better things to do than read
their email all day, or perhaps they're on the west coast USA) won't even
know about this until it is too late. If something strange happens I'm not
sure everyone will suddenly think better check to see if Lorenzo is
playing again.

I see the use of the Internet for experiments like this to be somewhat
frivilous, and the notice periods given as warning even more so. I'm sure
Lorenzo would not appreciate if I were to give 20 minutes warning of some
clandestine experiment, such as announcing more specifics of his
institutions prefixes as particularly helpful!

he is announcing his own bleedin' prefixes. get a life

randy

6

Jeroen Massar wrote:

Btw, if you postponed the 'experiment', how come I did pick up this one:

84.205.73.0/24 12654 12654 {1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,
1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,
1221,1221,1221,1221,1221}

That path was announced during the window of notice we had given for the announcements. However, you will notice that that was not the complete set of announcements we intended to make, which included 25-, 50-, 75-. and 100-element AS-sets.

Since we were not able to send all the announcements within the window of notice we had provided, we postponed them to avoid sending announcements when people were not expecting them.

PS: Is the 'technical difficulty' your own router falling over? :slight_smile:

No. :slight_smile:

Regards,
Lorenzo

7

Jeroen Massar wrote:
>Btw, if you postponed the 'experiment', how come I did pick up this one:
>
>84.205.73.0/24 12654 12654
>{1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,
> 1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,
> 1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,
> 1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,
> 1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,
> 1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,
> 1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,
> 1221,1221,1221,1221,1221}

That path was announced during the window of notice we had given for the
announcements. However, you will notice that that was not the complete
set of announcements we intended to make, which included 25-, 50-, 75-.
and 100-element AS-sets.

By the way, accoording to your annoucement:

The prefixes involved will be 84.205.73.0/24 and 84.205.89.0/24, both
orignating in AS12654. The AS-sets will consist of AS12654 repeated n
times, thus the paths will look like 12654 {12654, 12654, ..., 12654}.
No other AS numbers will be used. The values of n we will use are 25,
50, 75 and 100.

What's the '1221' doing there ?

Grtx,

MarcoH

8

That is the RIPE Meeting AS apparently, which is not used at the moment.
This was mentioned in todays mail (not in the original one).

According to whois though, I hope that you enlightened Geoff of taking
over his AS:
aut-num: AS1221
as-name: ASN-TELSTRA
descr: Telstra Pty Ltd

I also do hope that people are not going to filter out 1221 btw as that
would sever connectivity of that AS when it is needed.

Another thing to note is that neither of those two AS's have any
reference to these experiments in whois.

aut-num: AS12654
as-name: RIPE-NCC-RIS-AS
descr: RIPE NCC RIS Project.
descr: http://www.ripe.net/ris/
admin-c: HU266-RIPE
tech-c: RISM-RIPE
remarks: Different subsets of the routes in AS12654:RS-RIS are
announced
remarks: at each location.
remarks: Please send peering requests to rispeering@ripe.net.
mnt-by: RIPE-NCC-RIS-MNT
source: RIPE # Filtered

It would be nice to list it there too as mentioned before, not everybody
reads the various mailinglists and there is no mention on that site
either....

Greets,
Jeroen

9

and you forgot to add: "welcome to the internet, the largest beta-test network in the world"

-b

10

Wrong again. You used both AS1221 and AS2121:

Jun 20 17:00:45: %BGP-6-ASPATH: Long AS path 20965 1299 12654
12654
{1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,1221
,1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,1221
,1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,1221}
received from
62.40.103.69: More than configured MAXAS-LIMIT
Jun 20 17:31:14: %BGP-6-ASPATH: Long AS path 20965 1299 701 702
13030 12654 12654
{1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,1221
,1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,1221
,1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,1221,1221}
received from
62.40.103.69: More than configured MAXAS-LIMIT
Jun 20 19:00:43: %BGP-6-ASPATH: Long AS path 20965 1299 12654
12654
{2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121
,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121
,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121
,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121
,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121
received from 62.40.103.69: More than configured MAXAS-LIMIT
Jun 20 19:30:39: %BGP-6-ASPATH: Long AS path 20965 1299 3356
16034 12654 12654
{2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121
,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121
,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121
,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121
,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121,2121
received
from 62.40.103.69: More than configured MAXAS-LIMIT

-Hank

11

Hank wrote:

Wrong again. You used both AS1221 and AS2121:

I logged pretty much the same thing with max-as limit blocking/logging the announcements (of course, the paths, and time stamp seconds are different, YMMV) Perhaps the unforeseen technical difficulties have something to do with not sticking to the original plan? Announce something different to get around filters, and thus, detect who put filters in place? Or more innocent..... maybe fat fingers, or copy/paste gone horribly wrong?

Doesn't really matter what happened though, because a controlled announce is a lot better than a malicious one. Since a stupid person is the most dangerous type of person (fifth basic law of human stupidity), a malicious announcement is even safer than a clueless one. I'd much rather see this done by someone with clue that's going to announce and withdraw them, then check for damage, than by someone that might not know what the heck they're doing. If there is damage, we'll all hear about it, and figure out how to stop it in the future when someone else tries to be malicious. Or when someone else is just plain clueless.

Apparently that's not the case since this whole experiment was so disruptive that it took 16 hours for someone to notice and point out on NANOG that it neither did nor did not go off as previously announced.

(I'm guilty of looking it over in the logs, and not even noticing the difference between 2121 and 1221)

The internet is our playground... can't we just all get along? If someone's going to load 50 kids on a merry-go-round, and get 50 more kids to push it.... I'll just stand over by the monkey bars and try to avoid the flying vomit. :slight_smile:

-Jerry

12

> due to unforeseen technical difficulties, we have been forced to
> postpone these experiments. We plan to make the announcements at the
> same times on Monday 20 June.
>
> The prefixes will be the same (84.205.73.0/24 and 84.205.89.0/24) and
> will be originated by AS12654 as before, but the AS-set will consist of
> AS2121 repeated n times, so the paths will look like 12654 {2121, 2121,
> .., 2121}. AS2121 is the RIPE meeting AS, which is reserved for RIPE
> meetings and does not currently appear in the global routing table.

Wrong again. You used both AS1221 and AS2121:

Yups, smells like a small but very dangerous typo.

From a netcitizen's perpesctive and as being involved in operating a part

of the internet, I'm starting to dislike this whole experiment more and
more. I understand, as people already commented, the internet in fact is
one big experiment, but this is getting out-of-hand.

Can the people responsible for this please reconsider and put things on
hold until certain conditions are met:

Publish a detailed workplan including AS-es used, windows and risk
analysis to the various mailinglists.

A reasonable time between annoucements and the experiment instead of 24
hours.

Some assurance that input files are checked for typos.

Take another look wether it is smart to use 'production' AS-es for it,
such as the RIS or meeting AS numbers, instead of a seperate set. I think
people are going to get real unhappy if somewhere in October they found
out the meeting network is blocked at various places.

Just my 2 cents,

MarcoH

13

Can the people responsible for this please reconsider and put things on
hold until certain conditions are met:
Publish a detailed workplan including AS-es used, windows and risk
analysis to the various mailinglists.

given they are using their own prefixes, can you please tell
us what risk there might be.

Some assurance that input files are checked for typos.

hopefully better than the mean of operators doing so :-)/2

randy

14

Not much, but I guess people in the audience might get happy from the
small note that labtests showed IOS won't crash when it encounters these
annoucements.

MarcoH

15

Can the people responsible for this please reconsider and put things on
hold until certain conditions are met:
Publish a detailed workplan including AS-es used, windows and risk
analysis to the various mailinglists.

given they are using their own prefixes, can you please tell
us what risk there might be.

Not much, but I guess people in the audience might get happy from the
small note that labtests showed IOS won't crash when it encounters these
annoucements.

showing that ios won't crash is very difficult because the number
of versions of ios, and the amazing dependencies of things on which
blade is in which slot and what phase is the moon.

but reading the roma gang's papers and the main email note leads me
to feel they have done as good a job on this as we can reasonably
expect.

considering that we have fellow isps dumping horrifying garbage in
the rib, it's amusing how we attack a seemingly well-run very small
experiment.

randy

16

Randy Bush wrote:

showing that ios won't crash is very difficult because the number
of versions of ios, and the amazing dependencies of things on which
blade is in which slot and what phase is the moon.

Thank you. You've provided a clean, concise counter to Lorenzo's original claim that long AS sets won't trip on IOS bugs.

This may be a well-run, very small experiment, but it's experimenting with a space that's rarely explored and therefore less likely to have encountered the same level of operational testing that horrifying garbage leaks have tested. As such, I'm frustrated that the testers consider requests to provide more advance notice to be so "obtuse".

Many of us in the operational community are required to conduct testing in lab environments, followed by well-announced maintenance windows. Why is this operational test supposed to be given freer reign on the 'net than our own operations? Alternatively, why can't the test be conducted in a lab, with interested operators providing router configurations and xOS versions to give the test bed the most realistic sample of the 'net, without using the production 'net?

pt

17

Randy Bush wrote:

showing that ios won't crash is very difficult because the number
of versions of ios, and the amazing dependencies of things on which
blade is in which slot and what phase is the moon.

Thank you. You've provided a clean, concise counter to Lorenzo's original claim that long AS sets won't trip on IOS bugs.

..

Alternatively, why can't the test be conducted in a lab, with interested operators providing router configurations and xOS versions to give the test bed the most realistic sample of the 'net, without using the production 'net?

Has anyone considered that the project may have indeed done testing of available IOS/$ROUTER versions in a lab environment before even considering testing on the 'live' internet? Reading the cited material might be of benefit to the vocal complainers.

I think Lorenzo would be the first to admit that the timing of operator notifications, and more importantly the wording, may be less than desired, however this does not detract from the caution, and professionalism exhibited thus far in this set of experiments.

This may be a well-run, very small experiment, but it's experimenting with a space that's rarely explored and therefore less likely to have encountered the same level of operational testing that horrifying garbage leaks have tested.

Part of the problem is that because it is, as you put it, a rarely explored problem space, the number of interested parties with sufficient and varied resources is extremely small, resulting in a less-than-complete testing environment.

Another part of the problem is that you cannot put the concept back in the box. The black-hats do read operational lists, and with all the fuss being made over possible breakage caused by AS-sets, some of them will do will perform their own, possibly destructive experiments in order to find out what specific $ROUTER versions do under various inputs.

So, which would you prefer.. Lorenzo at a known contact number with known working hours (+31 20 535 4444, 10am to 5pm GMT +1), and with the Internet's best interests at heart, or some malcontent with unknown contacts, unknown hours, and very definitely not your best interests at heart ?

18

Date: Tue, 21 Jun 2005 14:40:47 +0100
From: Randy Bush

[ trimming CC list ]

considering that we have fellow isps dumping horrifying garbage in
the rib, it's amusing how we attack a seemingly well-run very small
experiment.

Bears would rather attack fish than wolverines.

Considering Lorenzo's attitude, I'm sure he's taking into account the
requests for more heads up. If he tickles an IOS bug, I'd rather have
it happen in this scenario than when a less-clued individual or a
miscreant tries announcing wacky routes.

Eddy

19

Thank you. You've provided a clean, concise counter to Lorenzo's
original claim that long AS sets won't trip on IOS bugs.

no problem. you're quite welcome.

This may be a well-run, very small experiment, but it's experimenting
with a space that's rarely explored and therefore less likely to have
encountered the same level of operational testing that horrifying
garbage leaks have tested. As such, I'm frustrated that the testers
consider requests to provide more advance notice to be so "obtuse".

could you please give me the command to configure ios to not crash
if given advance notice?

Why is this operational test supposed to be given freer reign on the
'net than our own operations? Alternatively, why can't the test be
conducted in a lab, with interested operators providing router
configurations and xOS versions to give the test bed the most realistic
sample of the 'net, without using the production 'net?

the first announcement of this experiment was months ago.

randy

20

Not everyone runs IOS. Wasn't it something similar to this that crashed
gated and possibly other BGP implementations a few years ago? Is this
test intended to make sure everyone upgraded / nobody's deployed new gear
with old affected code?

And finally, "we're doing it", "we're not doing it", "Surprise, we did it"
is a crappy way to "notify" the community that they're about to piss in
the global pool. At least there was some level of notification, but why
bother if you're not going to stick to what you publicize?