hi,
I am asked to build a large lab/test it. I'm provided crazy scale numbers
for lots of technologies (L*VPN, IPv*, IGP*, All Tunnels flavors...etc).
It took me a lot of time to build this lab, because when I got the
request/test plan handed over to me, I did not verify that these scaled
numbers are even possible, not to mention the combination. I assumed some
thought/research were done before.
I'm trying to put together a list of the lessons learned, and the right way
to do this for future reference, specially that this project was time
critical and I got beaten hard because I did not deliver on time.
So my question is, in your extensive experience, what is the right
method/approach to this kind of task:
1) Get started immediately (MVP), things will break, tune it along the way.
2) Do some planning and research first.
I'd appreciate any references to 'software engineering' or other industries/
Thanks
We never design in a vacuum. There's always some
target we're designing towards. Testing is no different.
Think about what it is you'll need to support. Look at
historical numbers related to those features/capabilities.
Yes, as the stork market keeps reminding us, past
performance is no guarantee of future results...but at
the same time, those who don't learn from the past
are doomed to re-implement it...poorly.
So, when we test, we look at protocols we've already
been running for years, and then we look at the growth
curves we've seen in those protocols over the past X years,
where X is approximately the estimated lifespan of the
hardware in question. So, if the current router platform
you're looking to replace has been in place in your
network for 8 years, and you're testing the next
generation for BGP route scaling, look at what
the global BGP table size was 8 years ago,
and look at where it is today; work out the percentage
growth curve for it; then take the current BGP table
size, apply the same compound growth percentage
to it for the next 8 years, and you'll come up with a
reasonable idea of the scale you'll need the box
to handle over its lifetime. Test that; then, to give
yourself a margin of error, double the number, and
test again. That way you have a realistic idea of
whether it can support your current growth rate,
and whether it can support your growth if the
growth rate is 1.4x what you expect.
Do those calculations for each of the protocols
under test, and you'll be able to come up with
a reasonable testing profile that's supportable
based on historical information, rather than flights
of fancy.
Hope that helps!
Matt
I'll second that! How can you do it any other way and get any sort of reliable data...especially a POC. Seems like you would waste a lot of time just plodding forward without doing the research.
This may not be an answer very specific to your problem/question, but if
you take a look at the following image, you will find a summary of what
they called the engineering design methodology:
http://www.cdn.sciencebuddies.org/Files/5083/9/2013-updated_engineering-method-steps_v6b.png
You can adapt it to your circumstances, for example: instead of defining a
problem in step 1, you can define a product, and after knowing what is
expected from that product, you can then move to background research, etc.
Hope that helps.
Rafael
This may not be an answer very specific to your problem/question, but if
you take a look at the following image, you will find a summary of what
they called the engineering design methodology:
http://www.cdn.sciencebuddies.org/Files/5083/9/2013-updated_engineering-method-steps_v6b.png
Seriously thought initially that you were going to link to:
http://i2.wp.com/tamingdata.com/wp-content/uploads/2010/07/tree-swing-project-management-large.png
Agreed, the margin of growth never seems to stay consistent or be what you predict.
Instead of asking "What do you want?" (or taking their specs). I have lots of ideas of what I want but maybe not what I need.
Ask these questions, they've gotten me better answers and have allowed me to do the hardware/software framework not the end user/client.
"What are you trying to sell?"
"What are you trying to do?"
Brian
Howdy,
The history of the Panama Canal (both the French failure and the
deadly first two years of the U.S. effort) should offer insight (and
appropriate anecdotes) as to why this is just about always the worst
answer.
The same history also shows how planning moves from the French effort
(an impossible sea-level canal through a river valley that suffers
seasonal floods) to the entirely different engineering which produced
a gigantic man-made lake with flood control dams plus a gravity-driven
lock system to move vessels from sea level to the lake level and back.
They did it very wrong. Then they redid it very right salvaging only a
little of the original effort.
Regards,
Bill Herrin
hahaha, that's a good one, remember seeing it a long time ago, i saved it
now.
