a radical proposal (Re: protocols that don't meet the need...)

[[pushed the wrong button last time. This is the complete reply]]

  - join a local IXP, which may be a physical switch or
    virtualized by a set of bilateral agreements.

Why should they join an IXP if they already have
private peering arrangements?

  - outside the region, they advertise the prefix of the
    regional authority

Mixing government with operations? If you favor doing
that then why not just give IPv6 addresses to the various
national governments and let the UN sort it out?

Personally I disagree with any scheme which calls for
national or municipal governments to assign IPv6 addresses
to end users. Dressing it up as a "regional authority"
does not make it any nicer.

Forcing people to join an unecessary IX is not the way
to solve the problem of regional aggregation of routes.
This is a purely technical problem which can be solved
by the RIR practices in allocating IPv6 addresses. If they
would allocate addresses in a geo-topological manner then
end users and ISPs would be free to aggregate routes
outside of their region without any involvement of governments
or any requirement to join consortia or IXes. It does
require the users of such geo-topological addresses to
ensure that in THEIR region, there is sufficient
interconnectivity (physical and policy) between ISPs for
the addressing to work. But that does not need to be determined
or managed centrally.

Geo-topological addressing refers to RIRs reserving large
blocks of designated addresses for areas served my large
cities (over 100,000) population. When end users are located
in fringe areas roughly equidistant between two or more such
centers, the RIR simply asks the end user (or ISP) which is
the center to which they want to connect (communicate).
This addressing scheme operates in parallel with the existing
provider-oriented IPv6 addressing scheme but uses a different
block of IPv6 addresses out of the 7/8ths that are currently
reserved. No hardware or software changes are required for this
to work, merely some geographical/economical research to determine
the relative sizes of the address pool to be reserved for each
of the world's 5000 largest cities.

Whenever I have talked about the model with an ISP, I have gotten
blasted. Basically, I have been told that

(1) any idea on operations proposed in the IETF is a bad idea because
the IETF doesn't listen to operators

This is true. Top-down does not work in Internet operations.
We need bottom-up, i.e. customer demand. The IETF needs to
view their role as enablers of customer demand. If the IETF
can create something that will work for ISP customers, then
ISPs will be happy to go along, once the customers demand
the service.

(2) the ISPs aren't going to be willing to make settlement payments
among themselves in accordance with the plan

Wait until this starts appearing as a requirement in
custome RFPs.

I'm not sure how to proceed, given the level of invective I get in
any discussion with anyone on the topic.

Perhaps the IETF needs to seek input, not just from ISPs,
but also from ISP customers, the end users of the network.

Note 2: Provider-provisioned addresses continue to make sense for
folks that don't plan to multihome.

Indeed they do. But the current IPv6 addressing model is completely
slanted towards provider-provisioned addresses for single-homed
entities. Calling a small block of these provider-provisioned
addresses PI (provider independent) does not really make the addresses
provider independent and does not help small enterprises to implement
meaningful multihoming. The IETF has imposed this provider-provisioned
model on IPv4 and is thus directly responsible for the ISP cartel
which now exists.

--Michael Dillon

Uh-oh, two postings to NANOG in as many days... hopefully, this will be
my last.

[[pushed the wrong button last time. This is the complete reply]]

Oh, the irony in that statement... this whole argument has certainly pushed
"the wrong button" for me.

> - join a local IXP, which may be a physical switch or
> virtualized by a set of bilateral agreements.

Why should they join an IXP if they already have
private peering arrangements?

> - outside the region, they advertise the prefix of the
> regional authority

Mixing government with operations? If you favor doing
that then why not just give IPv6 addresses to the various
national governments and let the UN sort it out?

Personally I disagree with any scheme which calls for
national or municipal governments to assign IPv6 addresses
to end users. Dressing it up as a "regional authority"
does not make it any nicer.

Forcing people to join an unecessary IX is not the way
to solve the problem of regional aggregation of routes.
This is a purely technical problem which can be solved
by the RIR practices in allocating IPv6 addresses. If they
would allocate addresses in a geo-topological manner then
end users and ISPs would be free to aggregate routes
outside of their region without any involvement of governments
or any requirement to join consortia or IXes. It does
require the users of such geo-topological addresses to
ensure that in THEIR region, there is sufficient
interconnectivity (physical and policy) between ISPs for
the addressing to work. But that does not need to be determined
or managed centrally.

Geo-topological addressing refers to RIRs reserving large
blocks of designated addresses for areas served my large
cities (over 100,000) population. When end users are located
in fringe areas roughly equidistant between two or more such
centers, the RIR simply asks the end user (or ISP) which is
the center to which they want to connect (communicate).
This addressing scheme operates in parallel with the existing
provider-oriented IPv6 addressing scheme but uses a different
block of IPv6 addresses out of the 7/8ths that are currently
reserved. No hardware or software changes are required for this
to work, merely some geographical/economical research to determine
the relative sizes of the address pool to be reserved for each
of the world's 5000 largest cities.

The routing system doesn't particularly care whether your "geo-topo"
addressing is imposed by governments, RIRs, or a beneveolent dictator;
in all cases, the result is Soviet-style central planning to force the
network topology to conform to your idea of what it "should" be rather
than following the economic realities of the those who would build the
network.

A "geo-topo" addressing scheme works great for address assignment *within*
a single AS and it even could have worked pretty well back in 1990, when
there was a "core" NSFNET and a bunch of regional networks. But the key
attribute of these scanerios is the existance of centralized control of
the topology. There is no such control of the topology today; those who wish
to impose such control are asking for a regulatory environment that would
radically change the nature of the Internet.

> Whenever I have talked about the model with an ISP, I have gotten
> blasted. Basically, I have been told that
>
> (1) any idea on operations proposed in the IETF is a bad idea because
> the IETF doesn't listen to operators

This is true. Top-down does not work in Internet operations.
We need bottom-up, i.e. customer demand. The IETF needs to
view their role as enablers of customer demand. If the IETF
can create something that will work for ISP customers, then
ISPs will be happy to go along, once the customers demand
the service.

Interesting to see an argument for bottom-up design in a post which
otherwise calls for top-down planning of the network architecture.

What the IETF, and more specifically the IAB, really needs to do is to
acknowledge that there is a very real problem with the ipv6 routing
architecture (which is identical to the IPv4 routing architecture), one
that cannot be fixed without making incompatible changes to protocol
implementation. Band-aids like shim6 just aren't going to cut it if the
goal is to build a highly-scalable network of autonomous routing domains
(in other worse, a really big network where end sites have very flexible
choices of providers). The first step to finding a solution is to admit
that there is a problem.

> (2) the ISPs aren't going to be willing to make settlement payments
> among themselves in accordance with the plan

Wait until this starts appearing as a requirement in
custome RFPs.

Then wait until governmental bodies step in to offer their help in the
form of regulation. The two go hand-in-hand. If you want to re-invent the
telco model of interconnection, this is a pretty big step in that direction.

...

> Note 2: Provider-provisioned addresses continue to make sense for
> folks that don't plan to multihome.

Indeed they do. But the current IPv6 addressing model is completely
slanted towards provider-provisioned addresses for single-homed
entities. Calling a small block of these provider-provisioned
addresses PI (provider independent) does not really make the addresses
provider independent and does not help small enterprises to implement
meaningful multihoming. The IETF has imposed this provider-provisioned
model on IPv4 and is thus directly responsible for the ISP cartel
which now exists.

Methinks we are re-interpreting history here. The IETF didn't create an "ISP
cartel" for IPv4. What CIDR did, and I think I can speak with some degree
of authority on this subject, was to allow routing state to scale
in a non-exponential manner by encouraging address assignment to follow
topology. Of course, the fact is that it is the providers which determine
network topology because it is they who create it (this is something of a
tautology). There are consequences of this, namely that provider changes
imply renumbering, but this really isn't some grand scheme to lock customers
in to providers; it is an unfortunate consequence of the combination of
addressing following topology and a poor, late-1960's design decision to
combine endpoint identification and routing locator into a single quantity
known as an IP address.

It is important to note that CIDR was explicitly specified as a short-term
measure to prevent the explosion of routing state from causing the Internet
to become unmanageable, which was the alternative to its adoption back in
the early-to-mid-1990s. It was also explicitly intended to be replaced by
a scalable, long-term solution which, unfortunately, has yet to be designed.
If you don't believe me, go read the documents for yourself - they say
exactly the same thing.

In the interests of demonstrating why "geo-topo" addressing can't possibly
work without radical changes to the business and regulatory models of the
Internet, consider the simple example of a provider who has connections
to two popular "geo-topo" addressing domains, say the Bay Area and the
DC area. Let's say that 10.0.0.0/8 is the "geo-topo" address block in the
Bay Area and 172.16.0.0/12 is the "geo-topo" block in the DC area. This
provider has four customers in the Bay Area:

  10.1.1.0/24
  10.10.4.0/22
  10.100.8.0/21
  10.200.0.0/16

How is the provider supposed to make use of the 10.0.0.0/8 aggregate? Does
he advertise it to other providers in the DC area or anywhere else where
he offers service (Asia, Europe, etc.)? By doing so, he is stating that he
can provide connectivity to all hosts which are numbered in that address
range. But he only provides transit service to the address ranges associated
with his customers. For him to provide connectivity to all the address range,
he must

  a) have full routing connectivity to all other providers that have
     addresses in the same range; this implies that he connects to all IXs
     within the region and maintaines a full-mesh of routing information
     (today, BGP sessions) to all of these providers

and

  b) must be willing to provide connectivity to all sites within the region
     to any place that he advertises the prefix 10.0.0.0/8 through routing
     exchanges; if he advertises this prefix to non-customers, it implies
     that he is will provide free transit to his competitors' customers
     which are numbered out of this block

Both of these requirements defy business sense, so absent the imposition of
strong regulation and negotiated settlements, they are unlikely to appeal to
any provider which wishes to offer service to and between multiple cities;
without such providers, you don't have a global Internet.

I'm not sure how I can make this much more clear. It seems appropriate to
re-state Dave's quote Yakov:

  "Addressing can follow topology or topology can follow addressing.
   Choose one."

and I'd offer a corollary:

  Transit relationships (i.e money) must follow topological relationships
  (and thus addressing); the alternative is some combination of inefficient
  or non-scalable routing, black holes, settlements, regulation, or other
  undesireable things.

If you really want to combine transport identifier and routing locator into
a single "address", you give up a lot of flexibility. For routing to scale,
addressing must follow topology, so in such a network architecture the term
"topology independent address" (aka "provider independent address") is truly
an oxymoron.

  --Vince

to two popular "geo-topo" addressing domains, say the Bay Area and the DC area. Let's say that 10.0.0.0/8 is the "geo-topo" address block in the Bay Area and 172.16.0.0/12 is the "geo-topo" block in the DC area. This provider has four customers in the Bay Area:

10.1.1.0/24
10.10.4.0/22
10.100.8.0/21
10.200.0.0/16

customers. For him to provide connectivity to all the address range, he must

a) have full routing connectivity to all other providers that have
    addresses in the same range; this implies that he connects to all IXs
    within the region and maintaines a full-mesh of routing information
    (today, BGP sessions) to all of these providers

That's not quite correct. They would have to:

    a) Have full routing connectivity to all other providers who
       provide transit in/out of the area concerned.

It does not imply:

   - having to peer with every provider in the area (some
     providers may be wholly within the area, you wouldn't need
     to peer with them, only their 'transit provider')

   - having to peer at every IX (you only need to fulfill
           condition a)

   - that peering with the other providers who provide
     inter-geo-area service, with whom you must peer as per a,
     must occur locally - it does not. (e.g. you could hand-off
     ACME providers Bay Area prefixes to ACME at DC if you
     want).

b) must be willing to provide connectivity to all sites within the region
    to any place that he advertises the prefix 10.0.0.0/8

Right.

    through routing
    exchanges; if he advertises this prefix to non-customers, it implies
    that he is will provide free transit to his competitors' customers
    which are numbered out of this block

That's not correct. Nothing says it has to be free.

If you're handing off X GiB of 10/8 Bay Area traffic to ACME provider each day, then you would (presumably) charge ACME your costs for those X GiB. ACME presumably would do likewise for traffic to 10/8 they carried that happened to be one of your customers instead.

So it's normal peering business; indeed it could be a beneficial business model to try carry as much of that 10/8 traffic as possible.

Some upsides:

- scenic routing would be far less prevalent.
- trivial provider-changing for customers / much increased
   competition (easier to attract new customers away from other
   providers).

Some big downsides:

- trivial provider-changing for customers (your competitors can
   get your customers to change over more easily than today) (I
   suspect providers would be more wary of this than they would
   welcome the /increase/ in competition :wink: ).

- every customer's (using these geo-assigned addresses) traffic is
   dependent on every transit provider. So ACMEs' customer could face
   an outage because "Barr's Internet Services" has a failure. This
   could be mitigated with good practices (ensure that those providers
   who provide transit into the area only ever originate the
   area-prefix from within the area, never outside - hard to know how
   that could be enforced)

- Co-ordination of origination the prefix: How do you ensure that
   those providers who announce the 10/8 prefix are only those
   providers who are peered with all the others? Squabbles could get
   really ugly and affect /all/ users in that block, regardless of
   whether they are customers of the squabbling providers.

"Addressing can follow topology or topology can follow addressing.
  Choose one."

and I'd offer a corollary:

Transit relationships (i.e money) must follow topological relationships
(and thus addressing); the alternative is some combination of inefficient
or non-scalable routing, black holes, settlements, regulation, or other
undesireable things.

We have settlements today already. The money factor isn't a problem really - seems to me at least the money aspect could work fine for geo-addressing, as it (should) do for transit services today. It's the other inter-provider co-ordination problems that would make it problematic.

There'd need be someone who could "enforce the law", after defining the "law" of course ;). Though, we happen to have such a body in my country funnily enough.

If you really want to combine transport identifier and routing locator into a single "address", you give up a lot of flexibility. For routing to scale, addressing must follow topology, so in such a network architecture the term "topology independent address" (aka "provider independent address") is truly an oxymoron.

Right.

The logical step then is for leaf-sites to build upon this topology-addressed network and advertise the lists of "topology identifiers" by which they are reachable to each other: shim6. Smart hosts communicating over a dumb network.

Providers aren't happy with that either though, judging by some of the grumbling wrt shim6. But that's the only solution left unless some new 'break-through' solution is discovered.

regards,

> Geo-topological addressing refers to RIRs reserving large
> blocks of designated addresses for areas served my large
> cities (over 100,000) population. When end users are located
> in fringe areas roughly equidistant between two or more such
> centers, the RIR simply asks the end user (or ISP) which is
> the center to which they want to connect (communicate).
> This addressing scheme operates in parallel with the existing
> provider-oriented IPv6 addressing scheme but uses a different
> block of IPv6 addresses out of the 7/8ths that are currently
> reserved. No hardware or software changes are required for this
> to work, merely some geographical/economical research to determine
> the relative sizes of the address pool to be reserved for each
> of the world's 5000 largest cities.

The routing system doesn't particularly care whether your "geo-topo"
addressing is imposed by governments, RIRs, or a beneveolent dictator;
in all cases, the result is Soviet-style central planning to force the
network topology to conform to your idea of what it "should" be rather
than following the economic realities of the those who would build the
network.

Which part of "CHOICE" do you fail to understand? How does
adding another choice get equated to Soviet central planning?
In my opinion, central planning is what we have now. The IETF
has imposed the provider-centric addressing model on us without
asking whether we want that or not.

Since only 1/8th of the IPv6 address space used this
provider-centric model, there is plenty of room to offer
an optional, geo-topological addressing model. Geo-top
addressing is not about imposing a topology. It simply
recognizes that the network largely follows the physical
geography of cities linked by roads, and railways. It
allows everyone to receive the benefit of the "nuclear
survivability" inherent in IP by multihoming in their
home city.

Interesting to see an argument for bottom-up design in a post which
otherwise calls for top-down planning of the network architecture.

That should have been a hint that you totally misunderstood
what I was proposing.

Methinks we are re-interpreting history here. The IETF didn't create an

"ISP

cartel" for IPv4. What CIDR did, and I think I can speak with some

degree

of authority on this subject, was to allow routing state to scale
in a non-exponential manner by encouraging address assignment to follow
topology.

This isn't about CIDR. This is about the idea that there is
a hierarchy of addressing with the ISP at the top, and the
end user as a serf of their ISP overlord. That model was
indeed imposed by the IETF, probably because at the time
they were mostly working with benevolent overlords, i.e.
universities. I want to see an alternative hierarchy so
that end users are not tied to one overlord/ISP.

In the interests of demonstrating why "geo-topo" addressing can't

possibly

work without radical changes to the business and regulatory models of

the

Internet, consider the simple example of a provider who has connections

Your example proves my point. There is no one right way
that works for all people. Let your provider continue to
use classic IPv6 addresses wherever it works better for
them. But create geo-topological addresses so that people
who want local multihoming can do so without breaking your
brittle Global Routing Table.

Both of these requirements defy business sense,

It's easy to make statements like this in theory. But when
customers come, cash in hand, with requirements like the above,
most businesses find a way to negotiate terms. Not all
business actors are greedy and stupid. And innovation is
not likely to come from the dinosaurs who dominate the
ISP space today. It will come from small upstarts and from
customers themselves demanding simple effective multihoming
without provider lock-in. In other words, small provider
independent geo-topological address blocks that are fully
routeable on the entire Internet, either as detailled
prefixes in their home city, or as a city/regional prefix
elsewhere.

If you really want to combine transport identifier and routing locator

into

a single "address", you give up a lot of flexibility. For routing to

scale,

addressing must follow topology, so in such a network architecture the

term

"topology independent address" (aka "provider independent address") is

truly

an oxymoron.

In geo-topological addressing, the address DOES FOLLOW topology.
Your problem is that you cannot see the forest for the trees.
A provider independent address does not necessarily mean topology
independent.

--Michael Dillon

I looked at some of these models back in ~2000, but the dotcom boom
ended and I didn't get laid off from my day job, so I didn't go
trolling for venture capitalists, and my employer sold off their cable
companies - since then, the market economics have changed a lot, and
routers have started to support enough memory to keep up with the
demand. The big questions about the dual-homed customer base are what
kind of connectivity they really need - Primary/Backup, or Primary /
Backup+extrabandwidth, or truly load-shared, and also what diverse
topology is available at the bandwidth they need. For a reasonably
large chunk of the ~Y2K market, the answer was "A T1 or two with
cable-modem backup", and another chunk was "T3 or bigger, able to
afford a telco or CLEC access ring", and most customers were more
concerned about backhoe fade, which takes a long time to fix, than
about ISP routing glitches, which were less common than 5 years
earlier and usually had a much shorter mean time to repair.

None of these solutions requires a World Domination Grand Master Plan
agreed to buy everybody before it can be deployed - almost anything
can start out with two carriers or a transit-buying service provider
and then grow.

One obvious business model to serve the smaller market was to start a
"Slash-19.net", which would get a routable chunk of address space, buy
transit from one or two colo providers, and use GRE/IPSEC/L2TPv3
tunnels to connect to the customer through whatever Layer 3 media is
available, e.g. cable modems, and optionally use LEC frame or similar
transport where available. In the emerging IPv6 world, a tunnel
broker service could do something like this. And for equipment-cost
reasons, you'd probably use PCs instead of routers as your tunnel
servers.

Another business model would be for a Tier 1 or Tier 2 ISP to do
something similar, using a smaller chunk of their own address space,
and using a tunnel server at one of their peering points (or colo
space served by another ISP) to handle tunnels through the secondary
carrier, such as cable modem companies. Making the addresses work
well would require them to use the dual-homing address space for those
customers' interfaces instead of whatever probably-geographical schema
they use for single-homed customers. The cable companies would be an
obvious ISP to do this - they've got control over the most common
small diverse access methods, and most of them use PPPoE to connect to
their customers so they've already got tunnelling. New wireless
access ISPs could do much of the same business.

Another model is cooperation between big carriers - if you're doing
the N**2 pairs-of-carriers model, there are ~30-35 Tier 1 carriers in
the US, so ~1000 address blocks would be enough (if it sounds like a
cabal, too bad), and probably a similar but smaller number for Europe.
Tier 2 players might need to arrange separate deals with one or more
of their upstream Tier1s, so they might double their address space
(still only adds ~10K routes), or else they might do an exchange point
approach (e.g. somebody like Linx starts Diverse-Linx.) If somebody
can get more than two Tier 1s to cooperate, they could do the
geographic approach, which can make a major dent with ~50-100 cities
in their market.