Will wholesale-only muni actually bring the boys to your yard?

This, Jean-Francois, is the assertion I hear relatively frequently.

It rings true to me, in general, and I would go that way... but there is
a sting in that tail: Can I reasonably expect that Road Runner will in fact
be technically equipped and inclined to meet me to get my residents as
subscribers? Especially if they're already built HFC in much to all of
my municipality?

Cheers,
-- jra

If there is competition offering next-gen type services, that they can't reasonably or more easily offer via their existing HFC plant, then I would expect they'd start using the muni network.

I think the biggest factor though, would be cost. If using the muni network is cheaper than their own HFC plant, they may actually phase out their HFC network over time.

--John

It doesn't actually matter. You don't necessarily need to be the only wholesale
offering, you just need to be open to all service providers. This means that
if Road Runner wants to pay for their own infrastructure instead of using yours,
then that will increase their costs and likely make it harder for them to compete
with ISPs (and other services) that choose to use your infrastructure.

Owen

From: "Jean-Francois Mezei" <jfmezei_nanog@vaxination.ca>

It is in fact important for a government (municipal, state/privince or
federal) to stay at a last mile layer 2 service with no retail
offering. Wholesale only.

Not only is the last mile competitively neutral because it is not
involved in retail, but it them invites competition by allowing many
service providers to provide retail services over the last mile
network.

As long as they support open peering they can probably operate at
layer 3 without harm. Tough to pitch a muni on spending tax revenue
for something that's not a complete product usable directly by the
taxpayers.

It rings true to me, in general, and I would go that way... but there is
a sting in that tail: Can I reasonably expect that Road Runner will in fact
be technically equipped and inclined to meet me to get my residents as
subscribers? Especially if they're already built HFC in much to all of
my municipality?

Not Road Runner, no. What you've done, if you've done it right, is
returned being an ISP to an ease-of-entry business like it was back in
the dialup days. That's where *small* business plays, offering
customized services where small amounts of high-margin money can be
had meeting needs that a high-volume commodity player can't handle.

Regards,
Bill Herrin

I am the administrator of a Municipally held ISP that has been providing services to our constituents for 15 years in a competitive environment with Charter. We aren't here to eliminate them, only to offer an alternative. When the Internet craze began back in the late 1990's they made it clear that they would never upgrade the plant to support Internet data in a town this size, until we started the discussion of Bonds. We provide a service that is reasonably priced with local support that is exceptional. We don't play big brother. Both myself and my Director honor peoples privacy. No information without a properly executed search warrant. Having said all that. We are pursuing the feasibility of the model you are discussing. My director believes that we would better serve our community by being the layer 1 or 2 provider rather than the service provider. While I agree in principle. The reality is, from my perspective is that the entities providing the services will fall back to the original position that prompted us to build in the first place. Provide a minimal service for the maximum price. There is currently no other provider in position in our area to provide a competitive service to Charter. Loosely translated, our constituents would lose. IMHO.

There isn't any reason that you couldn't offer ALL of those services. Spin off the layer 1 & 2 services as a separate entity as far as finance & legal is concerned, then treat the muni ISP as just another customer of that entity, with the same pricing and service that's offered to everyone else. If there is enough competition with the layer 1 & 2 services, the muni ISP may or may not have that many customers, but it'll still be there as an "ISP of last resort", to borrow a concept from the financial system, ensuring competitive and fair pricing is available.

- Pete

Having worked with lots of other municipalities who do the same thing, I
think you're 100% right. The L1/L2 solutions are nice to think of, but I
don't think in the end it actually works in the real world.

The only time a municipality operating in the L1 space has worked well from
my experience is when they were selling fiber to other carriers. Which
generally meant the only things that the carriers and the municipality
cared about and wanted fiber built to was large enterprises, telco spaces,
or as middle-mile pieces of another network. I don't think the residential
model could actually be financially feasible for any municipality.

That is actually one of the big picture scenarios we are reviewing, with the ISP component being the last to go if there is a fair and competitive market the arises for our constituents. We won't allow the return of the old monopoly play that existed back then. This is too vital for the growth of our business community. We also view it as a quality of life issue for our citizens.

The Australian NBN plan evolved because, when the Australian government put
out the original RFP, the incumbent telcos wanted anti-competitive
commitments in exchange for their build-out efforts (sound familiar here in
the USA?). The Australian government deemed the original telco RFP replies
as "non-responsive", and withdrew the RFP, deciding that only the
Australian government could build out a national network with broadband
local loops to every residence and business. The Australian wholesale model
opens the NBN to competitive market forces, as the wholesaled bandwidth
costs are the same for all ISPs. So the plan is to make the ISPs compete on
customer service features, let the marketplace decide as it were, as they
would all have the same wholesale bandwidth charges.

For those that argue that a national government plan would never work in
the USA, the interstate highway system, and the modern commercial Internet
itself refute that argument. The modern Internet was created by the Federal
High Speed Computing and Communications Act of 1991, and the original
build-out was directed by the National Science Foundation under the
management of the White House Office of Technology. Once the commercial
Internet was established, it was turned over to the telcos in 1993.
The Australian NBN also has plans to possibly turn the network over to
private hands once the build-out is established.

And the muni build-out model, where a hodge podge of local networks are
somehow coordinated such that all residences and businesses are connected,
nationwide, at the same price and speed, just will not work. Building from
the bottom up is not how today's commercial Internet backbone was created.

David

Perhaps, but well worth the effort. There are a wide variety of reasons
to want more than one L3 provider to be readily available and avoid
limiting consumers to a single choice of ISP policies, capabilities, etc.

Also, an L1/L2 fiber plant may be usable for other services beyond just
packets.

Owen

As long as they support open peering they can probably operate at
layer 3 without harm. Tough to pitch a muni on spending tax revenue
for something that's not a complete product usable directly by the
taxpayers.

Perhaps, but well worth the effort. There are a wide variety of reasons
to want more than one L3 provider to be readily available and avoid
limiting consumers to a single choice of ISP policies, capabilities, etc.

If the municipal provider offers open, settlement-free peering at the
head end then the customer *does* have a choice of L3 provider. Tunnel
service over IP has only minor differences from an L2 service in such
a scenario. Only one difference truthfully: MTU.

Also, an L1/L2 fiber plant may be usable for other services beyond just
packets.

True enough but rapidly dropping in importance. The 20th century held
POTS service with a rare need for a dry copper pair. The 21st holds IP
packets with a rare need for dark fiber.

Besides, I don't propose that a municipality implement fiber but
refuse to unbundle it at any reasonable price. That would be Really
Bad.

Regards,
Bill Herrin

As long as they support open peering they can probably operate at
layer 3 without harm. Tough to pitch a muni on spending tax revenue
for something that's not a complete product usable directly by the
taxpayers.

Perhaps, but well worth the effort. There are a wide variety of reasons
to want more than one L3 provider to be readily available and avoid
limiting consumers to a single choice of ISP policies, capabilities, etc.

If the municipal provider offers open, settlement-free peering at the
head end then the customer *does* have a choice of L3 provider. Tunnel
service over IP has only minor differences from an L2 service in such
a scenario. Only one difference truthfully: MTU.

No, they have the municipal provider as a single monopoly L3 provider.

While it's true that you can create an L2 service on top of existing
L3 service using a tunnel, that doesn't exempt you from the limitations
and policies of the underlying L3 provider.

Also, an L1/L2 fiber plant may be usable for other services beyond just
packets.

True enough but rapidly dropping in importance. The 20th century held
POTS service with a rare need for a dry copper pair. The 21st holds IP
packets with a rare need for dark fiber.

We all know these trends run in cycles. Today, the importance of other
services is dropping. However, if we should have learned anything from
the past development in this industry, it's that planning a 30 year plant
around today's realities is virtually guaranteed to be wrong in about 10
years or less.

Besides, I don't propose that a municipality implement fiber but
refuse to unbundle it at any reasonable price. That would be Really
Bad.

However, if they're competing for L3 business, it creates a conflict
of interest. If they're a monopoly L3 provider, it creates different
problems.

Owen

[ One of a batch of replies to today's traffic; I was busy yanking a
750GB drive out of the grave all day. --jra ]

From: "Owen DeLong" <owen@delong.com>

[ me: ]

> It rings true to me, in general, and I would go that way... but
> there is
> a sting in that tail: Can I reasonably expect that Road Runner will
> in fact
> be technically equipped and inclined to meet me to get my residents
> as
> subscribers? Especially if they're already built HFC in much to all
> of
> my municipality?

It doesn't actually matter. You don't necessarily need to be the only
wholesale
offering, you just need to be open to all service providers. This
means that
if Road Runner wants to pay for their own infrastructure instead of
using yours,
then that will increase their costs and likely make it harder for them
to compete
with ISPs (and other services) that choose to use your infrastructure.

It does actually matter, Owen, for the specific build I'm looking at,
since *Road Runner already has the city built*; they can do GHz CATV
with all the toys, and at least 25/5 cablemodem, if not 50/15.

That's pretty competitive, and already includes triple play.

What sort of money a build needs to make is of course largely a question
of how good a sales job you did to your city commission, but I shouldn't
think a small, largely residential, community is gonna make it on *just*
businesses and geeks.

Cheers,
-- jra

From: "William Herrin" <bill@herrin.us>

As long as they support open peering they can probably operate at
layer 3 without harm. Tough to pitch a muni on spending tax revenue
for something that's not a complete product usable directly by the
taxpayers.

That's one problem, yes. My solution there is to tap the Third Local
ISP, who are already the competitive provider over Bright House, and
have them at the letter of intent stage, at least informally, before I
go to the council with a proposal.

> It rings true to me, in general, and I would go that way... but
> there is
> a sting in that tail: Can I reasonably expect that Road Runner will
> in fact
> be technically equipped and inclined to meet me to get my residents
> as
> subscribers? Especially if they're already built HFC in much to all
> of my municipality?

Not Road Runner, no. What you've done, if you've done it right, is
returned being an ISP to an ease-of-entry business like it was back in
the dialup days. That's where *small* business plays, offering
customized services where small amounts of high-margin money can be
had meeting needs that a high-volume commodity player can't handle.

And this argument, Bill, plays right into my hands. Thanks.

Cheers,
-- jra

Yup; that point hadn't escaped me. That was why I was planning on making
sure that first wholesale provider is already signed up when I go to
make the sales pitch to the council.

Cheers,
-- jra

There you go, Art: as the muni, the goal is the *public good*; it can
break even, or even lose money, if it increases the tax revenue base by
making the city more attractive for both residents and businesses.

Cheers,
-- jra

In a message written on Wed, Jan 30, 2013 at 09:37:24AM -0500, Art Plato wrote:

While I agree in principle. The reality is, from my perspective is that the entities providing the services will fall back to the original position that prompted us to build in the first place. Provide a minimal service for the maximum price. There is currently no other provider in position in our area to provide a competitive service to Charter. Loosely translated, our constituents would lose. IMHO.

I'm not sure what your particular situation is, but I urge you to
look at the hurdles faced by a small business trying to use your
infrastructure.

Back in the day there were a ton of dial up ISP's. Why? Well, all
they had to do was order an IP circuit from someone, a bank of phone
lines from another person, and a few thousand dollars worth of
equipment and boom, instant ISP.

Exclude your muni-fiber for the moment, and consider someone who wants
to compete with Charter. They have to get permits to dig up streets,
place their own cable to each house, be registered with the state PUC as
a result, respond to cable locates, obtain land and build pedistals with
power and network to them, etc; all before they can think about turning
up a customer. The barrier to entry is way too high.

Muni-fiber shold be able to move things much closer to the glory
days of dial up, rather than the high barrier to entry the incumbant
telcos and cablecos enjoy. Look at your deployment, what are the
up front costs to use it? Do you require people to have a minimum
number of customers, or a high level of equipment just to connect?
What's the level of licensing and taxation imposed by your state?

Many of the muni-fiber plants I've read about aren't much better. They
are often GPON solutions, and require a minimum number of customers to
turn up, purchase of a particular amount of colo space to connect, and
so on. Just to turn up the first customer is often in the tens of
hundreds of thousands of dollars; and if that is the case the incombants
will in.

Some of this is beyond the reach of muni-fiber. State PUC's need to
have updated rules to encourage these small players in many cases. I
think the CALEA requirements need a bit of an overhaul. If the
providers want to offer voice or video services there's an entirely
different level of red tape. All of these things need to be moderized
with muni-fiber deployments, and sadly in many cases the incumbants are
using their muscle to make these ancillary problems worse just to keep
out new entrants...

[ One of a batch of replies to today's traffic; I was busy yanking a
750GB drive out of the grave all day. --jra ]

From: "Owen DeLong" <owen@delong.com>

[ me: ]

It rings true to me, in general, and I would go that way... but
there is
a sting in that tail: Can I reasonably expect that Road Runner will
in fact
be technically equipped and inclined to meet me to get my residents
as
subscribers? Especially if they're already built HFC in much to all
of
my municipality?

It doesn't actually matter. You don't necessarily need to be the only
wholesale
offering, you just need to be open to all service providers. This
means that
if Road Runner wants to pay for their own infrastructure instead of
using yours,
then that will increase their costs and likely make it harder for them
to compete
with ISPs (and other services) that choose to use your infrastructure.

It does actually matter, Owen, for the specific build I'm looking at,
since *Road Runner already has the city built*; they can do GHz CATV
with all the toys, and at least 25/5 cable modem, if not 50/15.

You don't think some small scrappy provider using muni fiber with good
customer service couldn't come in and start collecting customers from
Road Runner? I bet they could.

Having muni fiber with an open access policy makes it pretty easy to
stand up a local ISP without a lot of up-front investment. Having a
single incumbent doesn't strike me as being particularly dangerous to
the practicality of muni fiber.

That's pretty competitive, and already includes triple play.

Competitive by today's pathetic american standards, sure. You wouldn't
be able to find a single taker in SE, KR, or many other parts of the
developed world.

What sort of money a build needs to make is of course largely a question
of how good a sales job you did to your city commission, but I shouldn't
think a small, largely residential, community is gonna make it on *just*
businesses and geeks.

No, but most such communities, given a choice, the incumbent wouldn't
have too much difficulty losing customers to a competitor.

Owen