wet-behind-the-ears whippersnapper seeking advice on building a nationwide network

Mailman List Mirror (Read Only)
1

Wow this turned into a very long post....

No one replied with any useful information. I guess no one wants
competition on this list? Pretty poor tactic.

Mr hasserw@husmail.com, the net is big enough for many forms of networks and competition to exist. The fact that you write from a hushmail address is intriguing to me. That may have kept others from answering entirely.

Using ones real name/personal e-mail address builds a reputation. It also helps if you've posted other threads in the past. Looking over my post history (both replies and threads i started), one will see a progression of learning and participation. I don't recall seeing any posts from you in the past. As such, it may not have been wise to burst onto the scene and say "please to do my homework for me". Contributing to a few threads, starting a couple of your own (on a more specific subject) and saying "this is what I'm planning to do, here is what I've researched, please tell me if I'm doing it horribly wrong" is a good way to start in any community.

I had high hopes for the thread you had started, but am disappointed by the somewhat juvenile response that you sent. I believe you killed off the opportunity for some excellent discussion. So I'm starting another one, in the event people are ignoring the previous thread. Plus my title is cooler!

I did learn some things from that thread (such as nsrc.org). Thank you for posting those links and inspiring the title of this thread Bill.

In my case, I have knowledge (through consuming way too much *NOG lists and other resources). However all of my experience is in data center/enterprise LAN networking. WAN experience is limited to default BGP route delivery or statically configured links. So I have never built an ISP network before. I want to join the community, and as such am seeking advice before I blindly go off and end up being one of "those" AS. :slight_smile:

Here is what I am doing and how I plan to go about doing it. Feedback most welcome. Please be critical but polite. :slight_smile:

The previous thread mentioned business plan. That's absolutely critical. Competing on delivering the Internet is foolish at this point in the game. I'm giving net access away for free, and making money off of hyper localized advertising). I'm also using existing co location facilities and networks.

Looking over my linked in profile will demonstrate my existing expertise on the business and tech side of both online and hyper local advertising, and large scale, distributed server operations. However I'm currently not experienced on the network build out side. I figured the only way to get the level of experience I want, is to build a service provider network.

I'm in the process of building out a backbone network across the United States. Starting off small (3 points of presence: 600 West 7th st Los Angeles, 60 hudson NYC , 324 E 11th KC MO). In two cases I'm leveraging existing relationships with strong WAN engineers who will be receiving some equity in my startup, in one I'm a new customer off the street and doing everything myself other then the basic colo services (net drop, power, cooling, security, smart hands).

This backbone network will be used to terminate regional wireless networks. The wireless networks are being funded by the communities that the network serves through direct donations and by hyper localized advertising sales.

So here we go with technical nuts/bolts of the plan (as bill so eloquently put it):
"I am going to presume OSS and fully depricated kit to keep your costs down and to boost your learning skills."

Something like that.

1) Obtain ASN from ARIN (using LOA from existing upstream relationships).

2) Obtain ipv6 space from ARIN (inquired about getting space and ran into some issues. need to speak with my co founder and get details. evidently getting brand new v6 space for a brand new network is fairly difficult. for now may just announce a /48 from he.net. ) Yes I did come up with a sub netting plan for the entire United States out of a single /48. It's quite ingenious really. More details on request if anyone wants them.

3) Announce prefixes from initial point of presence locations for availability / traffic engineering reasons. Using a mix of Quagga on Linux virtual machiens, pfSense on dell servers and Cisco gear.

So more or less the steps that Bill mentioned in his response. It was somewhat tongue in cheek, but also quite accurate. I'm bootstrapping with personal funds / gear at the moment. However I believe it can be "done right". I also have a fair amount of gear I've been obtaining over the past few years with the specific intent of building an ISP. The business plan has evolved over time. It's now at a rather mature point, and it's time to get my hands dirty.

Whew. Sorry for the long post. Hopefully folks will read it. :slight_smile:

2

I wonder what would happen if a new ARIN member requested an IPv4 block of say a /16 for a new business? Or even a smaller block. I don't know what the current ARIN rules are but RIPE will currently give out six months worth of space. Now, in six months, I don't expect there to be any left anyway, so what will likely be all the v4 you ever get.

Very soon it'll be nigh on impossible for new entrants to the ISP business to get their own v4 space.

3

From: Charles N Wyble [mailto:charles@knownelement.com]
Sent: 16 September 2011 20:47
To: nanog@nanog.org
Subject: wet-behind-the-ears whippersnapper seeking advice on building
a nationwide network

Wow this turned into a very long post....

No one replied with any useful information. I guess no one wants
competition on this list? Pretty poor tactic.

2) Obtain ipv6 space from ARIN (inquired about getting space and ran
into some issues. need to speak with my co founder and get details.
evidently getting brand new v6 space for a brand new network is fairly
difficult. for now may just announce a /48 from he.net. ) Yes I did
come
up with a sub netting plan for the entire United States out of a single
/48. It's quite ingenious really. More details on request if anyone
wants them.

I wonder what would happen if a new ARIN member requested an IPv4 block of say a /16 for a new business? Or even a smaller block. I don't know what the current ARIN rules are but RIPE will currently give out six months worth of space. Now, in six months, I don't expect there to be any left anyway, so what will likely be all the v4 you ever get.

Hah. True.

I actually don't want any v4 space at all. I'm fine with using provider space for my minimal v4 needs. However I believe if I had existing v4 space, that v6 space would be easier to obtain.

Very soon it'll be nigh on impossible for new entrants to the ISP business to get their own v4 space.

Indeed.

In my case, I'm perfectly happy with v6 space. Can have very minimal v4 space for the time being. Google/netflix/facebook are reachable on v6. This is the vast majority of the net traffic. I can do large scale nat for v4 only content.

One aspect of my network, will be operational transparency. So as much as possible will be viewable in real time. This includes v4/v6 traffic statistics.

Also we do plan to expand into Europe and Asia. We are starting in the US first due to the relationships we have already established. If anyone is interested in supporting our activities in Europe, please let me know.

By our/we, I mean http://freenetworkfoundation.org/ (that's the non profit piece. the advertising part is separate but will help fund the non profit piece). Lots of dual use work being done.

4

a new entrant in the ARIN service region would have to meet the allocation criteria as
  specified in current policy. Same w/ any RIR. If the RIPE region policy is to hand
  out a six month supply, thats wonderful! (you mean if I state my six month need is a
  /28, RIPE will allocate that to me? I thunk there was a floor on min allocation size!)

  Which was why I mentioned address brokers. It will be possible to get IPv4 space after
  the RIR pools are exausted by leasing space from someone who has it. That has been the
  case since -prior- to any RIR coming to existance. Case in point, COMCAST leases
  IP space to its clients/customers.... as does ATT, VSN, TW, ad-nausa.

  Some brokers will not restrict what their clients can do w/ the space - unlike the
  brokers listed above.

/bill

5

I wonder what would happen if a new ARIN member requested an IPv4
block of say a /16 for a new business? Or even a smaller block. I
don't know what the current ARIN rules are but RIPE will currently
give out six months worth of space. Now, in six months, I don't
expect there to be any left anyway, so what will likely be all the
v4 you ever get.

Very soon it'll be nigh on impossible for new entrants to the ISP
business to get their own v4 space.

--
Leigh

As an ISP, ARIN will not give you any space if you are new. You have to already have an equivalent amount of space from another provider. I think it is really stupid, and encourages wasting IP space, but that is what the current policy is.

-Randy

6

Ah yes. I believe that is the problem we ran into. Where would I find more information about this? Is https://www.arin.net/policy/nrpm.html the best place? Am I considered an LIR if I simply run an access network and don't hand out space to 3rd parties for re assignment? (BTW should I be asking these type of questions here, or on an arin list?)

7

As an ISP, ARIN will not give you any space if you are new. You have
to already have an equivalent amount of space from another provider.

does arin *really* still have that amazing barrier to market entry?

arin claims to be a shining example of industry self-governance. to me,
this barrier to entry looks far more like industry self-protection from
new entrants.

and before anyone starts bleeding about the routing table, to me that
sounds like you fear new entrants forcing you to make a small upgrade to
your protected business as usual.

randy

8

> As an ISP, ARIN will not give you any space if you are new. You have
> to already have an equivalent amount of space from another provider.

does arin *really* still have that amazing barrier to market entry?

Yes. If you want PI space, you have to start off with PA space, utilize it, and then apply for PI space and an AS #, with contracts demonstrating your intention to multihome. Then, you have to *migrate* off the PA space and surrender it back to the 'owner'. You cannot get further PI allocations until you've done this.

9

good thing Mr Hushmail does not have to deal w/ this policy. He
  can go to Ripe and get space... :slight_smile:

/bill

10

If you go to ARIN, day one, and ask for address space, they have no way of determining if your request is justified, beyond whatever pie-in-the-sky guesses and growth projections you give them. You're asking for address space, sight unseen, in this case. That would be like someone going to a bank and asking for a loan, with no documentation, collateral, or anything else to give the bank confidence that they'll pay the loan back.

That's why the slow-start model has been used, particularly for v4 space.
If you started off by getting PA space from one or more of your upstreams, then there should be additional documentation to back up your request (SWIP entries, RWHOIS data, etc).

When I still worked in the ISP world, the startup I worked for started off with PA space, and then grew into PI space, and handed the PA space back to their upstreams as it was vacated. I had no problems getting subsequent
PI blocks because our documentation was in order.

jms

11

Alright. This seems fair.

Easy enough to get some big chunks of v6 space from up streams and then
justify the PI space.

I shall have to do that then.

12

As an ISP, ARIN will not give you any space if you are new. You have to
already have an equivalent amount of space from another provider. I
think it is really stupid, and encourages wasting IP space, but that is
what the current policy is.

If you go to ARIN, day one, and ask for address space, they have no way of
determining if your request is justified, beyond whatever pie-in-the-sky
guesses and growth projections you give them.

why is this not a problem in any other region?

randy

13

I don't have experience in working with the other RIRs, or their address assignment policies, so I can't speak to that.

jms

14

People have been bleating about routing tables sizes for years and everything has been fine. You could argue that the bleating has helped keep the size down of course, perhaps it has.

15

People have been bleating about routing tables sizes for years and
everything has been fine. You could argue that the bleating has helped
keep the size down of course, perhaps it has.

guy walks into a psychiatrist's office waving a newspaper. shrink
asks "why are you waving that newspaper?" guy responds "to keep
the elephants away." shrink says "heck, there are no elephants for
thousands of miles." guy responds "pretty effective isn't it!"

16

When I still worked in the ISP world, the startup I worked for started off with
PA space, and then grew into PI space, and handed the PA space back to
their upstreams as it was vacated. I had no problems getting subsequent PI
blocks because our documentation was in order.

The documentation isn't the pain. The renumbering is, *especially* if you're running a service provider network:

'Dear dedicated server customer, we're taking away your IPs, please don't be angry with us even though it will cost you untold hours of work to hunt down all the tiny implications of renumbering. Never mind the lost business it might cause if you miss something.'

'Dear internet access user who happens to run a bunch of IPSEC tunnels: Have fun fixing all your tunnels! Don't worry, we'll figure out an off-hours time that works for everyone, and that makes all the pain go away, right? You won't harbor any resentment, right?'

(Wow, that comes off more bitter than I expected...)

Oh well... Since new IPv4 allocations are fast approaching the same scarcity as unobtanium, I guess it's too late to worry about it now. Anyways, apparently IPv6 fixes all of this, or something.

Nathan

17

'Dear dedicated server customer, we're taking away your IPs, please don't
be angry with us even though it will cost you untold hours of work to hunt
down all the tiny implications of renumbering. Never mind the lost business
it might cause if you miss something.'

'Dear internet access user who happens to run a bunch of IPSEC tunnels:
Have fun fixing all your tunnels! Don't worry, we'll figure out an
off-hours time that works for everyone, and that makes all the pain go away,
right? You won't harbor any resentment, right?'

(Wow, that comes off more bitter than I expected...)

Oh well... Since new IPv4 allocations are fast approaching the same
scarcity as unobtanium, I guess it's too late to worry about it now.
Anyways, apparently IPv6 fixes all of this, or something.

Nathan

Yeah I'm going through this fun right now at a company I work for.
Definately not pleasant for us or our customers.

-Blake

18

The ARIN community is easily it's own worst enemy.

19

Not to mention the difficulty of actually getting a provider to let you
announce their PA IP space to other providers if you already are / want
multihoming.

I just got turned down by one of mine just yesterday for that. I'm looking
at having to keep a T1 at my office with one of my existing providers that
is going away due to footprint issues (Windstream will sell connectivity,
but requires the ip space to be localized, even if originated by customer,
so don't move or expand or anything) just to be able to announce their
number space because H.E. and my other providers refuses to do it outright.

I'm fairly fed up with the bunch at this point, and probably going to cancel
most of my current providers once I get my own space just out of spite.

Forcing PA space for multihoming before a minimum threshold is
understandable, but trying to obtain said PA space can be an exercise in
futility, which is amusing in a perverse way, because some of the providers
are the same employeers of people advocating for exactly that design in PPML
et al. Which is especially annoying coming from a provider that happily did
this for customers so its not like I don't understand the issues etc.

-Blake

20

As an ISP, ARIN will not give you any space if you are new. You
have to already have an equivalent amount of space from another
provider.

does arin *really* still have that amazing barrier to market
entry?

Yes. If you want PI space, you have to start off with PA space,
utilize it, and then apply for PI space and an AS #, with contracts
demonstrating your intention to multihome. Then, you have to
*migrate* off the PA space and surrender it back to the 'owner'. You
cannot get further PI allocations until you've done this.

The ARIN community is easily it's own worst enemy.

the arin policy weenie industry is one of the internet's worst enemies

randy