ipv4/25s and above

I am kind of curious as to the distribution of connections to smaller
companies and other entities that need more than one ipv4 address, but
don't run BGP. So, for as an ISP or infrastructure provider, what is
the typical percentage nowadays of /32s /31s /30s... /25s of stuff
that gets run "elsewhere"?

Is there any correlation between the number of IPs a customer gets and
the amount of bandwidth they buy?

Obviously "retail", home use is /32s and there's an increasing amount
of CGNAT, but I can't help but imagine there are thousands of folk
running /27s and /29s for every /24 or /22 out there.

I've been paying 15/month for a /29 for forever, but barely use it.

Dave,

I work for a smaller ISP in the Midwest with clients coast-to-coast. I deliver internet to about 2/3 rds of them over private ethernet circuits, the rest I deliver private addressed SIP service. Aside a handful of them who advertise their own /24 to me over BGP, the rest are exclusively smaller than that. /29's and /30's are the most common, with a peppering of /28's and /27's. My total IP space is about a /19.

Cheers!

For our DIA/Enterprise business, we offer customers a /30 for p2p link, and a /29 as initial standard for onward assignment within their LAN.

Interestingly, most customers will NAT on the p2p address space, and barely use the onward assignment. In such cases, link migrations cause issues, because customers bake their internal configurations against those p2p IP addresses, which are pegged to specific edge routers on our side that can't move due to our need to minimize the iBGP footprint in the network. It's not a major issue in absolute terms, but a headache nonetheless.

We can offer customers up to a /24 maximum (i.e., we will let the /29 expand into a /24), and no more. If they need more than a /24, time to speak to AFRINIC.

We don't charge for address space. Our Sales people want us to, but we don't feel it makes sense. It's not a big-enough deterrent for us to keep IPv4 stock. And when we do run out of IPv4 space, it will be like having billions of $$ on a deserted island.

Mark.

Mark Tinka wrote:

For our DIA/Enterprise business, we offer customers a /30 for p2p link, and a /29 as initial standard for onward assignment within their LAN.

You could instead use a /31. Or private/enterprise-private or unnumbered and route them the single /32 to use for their NAT on say a loopback interface. And the /29 ? I would reserve it but not assign it without a formal request.

Interestingly, most customers will NAT on the p2p address space, and barely use the onward assignment. In such cases, link migrations cause issues, because customers bake their internal configurations against those p2p IP addresses, which are pegged to specific edge routers on our side that can't move due to our need to minimize the iBGP footprint in the network. It's not a major issue in absolute terms, but a headache nonetheless.

See above.

We can offer customers up to a /24 maximum (i.e., we will let the /29 expand into a /24), and no more.

Either you have lots of fallow ground or very few customers. I dont see how this strategy would work elsewhere.

If they need more than a /24, time to speak to AFRINIC.

We don't charge for address space. Our Sales people want us to, but we don't feel it makes sense. It's not a big-enough deterrent for us to keep IPv4 stock. And when we do run out of IPv4 space, it will be like having billions of $$ on a deserted island.

Mark.

Your sales people are right. Since you can deliver quite usable service that enables them to operate just as they have before with a single /32, and with technical advantages to yourself, all the extra wasted integers should be bringing in value.

And since its quite nice to assign a loopback to every CPE anyways, two birds, one stone. Carry that in your iBGP.

Best,

Joe

You could instead use a /31.

We could, but many of our DIA customers have all manner of CPE's that may or may not support this. Having unique designs per customer does not scale well.

Or private/enterprise-private

Yeah, don't like that, although I understand why some operators use it.

or unnumbered and route them the single /32 to use for their NAT on say a loopback interface.

Same as the CPE issue I described above.

And the /29 ? I would reserve it but not assign it without a formal request.

We have some products that can be considered sub-DIA that do not come with the /29 as standard. Those tend to be the majority of the market.

Either you have lots of fallow ground or very few customers.

A bit of both.

Our main business is wholesale IP Transit. The Enterprise piece is growing, but we are not trying to save the world. It's a specific focus, and we don't do consumer.

I dont see how this strategy would work elsewhere.

To be honest, we'll keep using IPv4 for as long as we have it, and for as long as we can get it from AFRINIC. But it's not where we are betting the farm - that is for IPv6.

Your sales people are right. Since you can deliver quite usable service that enables them to operate just as they have before with a single /32, and with technical advantages to yourself, all the extra wasted integers should be bringing in value.

At the risk of using IP addresses to prop uo sales numbers, and then you run out sooner because one customer decided to pay lots of $$ for a /22, even when they don't need it. Not the kind of potato I want to taste, because we have seen that proposed far too many times to know it will become a reality when the commissions are due.

Mark.

Mark Tinka wrote:

You could instead use a /31.

We could, but many of our DIA customers have all manner of CPE's that may or may not support this. Having unique designs per customer does not scale well.

its almost 2023. /31 support is easily mandatory. You should make it mandatory.

How many customer addressing designs does that total, 2? So that would be 1 more than you have already? Dont buy it.

Its 2023, your folk should be able to handle addressing more advanced than from the 90s. And your betting the future on IPv6?

Or private/enterprise-private

Yeah, don't like that, although I understand why some operators use it.

The only issue with it is traceroute although that isnt necessarily a problem.

And the CPE sourced traffic should either be all internal or sourced from the loopback.

OTOH CoPP protection becomes a lot easier when fewer of the CP addressing is globally routed.

or unnumbered and route them the single /32 to use for their NAT on say a loopback interface.

Same as the CPE issue I described above.

Truth is the real issue isnt CPE support, its user support. Most users(even their "IT" folk) really cant wrap their brain around more than the bare basic concepts, if that much.

And you can simply say, IPv4 is limited, this is the base model addressing included with the service, if your inabilities are preventing you from using networking techniques that have been standardized and usable for decades, then feel free to pony up for either for your comfort level or for support of your inabilities.

To be honest, we'll keep using IPv4 for as long as we have it, and for as long as we can get it from AFRINIC. But it's not where we are betting the farm - that is for IPv6.

This is the crux. So long as you can obtain more, justifiable consumption is rewarded with additional resources, dis-incentivizing any addressing efficiency progress from the ancient /30 p2p + /29(or larger) routed block.

You may wish to lay groundwork to nibble backwards when runout occurs for you.

Its on Afrinic to try and preserve their pool if they wish to by doing things such as getting it across that progress in addressing efficiency is an important consideration in fulfilling requests for additional resources.

Your sales people are right. Since you can deliver quite usable service that enables them to operate just as they have before with a single /32, and with technical advantages to yourself, all the extra wasted integers should be bringing in value.

At the risk of using IP addresses to prop uo sales numbers, and then you run out sooner because one customer decided to pay lots of $$ for a /22, even when they don't need it.

If they need more, they pay more and they get more. Most of the rest of the world is operating or moving to operate in that fashion.

You would still require them to submit a formal request documenting their need. And paying more is likely to mean that its a more honest request.

But see the crux above. If your RiR isnt frowning on such behavior then its poor strategy to implement it.

Although, if you can get away with it, allocating the /30 + /29 and implementing it in an easy-to-clawback approach might be a good strategy.

Not the kind of potato I want to taste, because we have seen that proposed far too many times to know it will become a reality when the commissions are due.

Mark.

Thats a question of internal discipline without motivating external factors. Odds are those factors will arrive and I would advise preparedness for them.

Joe

Either you have lots of fallow ground or very few customers.

A bit of both.

Regarding the former, perhaps you should return some of that to AFRINIC as required in your RSA before throwing stones at other providers in the region.

Owen

Mark Tinka wrote:

You could instead use a /31.

We could, but many of our DIA customers have all manner of CPE's that may or may not support this. Having unique designs per customer does not scale well.

its almost 2023. /31 support is easily mandatory. You should make it mandatory.

Much of Africa in 2023 runs on what the US put into the resale market in the late 1990s, tragically.

Its 2023, your folk should be able to handle addressing more advanced than from the 90s. And your betting the future on IPv6?

They don’t really have a lot of alternatives.

To be honest, we'll keep using IPv4 for as long as we have it, and for as long as we can get it from AFRINIC. But it's not where we are betting the farm - that is for IPv6.

And yet you wonder why I consider AFRINIC’s artificial extension of the free pool through draconian austerity measures to be a global problem?

Its on Afrinic to try and preserve their pool if they wish to by doing things such as getting it across that progress in addressing efficiency is an important consideration in fulfilling requests for additional resources.

Instead of this, they’re mostly ignoring policy, implementing draconian restrictions on people getting space from the free pool, and buying into various forms of reality avoidance.

But see the crux above. If your RiR isnt frowning on such behavior then its poor strategy to implement it.

So far, AFRINIC has given a complete pass to Tinka’s organization and their documented excessive unused address space despite policy that prohibits them from doing so. However, AFRINIC management and board seem to have extreme difficulty with reading their governing documents in anything resembling a logical interpretation.

Owen

Dear Owen:

1) "... Africa ... They don’t really have a lot of alternatives. ...": Actually, there is, simple and in plain sight. Please have a look at the below IETF Draft:

2) If this looks a bit too technical due to the nature of such a document, there is a distilled version that provides a bird-eye's view of the solution:

3) All of the above can start from making use of the 240/4 netblock as a reusable (by region / country) unicast IP address resources that could be accomplished by as simple as commenting out one line of the existing network router program code. I will be glad to go into the specifics if you can bring their attention to this almost mystic topic.

Regards,

Abe (2022-11-19 22:50 EST)

its almost 2023. /31 support is easily mandatory. You should make it mandatory.

I don't make the gear.

How many customer addressing designs does that total, 2? So that would be 1 more than you have already? Dont buy it.

That's fine.

Its 2023, your folk should be able to handle addressing more advanced than from the 90s.

Customers will do what they do.

And your betting the future on IPv6?

Actually, yeah.

The only issue with it is traceroute although that isnt necessarily a problem.

We and some of our customers find that useful.

And the CPE sourced traffic should either be all internal or sourced from the loopback.

It's not my CPE, I don't get to make the rules.

OTOH CoPP protection becomes a lot easier when fewer of the CP addressing is globally routed.

As does not connecting any cables to the device.

Seriously, all good points. We just have more pressing issues to deal with.

Truth is the real issue isnt CPE support, its user support. Most users(even their "IT" folk) really cant wrap their brain around more than the bare basic concepts, if that much.

And you can simply say, IPv4 is limited, this is the base model addressing included with the service, if your inabilities are preventing you from using networking techniques that have been standardized and usable for decades, then feel free to pony up for either for your comfort level or for support of your inabilities.

Okay.

This is the crux. So long as you can obtain more, justifiable consumption is rewarded with additional resources, dis-incentivizing any addressing efficiency progress from the ancient /30 p2p + /29(or larger) routed block.

You may wish to lay groundwork to nibble backwards when runout occurs for you.

Its on Afrinic to try and preserve their pool if they wish to by doing things such as getting it across that progress in addressing efficiency is an important consideration in fulfilling requests for additional resources.

We aren't really going to expend too many resources on trying to delay the use of IPv6. I understand that I am speaking from a place of priviledge, having as much IPv4 as we do, but even then, we will connect as many as we connect using either "efficient" or "inefficient" techniques, and neither will prevent run-out, eventually.

We want to be able to keep servicing customers, long after we are gone. That is what we care about.

If they need more, they pay more and they get more. Most of the rest of the world is operating or moving to operate in that fashion.

It's fine for most of the world to do what it wants. I won't begrudge them that. Over here, we will do what we feel works for us.

You would still require them to submit a formal request documenting their need. And paying more is likely to mean that its a more honest request.

We do this already, only without asking them for cash.

But see the crux above. If your RiR isnt frowning on such behavior then its poor strategy to implement it.

I might have missed the part where RIR's tell me how to operate.

Although, if you can get away with it, allocating the /30 + /29 and implementing it in an easy-to-clawback approach might be a good strategy.

There is reasonable customer movement between competitors that address space comes and goes.

Thats a question of internal discipline without motivating external factors. Odds are those factors will arrive and I would advise preparedness for them.

Have you ever been in sales :-)?

Mark.

It's most amusing, to me, how Africa needs to be told how to be...

Some folk just can't help themselves.

Mark.

Dear NANOG-ers,
Hope this email finds you in good health!
Please see my comments below, inline…
Thanks.

Either you have lots of fallow ground or very few customers.

A bit of both.

Regarding the former, perhaps you should return some of that to AFRINIC as required in your RSA before throwing stones at other providers in the region.

Hi Owen,
Thanks for your email, brother!

Please, could you elaborate on the above?
Remark! you may need to start a separate thread :-/
…yes! i have already read your next email.

Shalom,
–sb.

Mikrotik still doesn't support /31 addressing. I had a customer who was
configuring their "router" the other day and we found this out. Has to move
to a /30 on the link.

He's in Africa, and I'm 100% certain Mikrotik is a go to customer router
there. There's people peering on Mikrotik even!

I do not like mikrotik, but I need to say that RouterOS does support /31.

All that you need to do, beyond set /31 at address for netmask, is check if the other address is defined at the network address.

Can you show some docs on this? I gave a subnet config to my customer of
255.255.255.254 and it wouldn't take it. I reconfigured it to a /30 (.252)
and it worked for them. This was a on a RB2011 about 3 months ago.

I search at the time showed it was a know issue. We laughed about it and
moved on.

Is /31 supported only in bleeding-edge versions such as v7 or in the
more conventional RouterOS versions ?

Rubens

But see the crux above. If your RiR isnt frowning on such behavior then its poor strategy to implement it.

I might have missed the part where RIR’s tell me how to operate.

Allow me to help:

[
preview.png

afrinic-rsa-en-201801
PDF Document · 128 KB

](https://afrinic.net/ast/pdf/services/afrinic-rsa-en-201801.pdf)

Specifically, I refer you to sections 2(b), 2(c), 2(d), 4, 6, and 7.

Owen

Dear Mark:

0) I am surprised at your apparently sarcastic opinion.

1) The EzIP proposal as referenced by my last MSG is the result of an in-depth system engineering effort. Since the resultant schemes do not rely on any protocol development, IETF does not need be involved. Especially, its first step of disabling one line of existing networking program code empowers any party to begin deploying EzIP stealthily for mitigating the IPv4 address pool depletion issues. Note that EzIP is a generic solution applicable to everyone, not limited to Africa.

2) Of course, constructive criticism is always appreciated. However, unspecific comments that confuse and distract the readers only provide dis-service to those disadvantaged population who are enduring the handicaps of being the late-comers to the Internet.

Regards,

Abe (2022-11-20 12:02 EST)

If IETF does not need to be involved, why have you submitted 12
versions of your Internet draft to IETF ?

Rubens

Hi Abraham,

I know I’m not the sharpest tool in the shed, but I’m having some
trouble understanding the deployment model for EzIP. Perhaps you
could help clear it up for me?

A non-EzIP web server is only going to see the global destination
IP address and TCP port number as the unique session identifiers
for communication, so the vast amount of additional IP space you
postulate existing behind the SPR functionally collapses down into
the 64K TCP port range available today in traditional port-based NAT
setups. As long as the top 50 websites aren’t EzIP-aware, there appears
to be no benefit for an ISP to deploy EzIP, because it doesn’t actually
gain them anything beyond what existing CG-NAT devices already provide
as far as their web-browsing customer base is concerned. Most of their
communication will still be port-based-NAT, with all the headaches and
restrictions inherent in that.

And for the top 50 websites, there’s a lot of expense and absolutely no upside
involved in deploying EzIP. They don’t care about how much IP space you have
behind your NAT device, and whether it’s uniquely identifiable within your local
realm; it’s not information they can access for tracking users, as they don’t know
what your internal mapping is, so they’ll continue to rely on browser cookies and
the like for tracking actual user sessions, regardless of the IP addressing format
being used.

So, you’ve got a model where there’s functionally almost no benefit to the end user
until the major websites start deploying EzIP-aware infrastructure, and there’s
pretty much no incentive for major websites to spend money upgrading their
infrastructure to support EzIP, because it provides no additional benefit for them.

This is pretty much exactly the same conundrum that IPv6 faced (and still faces
today). I don’t see why EzIP is any better than IPv6 or CG-NAT in terms of solving
the IPv4 address shortage problem, and it seems to involve more expense for web
providers, because it requires them to deploy additional SPR mapping routers into
their infrastructure in order to use it, with essentially no additional benefit.

Is there a piece to the picture I’m not understanding correctly?

Thanks!

Matt