Reaching out to ARIN members about their RPKI INVALID prefixes

Dear NANOG,

when I approached ARIN about how they feel about reaching out to their members about
prefixes that are unreachable in a route origin validation (ROV) environment,
John Curran (CEO ARIN) referred me to you (see email bellow - quoted with permission).

The question I asked ARIN was specifically:

Would you be open to reach out to your affected members to inform them about
their affected IP prefixes?

John Curran (CEO ARIN) wrote:

If there is evidence of community
Interest, then ARIN can conduct a community consultation to determine
our best role in this area, but you first should encourage discussion
within the network operator community at appropriate forums.

So here is my question to the network operator community in the ARIN region to
gather if there are any (dis)agreements/opinions about such a notification by ARIN:

What do you think about the idea that ARIN actively informs their affected members
about prefixes that are unreachable in an RPKI ROV environment?

The goal of that outreach/notification would be
- to reduce the number of broken legacy ROAs from the past
- reduce the negative impact on reachability of affected members.

looking forward to receiving your feedback!

kind regards,
nusenu

[1] Towards cleaning up RPKI INVALIDs | by nusenu | Medium

John Curran wrote:

Personally, since all RPKI accomplishes is providing a cryptographically signed notation of origin ASNs that hijackers should prepend to their announcements in order to create an aura of credibility, I think we should stop throwing resources down this rathole.

Owen

Owen,

Personally, since all RPKI accomplishes is providing a
cryptographically signed notation of origin ASNs that hijackers should
prepend to their announcements in order to create an aura of
credibility, I think we should stop throwing resources down this
rathole.

1/ You may be overlooking the fact that many networks peer directly with
what (for them) are the important sources/destinations. The semantics of
RPKI ROAs help block illegitimate more-specifics, and the short AS_PATH
between players prevents a hijacker from inserting themself. In other
words - the most important AS_PATHs are 1 hop. The Internet's dense
interconnectedness is saving its bacon.

2/ Another approach to achieve path validation for 1 hop is through
mechanisms such what NTT calls 'peerlock'.

3/ Lastly, some folks are innovating in this space to help automate
concepts such as peerlock through what is called ASPA. ASPA is intended
as an out-of-band, deployable alternative to BGPSec.

I think you underestimate how valuable RPKI based Origin Validation
(even just by itself) is in today's Internet landscape.

If you are aware of other efforts or more fruitful approaches please let
us know.

Kind regards,

Job

Perhaps said another way:

“How would you figure out what prefixes your bgp peer(s) should be sending you?”
(in an automatable, and verifiable manner)

-chris

(popping back to the top of the thread… sorry)

Dear NANOG,

when I approached ARIN about how they feel about reaching out to their members about
prefixes that are unreachable in a route origin validation (ROV) environment,
John Curran (CEO ARIN) referred me to you (see email bellow - quoted with permission).

Perhaps this was answered elsewhere, but:
“Why is this something ARIN (the org) should take on?”

Why can’t (or why isn’t) this something that ‘many’ monitoring/alerting companies/orgs are offering?
it’s unclear, to me, why ARIN is in any better position than any other party to perform this sort of activity?
I would expect that, at the base level, “I just got random/unexpected email from ARIN?” will get dropped in the spam-can, while: “My monitoring company to which I signed up/contracted emailed into my ticket-system for action… better go do something!” is the path to incentivize.

The question I asked ARIN was specifically:

Would you be open to reach out to your affected members to inform them about
their affected IP prefixes?

‘how?’ (email to the tech-contact? etc? did they sign up for said monitoring and point to the right destination email catcher?)

In theory, that’s what IRRs are for.

In practice, while they offer better theoretical capabilities if stronger authentication were added, the current implementation and acceptance leaves much to be desired.

However, even in theory, RPKI offers nothing of particular benefit even in its best case of widespread implementation.

Owen

Owen,

Personally, since all RPKI accomplishes is providing a
cryptographically signed notation of origin ASNs that hijackers should
prepend to their announcements in order to create an aura of
credibility, I think we should stop throwing resources down this
rathole.

1/ You may be overlooking the fact that many networks peer directly with
what (for them) are the important sources/destinations. The semantics of
RPKI ROAs help block illegitimate more-specifics, and the short AS_PATH
between players prevents a hijacker from inserting themself. In other
words - the most important AS_PATHs are 1 hop. The Internet’s dense
interconnectedness is saving its bacon.

While this may be true for a handful of well peered ASNs, it’s certainly not common
around the wider internet.

2/ Another approach to achieve path validation for 1 hop is through
mechanisms such what NTT calls ‘peerlock’.
https://www.youtube.com/watch?v=CSLpWBrHy10

Single hop is relatively easy. It’s 2+ hop where things get far more interesting.

It’s convenient to reduce the problem set to the one you can easily solve, but ignoring
the rest of the problem set smacks of hand-waving and “insert magic here”.

3/ Lastly, some folks are innovating in this space to help automate
concepts such as peerlock through what is called ASPA. ASPA is intended
as an out-of-band, deployable alternative to BGPSec.

OK, but IIRC, it’s rather orthogonal to RPKI.

https://tools.ietf.org/html/draft-azimov-sidrops-aspa-profile
draft-azimov-sidrops-aspa-verification-01

I think you underestimate how valuable RPKI based Origin Validation
(even just by itself) is in today’s Internet landscape.

I think you overestimate it.

Owen

Owen,

> Personally, since all RPKI accomplishes is providing a
> cryptographically signed notation of origin ASNs that hijackers should
> prepend to their announcements in order to create an aura of
> credibility, I think we should stop throwing resources down this
> rathole.
I think you underestimate how valuable RPKI based Origin Validation
(even just by itself) is in today's Internet landscape.

If you are aware of other efforts or more fruitful approaches please let
us know.

Perhaps said another way:

"How would you figure out what prefixes your bgp peer(s) should be sending you?"
   (in an automatable, and verifiable manner)

-chris

In theory, that’s what IRRs are for.

In practice, while they offer better theoretical capabilities if stronger authentication were added, the current implementation and acceptance leaves much to be desired.

Judging a global ecosystem just by what ARIN does is perhaps some of the issue. ARIN seems to be the outlier here as has been measured. An ARIN prefix ROA is less valuable than the other regions and this is IMO deliberate on the part of ARIN.

However, even in theory, RPKI offers nothing of particular benefit even in its best case of widespread implementation.

Disagree, but that’s ok. I know at $dayJob I’m preparing the way, but it’s much harder than it should be due to the nature of our business.

- Jared

Christopher Morrow wrote:

Perhaps this was answered elsewhere, but: "Why is this something
ARIN (the org) should take on?"

Thanks for this question, I believe this is an important one.

I reasoned about why I think RIRs are in a good position to send these emails here: [1]
but I will quote from it for convenience:

Notifying affected IP Holders

The natural next step (and that was our initial intention when
looking at INVALIDs) would be to send out emails to affected IP
holders and ask them to address the INVALIDs but although that could
be automated, we believe the impact would be better, if that email
came from some trusted entity like the RIR relevant to the affected
IP holder instead of a random entity they never had any contact
before (us).

Asking RIRs to reach out to their members also scales better since
every RIR would only have to take care of their own members.

[...]

[1] Towards cleaning up RPKI INVALIDs | by nusenu | Medium

Why can't (or why isn't) this something that 'many'
monitoring/alerting companies/orgs are offering?

There are companies offering BGP monitoring including RPKI ROAs, but
the affected IP holders are unlikely customers of those monitoring
services or generally aware of the problem.

it's unclear, to me, why ARIN is in any better position than any
other party to perform this sort of activity? I would expect that, at
the base level, "I just got random/unexpected email from ARIN?" will
get dropped in the spam-can, while: "My monitoring company to which I
signed up/contracted emailed into my ticket-system for action..
better go do something!" is the path to incentivize.

The problem is how do you make operators aware of the problem in the first place.

The question I asked ARIN was specifically:

Would you be open to reach out to your affected members to
inform them about their affected IP prefixes?

'how?' (email to the tech-contact? etc? did they sign up for said
monitoring and point to the right destination email catcher?)

Yes that is what I had in mind (notification via email to the tech contact).

kind regards,
nusenu

nusenu wrote :
What do you think about the idea that ARIN actively informs their affected members about prefixes that are unreachable in an RPKI ROV environment?

Support, although I doubt it would achieve the desired result. I support it for the following reason : when someone starts to block invalid prefixes, they would not have the excuse to say "we did not know about it".

Michel.

TSI Disclaimer: This message and any files or text attached to it are intended only for the recipients named above and contain information that may be confidential or privileged. If you are not the intended recipient, you must not forward, copy, use or otherwise disclose this communication or the information contained herein. In the event you have received this message in error, please notify the sender immediately by replying to this message, and then delete all copies of it from your system. Thank you!...

Christopher Morrow wrote:

Perhaps this was answered elsewhere, but: “Why is this something
ARIN (the org) should take on?”

Thanks for this question, I believe this is an important one.

I reasoned about why I think RIRs are in a good position to send these emails here: [1]
but I will quote from it for convenience:

Notifying affected IP Holders

The natural next step (and that was our initial intention when
looking at INVALIDs) would be to send out emails to affected IP
holders and ask them to address the INVALIDs but although that could
be automated, we believe the impact would be better, if that email
came from some trusted entity like the RIR relevant to the affected
IP holder instead of a random entity they never had any contact
before (us).

Asking RIRs to reach out to their members also scales better since
every RIR would only have to take care of their own members.
[…]

i don’t know that the contacts the RIR has for the entity is necessarily the one that controls/deals-with the RPKI data though.
I also think that generally if folk set all that up they probably know (or will soon enough) that they have a mistake.

Generally speaking, I think “folk should fix themselves, and maintain/monitor their configuration”, that ARIN (or anyone else sending ‘unsolicited email’) here is going to end badly in the worst case and ‘not have any effect’ in the majority of cases.

[1] https://medium.com/@nusenu/towards-cleaning-up-rpki-invalids-d69b03ab8a8c

Why can’t (or why isn’t) this something that ‘many’
monitoring/alerting companies/orgs are offering?

There are companies offering BGP monitoring including RPKI ROAs, but
the affected IP holders are unlikely customers of those monitoring
services or generally aware of the problem.

ok, maybe they should though? :slight_smile:

it’s unclear, to me, why ARIN is in any better position than any
other party to perform this sort of activity? I would expect that, at
the base level, “I just got random/unexpected email from ARIN?” will
get dropped in the spam-can, while: “My monitoring company to which I
signed up/contracted emailed into my ticket-system for action…
better go do something!” is the path to incentivize.

The problem is how do you make operators aware of the problem in the first place.

ideally they are aware of thier own config, have a person(s) responsible for managing that configuration and care about reachability… if they don’t have that today, they will soon enough.

The question I asked ARIN was specifically:

Would you be open to reach out to your affected members to
inform them about their affected IP prefixes?

‘how?’ (email to the tech-contact? etc? did they sign up for said
monitoring and point to the right destination email catcher?)

Yes that is what I had in mind (notification via email to the tech contact).

i’m positive that will end in sadness.

What does RPKI offer other than a way to know what to spoof in a prepend for your forged announcement?

Owen

Christopher Morrow wrote:

Perhaps this was answered elsewhere, but: “Why is this something
ARIN (the org) should take on?”

Thanks for this question, I believe this is an important one.

I reasoned about why I think RIRs are in a good position to send these emails here: [1]
but I will quote from it for convenience:

Notifying affected IP Holders

The natural next step (and that was our initial intention when
looking at INVALIDs) would be to send out emails to affected IP
holders and ask them to address the INVALIDs but although that could
be automated, we believe the impact would be better, if that email
came from some trusted entity like the RIR relevant to the affected
IP holder instead of a random entity they never had any contact
before (us).

Asking RIRs to reach out to their members also scales better since
every RIR would only have to take care of their own members.
[…]

i don’t know that the contacts the RIR has for the entity is necessarily the one that controls/deals-with the RPKI data though.

It sort of has to be, as managing your RPKI data (at least in the ARIN region) involves doing it through your ARIN On-Line account which must be associated with the ORG associated with the resources in question.

I also think that generally if folk set all that up they probably know (or will soon enough) that they have a mistake.

You overestimate some things here.

Generally speaking, I think “folk should fix themselves, and maintain/monitor their configuration”, that ARIN (or anyone else sending ‘unsolicited email’) here is going to end badly in the worst case and ‘not have any effect’ in the majority of cases.

Agreed.

[1] https://medium.com/@nusenu/towards-cleaning-up-rpki-invalids-d69b03ab8a8c

Why can’t (or why isn’t) this something that ‘many’
monitoring/alerting companies/orgs are offering?

There are companies offering BGP monitoring including RPKI ROAs, but
the affected IP holders are unlikely customers of those monitoring
services or generally aware of the problem.

ok, maybe they should though? :slight_smile:

I love a good tautology.

it’s unclear, to me, why ARIN is in any better position than any
other party to perform this sort of activity? I would expect that, at
the base level, “I just got random/unexpected email from ARIN?” will
get dropped in the spam-can, while: “My monitoring company to which I
signed up/contracted emailed into my ticket-system for action…
better go do something!” is the path to incentivize.

The problem is how do you make operators aware of the problem in the first place.

ideally they are aware of thier own config, have a person(s) responsible for managing that configuration and care about reachability… if they don’t have that today, they will soon enough.

Optimist!

Owen

Owen,

Personally, since all RPKI accomplishes is providing a
cryptographically signed notation of origin ASNs that hijackers should
prepend to their announcements in order to create an aura of
credibility, I think we should stop throwing resources down this
rathole.
I think you underestimate how valuable RPKI based Origin Validation
(even just by itself) is in today’s Internet landscape.

If you are aware of other efforts or more fruitful approaches please let
us know.

Perhaps said another way:

“How would you figure out what prefixes your bgp peer(s) should be sending you?”
(in an automatable, and verifiable manner)

-chris

In theory, that’s what IRRs are for.

it’s not worked out so far.
there’s no real authorization/authentication of note on the data set via the irr.
you have no real way of knowing that ‘as12 should be announcing 157.130.0.0/16’ … except by chasing the arin/ripe/etc records today, something that those orgs stamp and which machines could validate without people using eyeballs would sure be nice… Oh, that’s what RPKI is supposed to provide.

In practice, while they offer better theoretical capabilities if stronger authentication were added, the current implementation and acceptance leaves much to be desired.

and has for approximately 30 yrs… I don’t imagine magically it’s going to get better in the next 30 either.

However, even in theory, RPKI offers nothing of particular benefit even in its best case of widespread implementation.

“rir says owen can originate route FOO”
“ROA for 157.130.1.0/24 says OWEN can originate”

those seem like valuable pieces of information. Especially since I can know this through some machine parseable fashion.

-chris

> Perhaps said another way:
>
> "How would you figure out what prefixes your bgp peer(s) should be sending you?"
> (in an automatable, and verifiable manner)

In theory, that’s what IRRs are for.

You may be overlooking the fact that the semantics of an IRR route
object are subtly different than those of a RPKI ROA. The Prefix-to-AS
Mapping Database concept as introduced Section 2 of RFC 6811 is a huge
step forward compared to the (somewhat loosely defined) semantics of IRR
route objects.

RPKI ROAs are more than "IRR with crypto": IRR objects are basically a
list of unverifiable attestations with no proof of termination. Whereas
when that same information is published through the RPKI, we now know
two things: (1) the owner of the resource consented to the creation of
the object and (2) any BGP UPDATE that conflicts with covering RPKI ROAs
is invalid.

In other words, RPKI and the Prefix-to-AS validation procedure give us
much more definitive inputs for routing policies compared to what can be
derived from the IRR.

I simply view RPKI as a successor to the IRR system with some much
needed improvements.

In practice, while they offer better theoretical capabilities if
stronger authentication were added, the current implementation and
acceptance leaves much to be desired.

Luckily multiple projects are passing through the pipeline to reduce the
risk some IRRs represented.

https://www.ripe.net/manage-ips-and-asns/db/impact-analysis-for-nwi-5-implementation
https://teamarin.net/2018/07/12/the-path-forward/

Considering that RPKI and IRR will co-exist in one form or another for a
wihle it is encouraging to see success in patching loopholes.

However, even in theory, RPKI offers nothing of particular benefit
even in its best case of widespread implementation.

I can't really wrap my head around how you can arrive at such a
conclusion in light of all the information that has been provided to
you. So perhaps we'll have to agree to disagree.

RPKI is useful in context of a defense in depth strategy. If one
combines "peerlock" AS_PATH filters with origin validation the end
result is bullet proof. Even if NTT is the only one to deploy this
combination, the benefits are notable.

Kind regards,

Job

“rir says owen can originate route FOO”
“ROA for 157.130.1.0/24 says OWEN can originate”

Nope… ROA says (e.g.) AS1734 (or anyone willing to impersonate AS1734) can originate 192.159.10.0/24.

those seem like valuable pieces of information. Especially since I can know this through some machine parseable fashion.

I would agree if you had some way to distinguish AS1734 originating FOO from AS174 originating FOO with a prepend of AS1734.

Owen

Christopher Morrow wrote:

Perhaps this was answered elsewhere, but: “Why is this something
ARIN (the org) should take on?”

Thanks for this question, I believe this is an important one.

I reasoned about why I think RIRs are in a good position to send these emails here: [1]
but I will quote from it for convenience:

Notifying affected IP Holders

The natural next step (and that was our initial intention when
looking at INVALIDs) would be to send out emails to affected IP
holders and ask them to address the INVALIDs but although that could
be automated, we believe the impact would be better, if that email
came from some trusted entity like the RIR relevant to the affected
IP holder instead of a random entity they never had any contact
before (us).

Asking RIRs to reach out to their members also scales better since
every RIR would only have to take care of their own members.
[…]

i don’t know that the contacts the RIR has for the entity is necessarily the one that controls/deals-with the RPKI data though.

It sort of has to be, as managing your RPKI data (at least in the ARIN region) involves doing it through your ARIN On-Line account which must be associated with the ORG associated with the resources in question.

I think I can manage my employer’s RPKI data, and I’m not on the tech/admin/etc handles…
I think I can also ask the person(s) who are to do it and they may have no idea what I’m asking beyond: “click these 5 things, type that thing, thanks!”

I also think that generally if folk set all that up they probably know

(or will soon enough) that they have a mistake.

You overestimate some things here.

probably, but … eventually if your internet gets very small you’ll look at why.

Generally speaking, I think “folk should fix themselves, and maintain/monitor their configuration”, that ARIN (or anyone else sending ‘unsolicited email’) here is going to end badly in the worst case and ‘not have any effect’ in the majority of cases.

Agreed.

perhaps this is really my point: “I have no confidence that ARIN doing this (or anyone else except an upstream network/peer) is going to effect change”

[1] https://medium.com/@nusenu/towards-cleaning-up-rpki-invalids-d69b03ab8a8c

Why can’t (or why isn’t) this something that ‘many’
monitoring/alerting companies/orgs are offering?

There are companies offering BGP monitoring including RPKI ROAs, but
the affected IP holders are unlikely customers of those monitoring
services or generally aware of the problem.

ok, maybe they should though? :slight_smile:

I love a good tautology.

I’m glad you enjoyed it… you found the easter-egg!
Generally speaking though (and trying to be constructive) people who use BGP to manage reachability to their network resources really should monitor what their resources look like externally… and folk do offer services which do these things.

“As seen on TV … bgp monitoring for you!” :slight_smile:

it’s unclear, to me, why ARIN is in any better position than any
other party to perform this sort of activity? I would expect that, at
the base level, “I just got random/unexpected email from ARIN?” will
get dropped in the spam-can, while: “My monitoring company to which I
signed up/contracted emailed into my ticket-system for action…
better go do something!” is the path to incentivize.

The problem is how do you make operators aware of the problem in the first place.

ideally they are aware of thier own config, have a person(s) responsible for managing that configuration and care about reachability… if they don’t have that today, they will soon enough.

Optimist!

yea… it’s probably as optimistic as your hope that irr data will get better? :slight_smile:
I think as more things depend on reachability the state will improve… or that’s my hope, yes.

Perhaps said another way:

"How would you figure out what prefixes your bgp peer(s) should be sending you?"
  (in an automatable, and verifiable manner)

In theory, that’s what IRRs are for.

You may be overlooking the fact that the semantics of an IRR route
object are subtly different than those of a RPKI ROA. The Prefix-to-AS
Mapping Database concept as introduced Section 2 of RFC 6811 is a huge
step forward compared to the (somewhat loosely defined) semantics of IRR
route objects.

RPKI ROAs are more than "IRR with crypto": IRR objects are basically a
list of unverifiable attestations with no proof of termination. Whereas

Right… Hence my call for IRRs with stronger authentication and validation.
(i.e. IRRs run by RIRs that have the same level of maintenance authentication and
authorization requirements as current RPKI implementations).

when that same information is published through the RPKI, we now know
two things: (1) the owner of the resource consented to the creation of
the object and (2) any BGP UPDATE that conflicts with covering RPKI ROAs
is invalid.

Yes, but, what you don’t know is whether any BGP UPDATE that contains the valid
origin ASN as origin came from the origin ASN it claims.

Instead, you provided a cryptographically signed recipe for believable spoofing.

Actually, they’re quite a bit less… They provide no path data.

ROAs are useful for one hop level validation. At the second AS hop they are 100% useless.

In other words, RPKI and the Prefix-to-AS validation procedure give us
much more definitive inputs for routing policies compared to what can be
derived from the IRR.

Please explain to me how you distinguish good from bad in the following scenario…

You peer with AS6939.

You receive routes for 2001:db8:f300::/48 with the following AS Paths

  1. 6939 1239 54049 2312 1734
  2. 6939 44046 12049 174 1734

Which one is valid? Which one is not? How did the ROA tell you this?

With the IRR, I can (theoretically) compare an AS Path to the valid set of path
associations documented in RPSL. (modulo lack of participation/implementation,
which RPKI likewise suffers from). With RPKI, I can’t tell anything.

I simply view RPKI as a successor to the IRR system with some much
needed improvements.

Your view is, IMHO, very distorted.

In practice, while they offer better theoretical capabilities if
stronger authentication were added, the current implementation and
acceptance leaves much to be desired.

Luckily multiple projects are passing through the pipeline to reduce the
risk some IRRs represented.

https://www.ripe.net/manage-ips-and-asns/db/impact-analysis-for-nwi-5-implementation
https://teamarin.net/2018/07/12/the-path-forward/

Considering that RPKI and IRR will co-exist in one form or another for a
wihle it is encouraging to see success in patching loopholes.

Yep… Properly implemented and adopted, IRRs show some progress for actual path
validation abilities, including origin.

However, even in theory, RPKI offers nothing of particular benefit
even in its best case of widespread implementation.

I can't really wrap my head around how you can arrive at such a
conclusion in light of all the information that has been provided to
you. So perhaps we'll have to agree to disagree.

See above. What is it you believe RPKI offers beyond 1 hop that is anything more than
a cryptographically signed recipe for believable spoofing?

RPKI is useful in context of a defense in depth strategy. If one
combines "peerlock" AS_PATH filters with origin validation the end
result is bullet proof. Even if NTT is the only one to deploy this
combination, the benefits are notable.

Indeed, if peerlock gets wider deployment, it might prove useful. But unless I’m
really misunderstanding peerlock, I don’t see that RPKI brings much else to
the table in addition.

Owen

> "rir says owen can originate route FOO"
> "ROA for 157.130.1.0/24 says OWEN can originate"

Nope… ROA says (e.g.) AS1734 (or anyone willing to impersonate AS1734)
can originate 192.159.10.0/24.

I'd phrase slightly different (assuming there is only one ROA): the ROA
says ONLY AS1734 (or anyone willing to impersonate AS1734) can originate
192.159.10.0/24.

With IRR, the crucial addition of the word "ONLY" in the above sentence
is not possible.

> those seem like valuable pieces of information. Especially since I
> can know this through some machine parseable fashion.

I would agree if you had some way to distinguish AS1734 originating
FOO from AS174 originating FOO with a prepend of AS1734.

In the common scenario you can distinguish those with today's
technology. As mentioned before - dense peering (having the shortest
AS_PATH) or the peerlock approach for all *practical* intents solve the
issue of path validation. You can try spoofing AS 7018 - you'll notice
that your announcements won't make it into NTT. Having just that
validated path (which is only one hop) is a very strong defense
mechanism.

Let's take another example: Google offers access to one of the world's
largest DNS resolver services, Cloudflare hosts lots of authoritative
DNS. According to PeeringDB they have quite some locations in common
with each other so let's assume they directly peer with each other. If
both sides create ROAs for their prefixes, and ROAs for the prefixes
that host the auth side, and both sides perform RPKI Origin Validation;
an attacker cannot inject itself between the two.

One can argue "this only helped google and cloudflare" - but on the
other hand anno 2018 the Internet experience for the average end user is
composed from services hosted by a relatively small number of providers.
Preventing disruptions of communication between just those few players
has significant implications for all of us.

Kind regards,

Job

Christopher Morrow:

Yes that is what I had in mind (notification via email to the tech
contact).

i'm positive that will end in sadness.

we can also send snail mail :slight_smile:
after all ~80 or so entities is a manageable amount of organizations to
notify in the ARIN region.