Smaller than a /24 for BGP?

Gah, brain spat out the wrong info. Bad brain.

QUIC doesn't allow the server to change its IP address; only the
client can change. So it doesn't really help the multihoming
situation.

Regards,
Bill Herrin

William Herrin wrote:

That multihomed sites are relying on the entire Internet
for computation of the best ways to reach them is not
healthy way of multihoming.

This was studied in the IRTF RRG about a decade ago. There aren't any

> other workable ways of multihoming compatible with the TCP protocol,
> not even in theory.

A decade? The problem and the solution was thoroughly studied by me
long ago and the first ID was available already in 2000.

The 5th version is here:

  draft-ohta-e2e-multihoming-05

I've found that you can access the first one by "Compare
versions" feature of the web page.

So,
another way of multihoming critically depends on replacing the layer-4
protocols with something that doesn't intermingle the IP address with
the connection identifier.

Wrong. As is stated in my ID that:

    On the other hand, with end to end multihoming, multihoming is
    supported by transport (TCP) or application layer (UDP etc.) of end
    systems and does not introduce any problem in the network and works
    as long as there is some connectivity between the end systems.

end to end multihoming may be supported at the application layer
by trying all the available addresses, which is what DNS and
SMTP are actually doing.

TCP modification is just an option useful for long lasting
TCP connections.

            Masataka Ohta

William Herrin wrote:

Use Multipath TCP
Multipath TCP (mptcp)

Doesn't work well. Has security problems (mismatch between reported IP
addresses used and actual addresses in use) and it can't reacquire the
opposing endpoint if an address is lost before a new one is
communicated.

It merely means MPTCP is wrongly architected. Dynamically changing
IP addresses is for mobility (if you don't mind location privacy),
not for multihoming.

The following way in my ID:

    The easiest way for applications know all the addresses of the
    destination is to use DNS. With DNS reverse, followed by forward,
    lookup, applications can get a list of all the addresses of the
    destination from an address of the destination.

does not have any such problem and should be as safe as
happy eyeball for two or more IPv4/IPv6 addresses.

As for (long lasting) TCP, my ID says:

    With TCP, applications must be able to pass multiple addresses to
    transport layer (e.g. BSD socket).

which implies addresses are supplied from applications by
DNS look up.

Though a client may, at the time TCP connection is established,
send a list of its IP addresses to a server, which may have some
security complications, it is simpler to let the server just
rely on DNS:

    With DNS reverse, followed by forward,
    lookup, applications can get a list of all the addresses of the
    destination from an address of the destination.

As I pointed out in the previous mail, DNS already supports
end to end multihoming at the application layer to try all
the addresses of name servers, on which other applications
can safely rely.

            Masataka Ohta

The following way in my ID:

    The easiest way for applications know all the addresses of the
    destination is to use DNS. With DNS reverse, followed by forward,
    lookup, applications can get a list of all the addresses of the
    destination from an address of the destination.

The DNS provides no such guarantee. Moreover, the DNS does guarantee
its information to be correct until the TTL expires, making it
unsuitable for communicating address information which may change
sooner.

As for (long lasting) TCP, my ID says:

    With TCP, applications must be able to pass multiple addresses to
    transport layer (e.g. BSD socket).

which implies addresses are supplied from applications by
DNS look up.

Which is a bit of hand-waving since the protocol can't do anything
with that information regardless of whether you expand the API to
provide it. Even if it could, you also miss the point that the
information -may change- during the ongoing connection so you can't
just statically retrieve it.

Regards,
Bill Herrin

William Herrin wrote:

     The easiest way for applications know all the addresses of the
     destination is to use DNS. With DNS reverse, followed by forward,
     lookup, applications can get a list of all the addresses of the
     destination from an address of the destination.

The DNS provides no such guarantee.

Guarantee for what?

Remember that we have been enjoying secure confirmation that
certain IP address belongs to certain hostname by DNS reverse
look up without any guarantee.

> Moreover, the DNS does guarantee
> its information to be correct until the TTL expires, making it
> unsuitable for communicating address information which may change
> sooner.

I'm afraid you know very little about DNS operation. See rfc1034:

    If a change can be anticipated, the TTL can be reduced prior
    to the change to minimize inconsistency during the change,
    and then increased back to its former value following the
    change.

which is the way to operate DNS when host addresses are changing,
for example, by multihoming configuration changes.

In addition, when a dual homed site with end to end multihoming
changes one of its ISP, it is a good idea to offer all the three
addresses by DNS during the change. Make before break.

     With TCP, applications must be able to pass multiple addresses to
     transport layer (e.g. BSD socket).

which implies addresses are supplied from applications by
DNS look up.

Which is a bit of hand-waving since the protocol can't do anything
with that information regardless of whether you expand the API to
provide it.

Read my draft, which explains how TCP should be modified.

            Masataka Ohta

How do you expect to improve your draft if you don't listen when
people tell you you've screwed something up?

-Bill Herrin

I wrote:

So,
another way of multihoming critically depends on replacing the layer-4
protocols with something that doesn't intermingle the IP address with
the connection identifier.

Wrong. As is stated in my ID that:

On the other hand, with end to end multihoming, multihoming is
supported by transport \(TCP\) or application layer \(UDP etc\.\) of end
systems and does not introduce any problem in the network and works
as long as there is some connectivity between the end systems\.

end to end multihoming may be supported at the application layer
by trying all the available addresses, which is what DNS and
SMTP are actually doing.

To my surprise, I've found that the current (2017) happy eyeball
already does so as is stated in rfc8305:

: Appendix A. Differences from RFC 6555
: o how to handle multiple addresses from each address family

So, we are ready for end to end multihoming for which multiple
PA addresses are enough and /24 is not necessary. Though
not all the application protocols may support it, DNS, SMTP
and HTTP(S) should be good enough as a starter.

It should be noted that happy eyeball strongly depends on
DNS, even though someone might think DNS not guaranteed.

Your web server is multihomed if you assign it PA
addresses assigned from multiple ISPs and register
the addresses to DNS. You don't have to manage BGP.

TCP modification is just an option useful for long lasting
TCP connections.

A major obstacle for it, as most of you can see, is that there
are people who can't distinguish IP address changes by mobility
and by multihoming. Such people will keep reinventing MPTCP.

            Masataka Ohta

Thanks to Lars for this interesting input and results (which I wasn’t familiar with).

I want to mention another concern with the possible use of hyper-specific IP prefixes, i.e., longer than /24, which I haven’t seen discussed in the thread (maybe I missed it?). Namely, if you allow say /28 announcements, then what would be the impact of ROV? Seems this can cause few problems:

  • If origin, say AS 10, makes a ROA for the /28, this would allow an attacker, e.g., AS 666, to do origin-hijack (announce path 666-10 to the /28), and attract traffic for the /28 from anybody accepting /28 announcements (that didn’t receive the legit announcement). Plus, of course, you get more overhead in ROA distribution and validation.

  • If origin makes a ROA only for covering prefix (say /24) then the /28 announcement would be considered invalid by ROV and (even more likely) dropped. Also you get more instances of `invalid’ announcements, making adoption of ROVs and ROAs harder.

Just a thought… Amir

  • If origin makes a ROA only for covering prefix (say /24) then the /28 announcement would be considered invalid by ROV and (even more likely) dropped. Also you get more instances of `invalid’ announcements, making adoption of ROVs and ROAs harder.

AS 10 creates an ROA for X.X.X.X/24 , maxLength 28. This means AS10 can announce any /28 in that /24, and any validator should return valid. (This ignores if the longer than /24s would be accepted by policy or not. )

Tom, thanks. I forget to mention the problem of this case ( AS 10 creates an ROA for X.X.X.X/24 , maxLength 28). Security-wise, this may actually be the worst solution:

  • An attacker can abuse this ROA to perform origin-hijack of the /28 subprefix, just like the origin hijack if AS 10 publishes ROA for the /28
  • The max-length also allows similar origin-hijack of `intermediate prefixes’ (e.g., /25), which may be allowed by ASes which will not allow /28, so there may be an even higher chance for the attacker to succeed in attracting traffic.

Of course, this is basically what is discussed in the `maxlength considered harmful’ paper and RFC (RFC 9319), nothing really new here.

Best, Amir

I suspect the initial question was related to scarcity and not traffic engineering.

-----Mike HammettIntelligent Computing SolutionsMidwest Internet ExchangeThe Brothers WISP

I’m late to the conversation, but I would have to agree with most. Below a /24 route advertisement shouldn’t happen.

I have a /24 that I would love to advertise as 2 /25’s, but the affects on everyone else is just too much. I take full routes from 4 providers, and I certainly don’t want to add over 100K. Carriers and enterprises have to pay a lot for our edge routers doing bgp and most don’t want to upgrade. We would benefit from advertising /25’s but it hurt’s more than it helps.

I’m in the alarm industry and they still haven’t started adopting IPv6. If we allow /25 subnets, some industries will never change. In a sense, we have to “force” them to change.

Mike

We would benefit from advertising /25's but it hurt's more

> than it helps.

That is, IPv6 really hurts.

I'm in the alarm industry and they still haven't started adopting
IPv6. If we allow /25 subnets, some industries will never change. In
a sense, we have to “force” them to change.

FYI, WRT routing table bloat, IPv6 having a lot longer minimum
allocation prefix than /24 (which forbid operators cut IPv6
prefixes longer than /24), that is, a lot beyond direct SRAM
look up, and, worse, needing longer TCAM word size (64 or 128
bits?) than IPv4, is, in a not so long run, a lot lot worse
than IPv4.

            Masataka Ohta