two interfaces one subnet

Mailman List Mirror (Read Only)
1

Hi,

This is a pretty moronic question, but I've been searching RFC's on-and-off for a couple of weeks and can't find an answer. So I'm hoping someone here will know it offhand.

I've been looking through RFC's trying to find a clear statement that having two interfaces in the same subnet does not work, but can't find it that statement anywhere.

The OS in this case is Linux. I know it can be done with clever routing and prioritization and such, but this has to do with vanilla config, just setting up two interfaces in one network.

I would be grateful for a pointer to such an RFC statement, assuming it exists.

Thanks!

Chris

2

Why would an RFC prohibit this?

Most _implementations_ do, but as far as network "rules" in general it is a valid configuration.

3

I don't know if it still works, but it did in Linux little over 10 years back. Proxy-arp:ed all the IPs in the /27 in the /24 and everything was fine (legacy reasons plus radiolink which I didn't want to run a lot of broadcasts over). There are "legitimate" cases where you might want to do this.

4

I would be grateful for a pointer to such an RFC statement, assuming it exists.

Why would an RFC prohibit this?

Most _implementations_ do, but as far as network "rules" in general it is a valid configuration.

That was essentially my conclusion as well: logically it can't work, but I wasn't certain where it might be forbidden.

Thusly did I come to NANOG with the question, thinking smarter people than I might know. If it's completely down to implementation, or really to the interaction between TCP and underlying IP, then so be it. I was hoping that I might just not have thought of the right place to look.

I've been looking through RFC's trying to find a clear statement that having two interfaces in the same subnet does not work, but can't find it that statement anywhere.

I don't know if it still works, but it did in Linux little over 10 years back. Proxy-arp:ed all the IPs in the /27 in the /24 and everything was fine (legacy reasons plus radiolink which I didn't want to run a lot of broadcasts over). There are "legitimate" cases where you might want to do this.

Yes, I've gotten it to work as well as little as 10 days ago, but it's not something that $random_customer should be doing as a matter of practice.

Thus, again, my hope that I just wasn't thinking of the right place to look to find an IETF recommendation against doing so.

Thanks for the input!

Chris

5

NetApp filers consider this to be a legitimate configuration, even a
"supported and recommended one". If I understand the documentation
correctly, NetApp will (somehow) remember the physical interface a
request arrived on, and make sure to its send replies out same. It
sounds like a slight breakage of the expected behavior in order to
gain load-sharing for their multiple NICs attached to the same subnet.
IIRC, you can turn the feature off if it makes an issue.

--D

6

What does two interfaces in one subnet mean?

Two NICs? Or virtual interfaces?

Mikael Abrahamsson wrote:

7

Two NICs, as in physical interfaces.

8

Unless you configure Layer 2 for two interfaces, it's not going to work.
It is invalid from networking principle.
If you have to send the traffic for host in same subnet you configured,
which interface it should send out ?
Basically it may create broadcast storm loop by putting two ip addresses
in same subnet in different interface.
It may be allowed from host-level, but from router equipment, I don't
think it was allowed at all.

Alex

Chris Meidinger wrote:

9

Chris,

I work with iHDTV <http://ihdtv.org>, a project that sends uncompressed high
definition television (1.5 Gbps) as UDP over two 1 Gbps interfaces. If both
interfaces are on the same subnet, the OS sees the same router (gateway)
address on both interfaces, and the results are sub-optimal ... around 50%
packet loss.

Dave

10

Alex, I _personally_ know that it's a problem. I was hoping for an RFC-reference, or similar standards document, to show to customers to convince them to stop trying to hack things to make it work.

Chris

11

packet loss is probably due to the network switch having to re-learn
the location of the MAC address constantly as it sees packets on two
or more ports with the same MAC address (think STP loops).

If your network stack and network device (switch) supports LACP, then
you can have multiple connections between a host and a network device.
That is a very easy way to increase capacity and add redundancy.

That is how all of our VMWare ESX 3.5i servers are connected.

Hector

12

Chris Meidinger wrote:

Hi,

This is a pretty moronic question, but I've been searching RFC's on-and-off for a couple of weeks and can't find an answer. So I'm hoping someone here will know it offhand.
I've been looking through RFC's trying to find a clear statement that having two interfaces in the same subnet does not work, but can't find it that statement anywhere.
The OS in this case is Linux. I know it can be done with clever routing and prioritization and such, but this has to do with vanilla config, just setting up two interfaces in one network.
I would be grateful for a pointer to such an RFC statement, assuming it exists.

If your goal is to achieve redundancy or to increase bandwidth, you can bond the interfaces together - assuming that you have a switch / switch stack that supports 802.3ad.

Then you could assign multiple IPs to the bonded interface without any layer 3 messyness.

- Dan

13

I should have been clearer. The case in point is having two physical interfaces, each with a unique IP, in the same subnet.

For example, eth0 is 10.0.0.1/24 and eth1 is 10.0.0.2/24, nothing like bonding going on. The customers usually have the idea of running one interface for administration and another for production (which is a _good_ idea) but they want to do it in the same subnet (not such a good idea...)

Chris

14

:Hi,
:
:This is a pretty moronic question, but I've been searching RFC's on-
:and-off for a couple of weeks and can't find an answer. So I'm hoping
:someone here will know it offhand.
:
:I've been looking through RFC's trying to find a clear statement that
:having two interfaces in the same subnet does not work, but can't find
:it that statement anywhere.
:
:The OS in this case is Linux. I know it can be done with clever
:routing and prioritization and such, but this has to do with vanilla
:config, just setting up two interfaces in one network.
:
:I would be grateful for a pointer to such an RFC statement, assuming
:it exists.

RFC1122, Section 3.3.4.1 explicitly says this IS a legal config
from an IP perspective:

      3.3.4 Local Multihoming

         3.3.4.1 Introduction

            A multihomed host has multiple IP addresses, which we may
            think of as "logical interfaces". These logical interfaces
            may be associated with one or more physical interfaces, and
            these physical interfaces may be connected to the same or
            different networks.

There are other considerations here -- OS, link-layer, etc.
Obviously, you want to do such things with care. But simply
from a "standards" perspective, it's ok. There are a lot of
hosts that historically didn't have enough RFC1122 compliance
to make such configurations problematic (e.g. section 3.3.1.2
and multiple default route support vs. old BSD IP stacks) but
that doesn't invalidate the standards.

15

My understand of the scenario is: Two physical interfaces, each with
a unique IP address, in the same Ethernet broadcast domain, on the
same IP (sub)network.

  If that's the case, the MAC address won't change. The cards stay
put. So a layer two switch will be none the wiser.

  The reason this doesn't work (for most implementations) is that most
IP routers look only at the destination IP address, and keep no state.
(Here, I'm using "router" to include the routing engine built-in to
any full IP implementation, not just dedicated equipment from Cisco,
et. al.)

  So we have a host with IP addresses A and B on the same subnet. A
packet comes in from some other host X. The application software does
whatever it does, and sends a response. The router looks at the
destination IP address X, and sees that it has two routes, A and B.

  Depending on implementation, the router may send everything out the
first interface it finds in the routing table (e.g., use A and ignore
B), or round-robin between the two, or who-knows-what. Either way, if
the packet *from* X was addressed *to* B but the response comes back
from *A*, then host X is going to drop the packet as
invalid/irrelevant/etc.

  With Linux, at least, you *can* use the routing policy database to
configure the kernel router to pay attention to more than just the
destination IP address. For example, you can have it look at the
source IP address (A or B), and route out the appropriate interface.
However, IIRC, this only works if the application software binds to
individual network interfaces. If the app software just listens for
anything (0.0.0.0), then the kernel gets to pick the source IP address
for any response.

  I can post examples with gory details from our firewall, if anyone needs them.

-- Ben

16

Hi Chris,

I remember this. I remember it in an early IP RFC, but couldn't find it in 10 minutes of searching. It had to do with intefaces cannot have overlapping address space. One of the IETF greybeards ought to know.

It's been a while since I was writing code with marked up rfc's in front of me...

Chris

17

Unless you configure Layer 2 for two interfaces, it's not going to work.

It can work. Of course it _may_ not, depending upon your implementation, but then some implementations can't get a single interface to work properly per subnet.

It is invalid from networking principle.

You are confused, there is nothing invalid about the configuration.

If you have to send the traffic for host in same subnet you configured,
which interface it should send out ?

Pick an interface and send the packet. It's not rocket science. I can come up with half a dozen algorithms off the top of my head while typing the last sentence.

Basically it may create broadcast storm loop by putting two ip addresses
in same subnet in different interface.

That is an interesting statement. Could you explain how this can happen without crafting an idiotic implementation spec (e.g. every packet goes out both interfaces)?

It may be allowed from host-level, but from router equipment, I don't
think it was allowed at all.

Ever used HSRP / VRRP? Two interfaces in the same subnet. Works fine. In fact, most people think it works _better_ than one interface in the same subnet.

18

I just posted on this, but I didn't really address your original
question, so: I'm not aware of anything in the RFCs or other standards
which prohibits this. But then, I haven't gone looking, because...

  It *can* be made to work in practice, for certain scenarios. For
example, if you're talking a web server, and you bind the "production"
site to 10.0.0.2 and the "administration" site to 10.0.0.1, and
configure policy routing (you said Linux, right?) to route
appropriately, it should work. It works because Apache can bind sites
to individual interfaces.

-- Ben

19

If it were me and had the requirement of having both NICs in the same L2
segment, but unique IP addresses, I'd assign a secondary IP address to
the Layer3 SVI on the upstream device, and give the 2nd NIC on the
server an IP on that secondary IP block.

Ken Matlock
Network Analyst
Exempla Healthcare
(303) 467-4671
matlockk@exempla.org

20

How did you read what I wrote and say what you said? I've read it several times and I can't get from point A to point B. Did you do a major typo or something?

I said it is valid. After saying you came to the same conclusion, you then said "logically it can't work". Your statement not only contradicts what I said, but contradicts the fact is has and does work.

Let me be clear: IT IS NOT FORBIDDEN. IT WORKS FINE.