I have 2 core routers (CR) and 3 access routers (AR)
currently connected point-to-point where each AR connects to
each CR for a total of 6 ckts. Now someone has decided to
connect them with Gig-E. I was wondering about the benefits
or disadvantages of keeping the ckts each in their own
individual LANs or tying them all into one VLAN for a
"Transit LAN" as those folks that decided on going to Gig-E
aren't doing any logical network architecting (is that a
real word?).
Anyone got any suggestions, comments or helpful hints?
Personally, I like the to KISS, so one big 'transit LAN'.
An argument could be made for individual VLANs to keep things like b-cast storms isolated. But I think the additional complexity will cause more problems than it will solve.
Or maybe I'm just too dumb to keep up with the additional complexity.
An argument could be made for individual VLANs to keep things
like b- cast storms isolated. But I think the additional
complexity will cause more problems than it will solve.
Vlans will not stop all typres of broadcast storm.
In my experience either solution has tradeoffs and the correct
one depends greatly on your traffic patterns. Having said that,
what I find causes most of the problems in either solution is
when the Layer 3 topology starts to diverge from the Layer 2
topology.
Thus spake "Patrick W. Gilmore" <patrick@ianai.net>
I have 2 core routers (CR) and 3 access routers (AR)
currently connected point-to-point where each AR connects to
each CR for a total of 6 ckts. Now someone has decided to
connect them with Gig-E. I was wondering about the benefits
or disadvantages of keeping the ckts each in their own
individual LANs or tying them all into one VLAN for a
"Transit LAN" as those folks that decided on going to Gig-E
aren't doing any logical network architecting (is that a
real word?).
Personally, I like the to KISS, so one big 'transit LAN'.
ITYM two big transit LANs -- one must be prepared for a switch to fail.
An argument could be made for individual VLANs to keep things like b- cast storms isolated. But I think the additional complexity will cause more problems than it will solve.
If you have broadcast storms on a subnet with five routers and nothing else on it, you've got bigger problems than config complexity.
Or maybe I'm just too dumb to keep up with the additional complexity.
One must keep in mind that human error is the dominant cause of outages, and since there's not likely to be backhoes running around in a data center, IMHO the goal should be to remove as many ways as possible that your coworkers can muck things up.
I'd go with two plain GigE switches, as dumb as I could find them, barely configured or possibly not even managed at all, and one /28 (and one /64) on each to allow for adding more ARs later.
There are a few advantages to going with PTP VLANs, such as eliminating DR/BDR elections needed on shared ones, but you'd need 10 of them to get a full mesh, and 15 if you add one more router. That's just too much complexity for virtually no gain, and as Owen notes, it is generally bad for your logical topology to not match the physical one.
S
Stephen Sprunk "Stupid people surround themselves with smart
CCIE #3723 people. Smart people surround themselves with
K5SSS smart people who disagree with them." --Aaron Sorkin
So, perhaps I missed the earlier explanation, but why use switched
segments at all? if the purpose is to connect routers to routers putting
something that WILL FAIL in the middle is only going to increase your
labor costs later
So, for router-router links, GE doesn't have to mean switched...
Thus spake "Patrick W. Gilmore" <patrick@ianai.net>
<snip>
There are a few advantages to going with PTP VLANs, such as eliminating
DR/BDR elections needed on shared ones, but you'd need 10 of them to get a
full mesh, and 15 if you add one more router. That's just too much
complexity for virtually no gain, and as Owen notes, it is generally bad for
your logical topology to not match the physical one.
Even if you have a small number of routers on a segment, you can set the
ethernet interface type to point-to-multipoint, at least on Ciscos.
Automatic nighbour discovery via multicast hellos still happens, the
difference is that the routers establish direct adjacencies between each
other, rather than with the DR. While this costs additional RAM, and CPU
during the SPF calc, the benefit of avoiding DR/BDR elections, and the
'DR/BDR' approximately 40 second listening phase when a third and
subsequent routers come online may be well worth those costs.
I've also found you can set the OSPF interface type on ethernets to
point-to-point. From memory, it results in a slightly smaller Router LSA
than point-to-multipoint. That probably doesn't matter much. I haven't
tested it, however setting the type to point-to-point might prevent a
third OSPF router being accidentally added to the segment and then
establishing an unwanted adjacency, which might provide a robustness
against human error advantage.
Very true. In fact, GE is even easier because part of the GE standard
for UTP requires it to be Auto-MDI-Sensing (MDI vs MDI-X is handled
automatically in ALL compliant GE/TP interfaces). Thus, you can use
any eia-568[ab] cable, straight or crossed between them. (Note, USOC
cables still won't work, it has to be 568a or 568b pairing)
An argument could be made for individual VLANs to keep things like b- cast storms isolated. But I think the additional complexity will cause more problems than it will solve.
One must keep in mind that human error is the dominant cause of outages, and since there's not likely to be backhoes running around in a data center, IMHO the goal should be to remove as many ways as possible that your coworkers can muck things up.
Individual PTP links means a muckup probably affects only two devices. Switched LANs means a muckup possibly affects all devices (on one of the LANs), and not all of them may detect the problem at the same time.
Except when you implement the PTP links as VLANs on switches, it means
a muckup (to use your term) at the switch side can really muckup your
PTP links in non-obvious and often hard-to-troubleshoot ways. There
are tradeoffs either way. Personally, when interconnecting routers, I
tend to prefer the PTP hard link and skip the switches. Sometimes that's
not feasible. In those cases, generally, I prefer to go with rational
groups of routers on VLAN segments rather than synthetic PTP links.
However, each situation is different and the tradeoffs should be
considered in light of the particular situation.
...a physical mesh requires too many ports to be economical.
...a logical mesh has a couple of things against it. It
requires a lot of configuration, and each router will be
connected with a trunk interface, (on the antique switches
I've worked with) every trunk will carry all the traffic in
the switch, your maximum bandwidth across the whole switch
is 1gbps, instead of the next option which gives you more
bandwidth across the switch.
...two connections (non-trunked!) from each router to
seperate switches, with each switch having a separate /29 or
/28 for connected devices and a fast-responding IGP running
between the 5 routers gives you the most bang-for-the-buck
in terms of throughput and failure responsiveness. With
non-trunked interfaces, the switches can actually switch,
and you can squeeze more than 1gbps of bandwidth through it.
Even if you don't have that much traffic, you still have
less latency (ok, it's an infestimally tiny amount, but
every little bit helps.) If a switch completely fails and
the ethernet ports of the connected routers go down/down,
your IGP triggers and you have a fast failover. If a switch
fails and the ethernets stay up/up, you have a slow
failover, based on the timers of your IGP.
Behalf Of Scott Weeks
Sent: Friday, February 24, 2006 8:03 PM
To: nanog@merit.edu
Subject: Transit LAN vs. Individual LANs
I have 2 core routers (CR) and 3 access routers (AR)
currently connected point-to-point where each AR connects
to
each CR for a total of 6 ckts. Now someone has decided to
connect them with Gig-E. I was wondering about the
benefits
or disadvantages of keeping the ckts each in their own
individual LANs or tying them all into one VLAN for a
"Transit LAN" as those folks that decided on going to
Optimal solution is to dual-home each AR via PTP links
into the CR's. This has the simplest topology, fewest
components, highest throughput, and highest availability.
The only reason not to do this would the if you have poor
pricing on the CR ports or need a larger number of them:
Dual-home via PTP: 2 CR ports per AR
Dual-home to switched LANs: 4 CR ports + switches
If you know you are going to have just a handful of AR's
and have reasonable CR port costs, then the costs will be
similar and you should take the PTP approach.
An argument could be made for individual VLANs to keep things
like b- cast storms isolated. But I think the additional
complexity will cause more problems than it will solve.
Vlans will not stop all typres of broadcast storm.
So, perhaps I missed the earlier explanation, but why use switched
segments at all? if the purpose is to connect routers to routers putting
something that WILL FAIL in the middle is only going to increase your
labor costs later
So, for router-router links, GE doesn't have to mean switched...
Very true. In fact, GE is even easier because part of the GE standard
for UTP requires it to be Auto-MDI-Sensing (MDI vs MDI-X is handled
automatically in ALL compliant GE/TP interfaces).
Unfortunately it seems that not all devices actually implement MDI/MDI-X
IEE Std 802.3ab-1999, 40.4.4 (Page 93) says:
"Implementation of an automatic MDI/MDI-X configuration is optional for 1000BASE-T devices".
IEE Std 802.3ab-1999, 40.8,2 (Page 93) says:
"Although the automatic MDI-<DI-X configuration (see 40.4.4) is not required for successful operation of 1000BASE-T, is is a functional requirement that a cross-over function be implemented in every link segment to support the operation of Auto-Negotiation"
Now, seeing as Auto-Negotiation is required, it implies that automatic MDI/MDI-X is also required -- however, certain vendors seem to ignore this....
From my perspective...
...a physical mesh requires too many ports to be economical.
But, if one has the money, it's probably the better technical choice. Since his folks are already familiar with having things set up PTP using some other physical layer, that also reduces the odds of human error.
...a logical mesh has a couple of things against it. It
requires a lot of configuration, and each router will be
connected with a trunk interface, (on the antique switches
I've worked with) every trunk will carry all the traffic in
the switch, your maximum bandwidth across the whole switch
is 1gbps, instead of the next option which gives you more
bandwidth across the switch.
Not true, unless you're using some antique switching gear. Assuming all traffic is up/downstream and not sideways, you can get 4Gb/s in each direction (two CRs connected to two switches each). Whether you break that into PTP VLANs or shared VLANs shouldn't affect anything.
[ Note that this is moot since the OP responded he's running a physical mesh ]
S
Stephen Sprunk "Stupid people surround themselves with smart
CCIE #3723 people. Smart people surround themselves with
K5SSS smart people who disagree with them." --Aaron Sorkin
