IP-Internets

Coming from an academic research background, I need a reality check.
I would appreciate your answers/thoughts etc. to the questions
I have below on WANs.

1- are there any all-IP backbones? what is the layer 2 in IP
backbones (PPP, ATM, Frame relay...)?

2- how commonly is IP over ATM used? Do network operators really care
about the cell-tax?

3- what is the approximate number of connections to a
router (i.e., fan-in, fan-out of a router (i) at the edge, (ii) at the
core, and (iii) at the backbone?

4- what is the maximum distance between any two points in a autonm.
system?

5- what is the approximate ratio of copper/fiber
   at the edge, core, backbone links?

6- how tight is the physical space in the router rooms
(i.e., is it almost a must to take out an old box in order to add one)?

7- in case of a node/link failure what is the average/approx period of
time for (i) detection and (ii) recovery

I thank you in advance
Bulent

4- what is the maximum distance between any two points in a autonm.
system?

40,000 Km / 2 = 20,000 Km. -:slight_smile:

And I know at least 1 example, surely there is dozen of such networks.

Date: Thu, 14 Oct 1999 11:02:04 -0400 (EDT)
From: Bulent Yener <yener@research.bell-labs.com>
Subject: IP-Internets
  [...]
2- how commonly is IP over ATM used?

   From the ATM Forum Board of Directors Report - Rome, Italy, April 23,1999:
   
   ... Fred [Baker, chair of the IETF] stated that 75% of all Internet traffic
   today touches an ATM device and acknowledged the importance of the role
   of ATM in providing traffic engineering in today's networks. ...

I don't really know where this number comes from nor whether it is
really true.

A number of facilities-based carriers are either migrating or talking
about migrating from ATM to some sort of IP-over-fiber solution. I
believe that the economics of ATM versus IP-over-fiber solutions are
tremendously different for those who own fiber compared to those who
don't own the fiber.

Do network operators really care about the cell-tax?
  [...]

Define "care". Certainly, one sees a lot of whining about the ATM
"cell tax". On the other hand, if Fred Baker is correct, ATM is heavily
used in the Internet. Having said that, I believe that any ISP that
selects a service on the basis of some sort of overhead measurement, rather
than an evaluation of cost versus performance, is confused.

-tjs

   From the ATM Forum Board of Directors Report - Rome, Italy, April 23,1999:
   
   ... Fred [Baker, chair of the IETF] stated that 75% of all Internet traffic
   today touches an ATM device and acknowledged the importance of the role

     ^^^^^^^ - KEY WORD.

Touched - yes (this message touch ATM here twice). More interesting are
the slow tendencies.

The ATM as the background has the problem - the lack of QoS in
this 2 level schema if don't use very complex methods (compare RED+
PRECEDENCE for the DIRECT LINK - and MLPS or ATM Bundle /cisco/).
The common idea - if you have an hierarchy of the
different network stacks (ATM + IP this case), and there is not good way
to translate QoS markers from one level to another (in case of ATM, the
only way to do it is to distribute different QoS streams by the different
ATM circuits, which cause a lot of headache, and restrict the main
principle KISS - don't do QoS calculations where there is not
congestion). On the other hand, 2 level network structure realize another
GOOD network rule - make every decision ONCE /read packet, analyze it,
find outgoing device, mark it and send it there to this device - no
transit points should analyze the same packets again; and it makes
this 2 level schemas not so worst. The administrative issues (different
administration for the background network and IP network) are the benefits
(sometimes) too.

Really, if someone know some deep analyze of this tendencies, please, send
the reference here.

Alex.

   of ATM in providing traffic engineering in today's networks. ...

I don't really know where this number comes from nor whether it is
really true.

A number of facilities-based carriers are either migrating or talking
about migrating from ATM to some sort of IP-over-fiber solution. I
believe that the economics of ATM versus IP-over-fiber solutions are
tremendously different for those who own fiber compared to those who
don't own the fiber.

> Do network operators really care about the cell-tax?
> [...]

Define "care". Certainly, one sees a lot of whining about the ATM
"cell tax". On the other hand, if Fred Baker is correct, ATM is heavily
used in the Internet. Having said that, I believe that any ISP that
selects a service on the basis of some sort of overhead measurement, rather
than an evaluation of cost versus performance, is confused.

-tjs

Aleksei Roudnev, Network Operations Center, Relcom, Moscow
(+7 095) 194-19-95 (Network Operations Center Hot Line),(+7 095) 230-41-41, N 13729 (pager)
(+7 095) 196-72-12 (Support), (+7 095) 194-33-28 (Fax)

Bulent,

  I thought Lucent and Bell Labs already knew everything! :slight_smile:

1- are there any all-IP backbones? what is the layer 2 in IP
backbones (PPP, ATM, Frame relay...)?

  All networks which run IP are "all-IP" backbones. Perhaps
  this definition means "A Layer 2 forwarding infrastructure
  with only IP traffic". In this case, there are many.

  An RBOC FR network could be considered a backbone w/ partial
  IP, partial SNA, partial IPX, etc.

  Most all facilities-based ISPs run their IP networks on
  infrastructures with dedicated L1 TDM or WDM bandwidth.

  In Marketing-ese many companies, perhaps mine, say they have a
  "pure IP network" -- where they mean that there is no ATM or
  FR in the middle, no SONET APS below that, such that all of
  the brains and intelligence is embedded in the IP stuffs.

  Most of us that use to think that was a really sexy idea now
  think MPLS is sexy, so we're calling MPLS the 'forwarding
  protocol for IP' and putting some intelligence into MPLS
  and some into IP; so as to get the proper blend of intelligence
  for constructing robust, scalable, resilient IP transport
  networks. Except those wacky people at Qwest who just want
  to be different and are overtly influenced by brilliant
  yet obstinate swedes.

  All IP networks need a layer 2 framing protocol. Most use
  ATM, Frame_Relay, HDLC or PPP.

------------------------ = ------------------------
wr1.sfo1#show int pos0/0
POS0/0 is up, line protocol is up
  Hardware is Packet over SONET
  Description: WR2.SFO1-POS0/0-OC48
  Internet address is 206.132.110.73/30
  Encapsulation HDLC, crc 32, loopback not set
------------------------ = ------------------------

  Some stick MPLS in the middle ala IP/MPLS/HDLC or IP/MPLS/SONET.

  Additionally, IP/SONET may actually be IP/DWDM wherein the
  Layer 2 framing protocol uses SONET frames but w/out BLSR APS.

2- how commonly is IP over ATM used? Do network operators really care
about the cell-tax?

  Fairly commonly, especially among 'mature established large backbones'.
  Approximately 5 years ago, ATM switches had OC-12 interfaces while
  IP routers only had DS3 or partial-rate OC-3 interfaces.

  I'd estimate that 40-60% of Internet traffic traverses IP networks built on
  top of dedicated ATM networks. This is shrinking significantly, with at
  least one large atm based network transitioning to IP/MPLS.

  Therefore the backbone trunks were done w/ ATM to provide interface
  capacity. Since then perhaps people/systems have invested so much
  in them that it's hard to move away.

  Another benefit to using an ATM or FR infrastructure is the Traffic
  Engineering (TE) ability.

  Network operators care about the 'cell tax' but often one can
  increase the overall efficacy of the network with TE to result
  in higher gains than are costed by the ATM 'cell tax'.

3- what is the approximate number of connections to a
router (i.e., fan-in, fan-out of a router (i) at the edge, (ii) at the
core, and (iii) at the backbone?

  This is a function of design.

  In an ATM network, there will likely be 2 physical connections
  to the backbone ATM fabric, w/ N PVCs, in a full mesh, or truncated
  start topology.

  edge - usually 2 to the BB - 1 to 50,000 for customer aggregation
   varies by design

  core - usually 2-64 backbone links. most routers have 8-16
         line interface slots. Each slot will support 1 - 4
   interfaces, kinda usually...
   varies by design

  BB - see core, generally kinda the same.

4- what is the maximum distance between any two points in a autonm.
system?

  I suppose this is infinite. In practice, IP packets provide
  8 bits for the TTL, so a maximum 'diameter' is 2^8== 256 IP
  hops.

  Note that IP hops are just that, Layer 3 opportunities for
  Layer 3 forwarding decisions along w/ requisite TTL decrementation.

  These are often hidden, in networks with IP/ATM or IP/FR, and also
  sometimes with IP/MPLS. This is psuedo analagous to not counting
  SONET switches, amplifiers, repeaters, monitors, etc...

5- what is the approximate ratio of copper/fiber
   at the edge, core, backbone links?

  edge : 90% copper, 10 optical
  core: usually inter-office cabling is mostly optical, be it
   OC-n or gig-e.
  backbone: if BB ge OC-3; optical, if under OC-3, copper.

6- how tight is the physical space in the router rooms
(i.e., is it almost a must to take out an old box in order to add one)?

  space is one of the limited resources that an ISP has. It is
  generally a very significant issue.

  In many cases, 'silicon economics' allows folks to take out 2 boxes
  to put in one box that does 4 times as much.

  'forklift upgrades' are not always required.

  utopian designs provide 'buffer floor space' such that unused rack
  capacity is always available for new product insertion to avoid the
  need for 1:1 swap-outs during real time modifications.

7- in case of a node/link failure what is the average/approx period of
time for (i) detection and (ii) recovery

  depends. SONET/APS is alleged to be 50ms.

  ATM and FR PVCs tend to re-route on the order of 1-5 seconds.

  IP convergence tends to re-route on the order of 30 seconds.

  MPLS tends to reroute today on the order of 30 seconds, but real
  soon now on the order of 1 second.

  -alan

Not necessarily.

The bits would travel significantly longer distances than that assuming
even nearly perfect straight line runs.

If the question was in terms of equipment, (how many internal AS hops)
it could be pretty bad in a non-optimal situation. (like many primary
links down).

Don't know whether the question as "actual, optimal", "actual, worst",
"worst, worst", or "worst, actual".

Deepak Jain
AiNET

In article <Pine.SUN.4.10.9910142002090.547-100000@virgin.relcom.eu.net>,

Tim Salo <salo@networkcs.com> writes:

   From the ATM Forum Board of Directors Report - Rome, Italy, April 23,1999:
   
   ... Fred [Baker, chair of the IETF] stated that 75% of all Internet traffic
   today touches an ATM device and acknowledged the importance of the role
   of ATM in providing traffic engineering in today's networks. ...

I don't really know where this number comes from nor whether it is
really true.

99.9% of all Internet traffic touches an Ethernet device. To follow
Fred Baker's line of reason, this presents irrefutable proof that that
Ethernet is an appropriate technology for building one's backbone
network.

                                        ---Rob

3 bits of clarification:

  Some stick MPLS in the middle ala IP/MPLS/HDLC or IP/MPLS/SONET.

  This was dumb. I meant, ala IP/MPLS/HDLC or IP/MPLS/PPP.

  A couple of folks mailed me to state that MPLS as a transport
  technology was in the future, isn't it? In fact, 2 networks
  including our own have deployed large [mostly ubiquitous]
  MPLS deployments, where most traffic runs on it, one all over,
  another in certain regions/areas.

  Finally, I was 'reminded' that MPLS convergence will actually
  surpass SONET w/ restoral times, on the order of << 50 ms. Really
  real soon now.

  -alan

Of course, the laws of physics and mathematics will continue to
apply, even to MPLS.

                                         dave

That is confusing. Do the network operators actually
use the mpls on Label swithcing routers (LSR) now, or
the answers I am getting are for the future?

Are there any LSRs with MPLS in use anywhere?

I am trying to learn about the infrastructure currently
deployed not the one to come in the future?

That is confusing. Do the network operators actually
use the mpls on Label swithcing routers (LSR) now, or
the answers I am getting are for the future?

Are there any LSRs with MPLS in use anywhere?

  Yes, a few production IP networks use MPLS on Label Switching
  Routers now.

  Probably 90% of our backbone traffic traverses an MPLS LSP.

  As a matter of operational concern, we do not display Layer 3
  IP hops within the MPLS LSP. Therefore troubleshooting from
  an external perspective may be more difficult -- feel free to
  involve our NOC.

  LSRs with MPLS are in use in at least 3 networks today.

  1 has an almost complete deployment, 1 has a regional deployment,
  and 1 uses them as 'one-offs' for traffic engineering.

  (as another potentially confusing detail, 'standard IP routers' are
  'retrofitted' with MPLS, such that the same box becomes a hybrid
  IP and MPLS router/switch)

  -alan

Hello All, Anyone else noticing large packet drops thru either
  Seattle or SanJose (or both) ? Tia, JimL

Hello All, Anyone else noticing large packet drops thru either
Seattle or SanJose (or both) ? Tia, JimL

you may want to be specific about the network concerned. and then maybe
call their noc.