2008.02.20 NANOG 42 current state of 10gig pluggable technologies

Continuing on with the notes from this morning's talks,
pushing out the notes while we take a bit of a break. :slight_smile:


2008.02.20 10 gig pluggable technology

Richard A Steenbergen from nlayer

Many thanks to Todd for donation of the

Title is 10gig, but it applies to other
technologies as well; fiberchannel, etc.

What are pluggables? transcievers you
plug into your router or switch, they
can drive multiple media types (copper,
fiber, short reach, long reach, etc)

There's standardization among the parts,
so that vendors can use common parts.

technical benefits, you can mix-and-match
media types on the same router/switch card;
easier replacement in the event of optical

financial benefit--pay as you populate.
reusable, easily re-deployable elsewhere
cards are reusable as technology evolves;
can migrate from fiber to copper easily.
standardization lowers costs.

standardized, allows for interoperability
standardized via vendor MSA (multiple source
specs published under SFF, which defines the
strict specs to be followed.

First gen pluggable technology; started in
ethernet land with GBIC, first came from the
fiberchannel world; commonly in use from 1998
to 2002.

From 2002 onward, started shifting to sfp

form factor; multirate, so you can use same
pluggable for 1g, 2g, 4g fiberchannel, can
use for gig or OC48 SONET
First introduced digital optical monitoring
(DOM) in 2004.
better density.

1g to 2.5g, had GBIC and SFP.
For 10G, many, many choices (7)

Problem in operator community now is that with so
many options, sparing is hard.

slide with overall size comparisons.

First 10G was 300-pin MSA; not pluggable, but a
snap-on connector; 8-14W, 16x622Mbps interface.
Same thing used for 40G

XENPAK MSA, march 2001, second most popular at this
power 6-10W
XAUI (4x3.125G)
PHY framer onboard
80km+ ZR, DWDM tuned, other exotic optics

X2 MSA, July 2002
same electrical as Xenpak
4-5W, rest is same as xenpak, just smaller.
Cisco and HP are only users of it; lukewarm
support for it.
very few exotic optics choices for it.

similar to X2, XAUI interface
uses LC connector, different form factor, for
use on PCI cards
no exotic optics
not used in network world.

XFP MSA, march 2002
XFI (9.995-11.1G serial interface)
variable speed serial interface
PHY framer is offloaded; can be 10G, OC192, transport,
One of most commonly deployed optics
exotic: ZR/DWDM, limited CX4, no LX4
very, very popular
eliminating serdes for 10GBASE-RW is big power saver
power 1.5-3.5W

SFP+ MSA draft dec 2007
interface SFI (8.5 - 11.1G)
same dimensions as SFP, same connector
extend down to do 8G fiberchannel
PHY framer offloaded
no exotic optics
smaller than XFP, offloads CDR function
SFI similar to XFI, adds 8G FC speed
limited power use
power 1-1.5W

10gig pluggable component technology
PHY, physical layer component
PCS --physical coding sublayer
PMA physical medium attachment sublayer
CDR clocking data recovery chip

10gig PHY comes in 3 variants
  similar to R, but wrapped in OC192 SONET compatible frame
10GBASE-X Lan phy 4x2.5G parallel signal
  LX4 is 4 optical channels, mux built in
  CX4 is 4 copper channels

same choices for interconnect; XAUI is 4x3.125G parallel
channels (8B/10B encoding)
XFI/SFI, single 10gig serial channel
  uses 64B/66B, much more efficient encoding

each time you convert, it increases heat, power,
consumes port space, etc.

Ultimate goal is to match technologies; if you
have 4 parallel lanes on each side better

So XAUI to 10GBASE-R, inefficient
XENPAK has room for serdes stages, fortunately

Future directions for technology?
new technogies replace old
10GBASE-LRM long reach multimode replaces LX4
10GBASE-T to replace old CX4 for copper
  uses standard cat6/cat7, 8p8c (RJ45) connectors

Advantages of offloading PHY framer
can use same pluggable for multiple protocols
allows better component reuse, lowers costs
can use in transport gear with forward error correction
10GE WAN PHY much bette when done on host
can use any PMD without having to buy special pluggables
improves sparing, lowers costs, expands PMD options

improves WAN layer control signalling and alarms
provides access to SONET alarms, path trace, etc.
vastly improves troubleshooting with OC192 carriers

Advantages of offloading the CDR
make pluggable smaller, uses less power
  mostly zero sum gain; no components saved, just moved
  to board
not all CDRs are created equal
  may want higher quality on longer runs
  can't upgrade pluggable to get better EDC (electronic
  disperson compensation) technology

Evolution of pluggable power use (watts)
power consumption going down over time;
different classes of power use.
Not only dropping power, but removing entire
power classes.

comparison chart of optics slide.

Software side of pluggables
talk to host via low speed control bus; power down,
alarm signalling, etc.

Digital Optical Monitoring lets you see power levels
on the link, speeds up troubleshooting.
TDR functionality for copper
hosts can also read EEPROM data
allows vendors to lock users into particular

Vendors don't make their own pluggables, they're
made by JDS Uniphase, finisar, hitachi, etc.
pluggable has EEPROM, set vendor string
$1.4B in profit at Cisco from reselling pluggable

how keep customers locked in:
FUD (won't support it, might blow up, support contract
is void, etc.)
financial component; many sellers, few buyers; if
Cisco controls 70% of market, they can set the
prices, and prevent them from being resold outside
their channel.

Vendor locking--if it's not ours, router won't support
Big customers are pushing back on that.

counterfiting cisco is now the new way to make money,
as you make profit by charging for name.

new model is doing feature impairment; certain features
like DOM are set so that it's only usable with their
own pluggable.

What format is right for you?
XENPAK has best selection, well stocked, most exotic
  optics options
downside; very large, draws a lot of power, not protocol
  agnostic, not friendly with WAN PHY

X2--smaller than xenpak, lower power allows for higher
full support for LX4/CX4, pushed by cisco
very little deployed base, few sources, few exotic optics
same as XENPAK, not protocol agnostic

XFP--advantages--large deployed base, cheap, easy to
  find, technologies still being upgraded on it.
power/density pretty good for most users.
protocol agnostic, easy spare, good WAN PHY
cheapest option for long reach/DWDM

SFP+ -- very high density (48 per blade)
  physically same as SFP
may allow for extremely low cost device (GoogleSwitch)
downside; hard cutoff on power budget, hard to do longer
  reach optics; no DWDM, no LR, no ZR; won't ever be able
  to be full replacement for XFP because of that.