Fiber only in DataCenters?

Hello all,

Looking for input from "providers" as well as "consumers" of data center space and facilities. Specifically speaking to the types of available physical cross connects.

Are there data centers out there that are "fiber only"? That is to say that the cross connects are fiber only and no copper cross connects are available?

I understand that fiber is great and all, but to my knowledge there are a lot of folks still using and needing copper cross connects. Am I wrong?

In general the physical size and space needed for security make fiber
a requirement. Copper is very useful but ladder racks and managed
cabling can make runs much longer. Fiber is simply the next scale.

Many of the colocation datacenters or carrier hotel datacenters we are in only have copper facilities for tdm based circuits such as DS1 and DS3. The distance in many of these are simply too great for a copper ethernet connection. ( > 100m )

Some smaller ones prefer copper, where distance is not an issue, but I've found that to be the minority.

Hello all,

Looking for input from "providers" as well as "consumers" of data center space and facilities. Specifically speaking to the types of available physical cross connects.

Are there data centers out there that are "fiber only"? That is to say that the cross connects are fiber only and no copper cross connects are available?

I understand that fiber is great and all, but to my knowledge there are a lot of folks still using and needing copper cross connects. Am I wrong?

If the facility is big enough the utility of twisted pair becomes quite limited, both due to distance and differing electrical potential, multibuilding campuses in particular make this is a nonstarter.

In one facility I'm in, I'm over 300 meters from each of the MMRs, with the results that the OOB for the serial console server for out equipment located out there in the MMR's being on serial over fiber transceivers connected by om4 multimode.

walter, i couldn't care less about your problem! get the message!

I have yet to see a datacenter that doesn't offer both.

That does not mean that all runs within any given datacenter can be done via copper, but I don't now of any datacenters that do not make copper cross-connects available to customers that wish to purchase them.

Owen

I have not personally run across any DCs that do not permit copper Xcon,
but I am young, and not well-travelled.

Cheers,
-- jra

I believe all datacenter allow copper cross connections but maybe not
actually provisioned by copper. Most of my copper cross connections
are actually provisioned by in-house fibers plus FOT on both end.
Hence the hand-off looks like copper.

If the facility is big enough the utility of twisted pair becomes quite limited, both due to distance and differing electrical potential, multibuilding campuses in particular make this is a nonstarter.

For twisted-pair Ethernet: Distance yes. Differing electrical potential no. It is a balanced pair, transformer coupled at both ends. As long as AC common-mode pickup doesn't saturate the transformer core, it just works.

In one facility I'm in, I'm over 300 meters from each of the MMRs, with the results that the OOB for the serial console server for out equipment located out there in the MMR's being on serial over fiber transceivers connected by om4 multimode.

RS232 serial is another story. Here the potential difference between the ends is a big deal. (I've even seen burned-through PC boards from what happens when pin 7 has 220 VAC flowing from one device to the other) But you can just run Ethernet out to the console server and plug it in next to the gear with the serial port to fix this.

Matthew Kaufman

If the facility is big enough the utility of twisted pair becomes quite
limited, both due to distance and differing electrical potential,
multibuilding campuses in particular make this is a nonstarter.

For twisted-pair Ethernet: Distance yes. Differing electrical potential no.
It is a balanced pair, transformer coupled at both ends. As long as AC
common-mode pickup doesn't saturate the transformer core, it just works.

...Up to certain limits of DC / ground differential between the ends,
at which one can cause sparks anyways.

Yes, the POTS telcos use 48V in the same or lower quality wire pairs,
and the various CatN wires should be able to take it, and the
connectors. I'm not sure whether the sparks were from 110 or 220 V of
differential, but I saw sparks.

In one facility I'm in, I'm over 300 meters from each of the MMRs, with
the results that the OOB for the serial console server for out equipment
located out there in the MMR's being on serial over fiber transceivers
connected by om4 multimode.

RS232 serial is another story. Here the potential difference between the
ends is a big deal. (I've even seen burned-through PC boards from what
happens when pin 7 has 220 VAC flowing from one device to the other) But you
can just run Ethernet out to the console server and plug it in next to the
gear with the serial port to fix this.

Matthew Kaufman

Ah, yes, those magic smokes.

Sparks come from voltage, but wire tolerance is entirely a matter of amperage.

A 24ga cat-6 wire can take millions of volts as long as you keep the amperage
low enough.

Owen

In the ultimate limit, Insulator breakdown voltage is measured in
V/mm, but in this case it was almost certainly not that, and merely a
case of excessive amps at sufficient volts to give a nice large watts.
The subsequent facility power get-well was not cheap.

I have also, independently, melted and partly vaporized multiple cubic
centimeters of 8 ga wire with a (purely accidental, I assure you)
short of 12 volts from a serial stack of D-cell sized NiCd
rechargeable batteries. The same works well with an old car 12 V
battery and any conductor up to wrenches (not recommended at home...).

What's the old saying? Volts hurt, Amps kill?

It takes a lot of voltage to cause an arcing spark. I would suspect
static buildup along the way and bad grounding. Even a big facility
with a good ground should not have enough voltage differential between
grounding points to cause sparks. Having the right size rack grounding
should give you a very low resistance to ground from any point. The
most common problem I have seen in large facilities is multiple grounds
that are not tied together or cables that are grounded at multiple
points causing a loop current. It is critical that everything have a
single ground, that includes racks, electrical distribution, cable tray,
etc. Most Cat X cables are unshielded and do not have a ground
conductor so you must have equipment at the same potential at both ends
or you will get loop current for sure.

As far as voltage in Cat X cables, the real factor is the current
carrying capacity of a particular wire gage. It does not really matter
whether it is Cat 6 or a coat hanger, current capacity is a function of
cable cross section and what material it is made of. Copper has a
specific resistance as do all other metals. A copper cable needs to
have enough cross section to dissipate the heat generated by its
resistance. A less conductive material requires more cross section to
dissipate the increased heat. At extremely high voltages things become
more complex because of the skin affect that causes the power to move
through the outer parts of the cable more than the inner parts. These
levels are not a factor in communications cables.

The main factor for fiber over copper in data centers is all about cost.
Most servers include copper connections and fiber costs something extra.
For switches, the cost of the optics is significant. Fiber does help
prevent damage due to surges or electrical faults but if these are a
problem in your datacenter you have bigger fish to fry.

Steven Naslund

Yep.

Back in... this would have had to be 1994 or 5, the St Petersburg Main GTE
CO (SPBFLXA89F, I think) was off the air for about 12 hours.

Completely off the air.

As I understood the story later, some New Guy went into the battery room to
do some work, and had not yet gotten the memo about vinyl-dipped tools.

He supposedly got a 12" crescent wrench across the main bussbars.

Now, to properly appreciate this, you have to know that while the
load *at that time* was 30k lines of GTD5 and 2 50k line 5ESS remotes...
the building was 7 stories tall... and was originally all crossbar.

That battery room was rated, I was told, for about 8000A continuous at
-52VDC nominal.

The wrench flashed over into plasma on the spot; they never found any
of it. Just zits on the bussbars where it hit. The guy was deaf for
about a week.

I never did hear if he lost his job or not. They probably took the
cost of deploying all of St Pete's Police out to strategic street
corners to take emergency reports out of his paycheck, though.

Cheers,
-- jra

No it can't; the insulation's only rated for 250V or so. You get much past
a kilovolt and it will flash over.

Cheers,
-- jra