An area for operations growth - Storage Area Nets in MANS

Mailman List Mirror (Read Only)
1

Roxane Googin is usually ahead of the competition in spotting trends in data net use. here is what has has to say about enterprise use of MANs.

COOK Report: Where are the metro area networks going?

Googin: Probably the killer application for the MAN is grid computing and storage area networks. A lot of people think the driver for broadband access to the home is going to be multi-media. People, when they think about next generation networks, think home use. But this is never where the money has been. And no phone company is ever going to build the infrastructure.

Although these new real time applica tions will clearly send more data over the network, the real killer application is going to be remote storage and synchronous storage. Synchronous storage means that you have two large servers doing the exact same thing at the exact same time in two different locations.

COOK Report: Like a decentralized disk array?

Googin: Yes. The backbone has to be incredibly fast because you cannot complete a transaction until you have acknowledgments from both disk drives. This will happen. Probably this year. What they are already doing is taking fiber channel and putting that on a Cienna Core Director optical switch port. Half of the ports being sold on the Core Director now are fiber channel. They aren't even Ethernet. And this is used for storage area nets (SANs). These are corporate MANs and will have nothing to do with sales to service providers. They are bypass business services where the storage arrays may not be more than a kilometer or two apart. These SANs are backing up continuously terabytes of data. We are talking huge applications that will use every bit of access to every bit of capacity they can get.

COOK Report: Is 9/11 a motivation for this?

Googin: Partly. Not only that but the whole paradigm of the real time organization will drive it. It used to be that your server had its own storage. It was a "stove pipe" connected to the CPU. Now as soon as you decouple that "stove pipe," you can put it anywhere. What they are finding is that if they have two of them that are mirrored in real time and place remotely that they do not have to do as much management of resources.

COOK Report: Then forget the disaster back up and recovery operations? Those are the next to be marked for extinction?

Googin: Oh yes. It is so elegant. The new architecture cannot be supported on direct attached storage. It must move off the server and it is doing so. If you have to go through a server to get to the storage, it simply doesn't work. Storage can't become just a utility until you move it off the server. The MAN will be backing up terabytes of data regularly and in real time. Disaster recovery, in real time, comes for "free".

For more detail please see

Fiber & Wireless as First Mile Technology - Fiber Business Models & Architecture http://cookreport.com/12.04-06.shtml

2

Although these new real time applica tions will clearly send more
data over the network, the real killer application is going to be
remote storage and synchronous storage. Synchronous storage means
that you have two large servers doing the exact same thing at the
exact same time in two different locations.

COOK Report: Like a decentralized disk array?

Googin: Yes. The backbone has to be incredibly fast because you
cannot complete a transaction until you have acknowledgments from
both disk drives. This will happen. Probably this year. What they
are already doing is taking fiber channel and putting that on a
Cienna Core Director optical switch port. Half of the ports being
sold on the Core Director now are fiber channel. They aren't even
Ethernet. And this is used for storage area nets (SANs). These are
corporate MANs and will have nothing to do with sales to service
providers. They are bypass business services where the storage
arrays may not be more than a kilometer or two apart. These SANs are
backing up continuously terabytes of data. We are talking huge
applications that will use every bit of access to every bit of
capacity they can get.

Erm - perhaps I'm misunderstanding what Googin is trying to say here,
but if he's talking about synchronous remote replication over fibre
channel, this exists today. In fact, it has existed for years, and
before it was over fibre channel, it was over ESCON. Today, still, if
you want to go father than a certain distance (70km maybe?) its
generally recommended to switch over to ESCON rather than fibre channel.

In practicality, complete remote replication is often inadvisable for
high-performance, heavy-write applications. The rule of thumb is that
every KM adds another millisecond worth of I/O transaction time, so a
few kilometers distance can add a significant overhead to writes.

Not to mention Fibre Channel is very unkind when it comes to recovery
from fabric segmentation. A poorly designed (or just very unlucky) SAN
can be completely downed on both sides of the split, somewhat ruining
the disaster recovery strategy if the production and DR storage networks
are both taken down.

Ugh - I'll be very happy when fibre channel is dead and buried.

Thanks,
Matt

3

Sorry for any confusion. Fiber channel is old tech for sure and there was no implied intent to evaluate the tech itself. Roxane has apparently observed that on devices like the core directors the number of fiber channel ports being ordered is going up dramatically.... apparently about the only thing it makes sense to do with such ports is to use them for SANs. The remark was not intended to be definitive or in great depth and was added almost as an after thought at the end of a short (35 minute) interview at the san jose airport.

  Certainly there is major movement in large fiber based in metropolitan areas that have been built and are being built by banks, and fortune 500 kinds of enterprises as they move a LOT of data and voice traffic off the PSTN and on to new cheap equipment that they can get a much better return for on the dollars they invest. All a part of the commoditization and decentralization of telecom.

I am not an expert in the application of the technology being discussed. However at some time in the next six month i probably will want to explore this in much more depth.

4

Although I agree that storage networking as a backup/tandem resiliency operation makes a lot of sense,
it does not come anywhere nearly "free" because the assumptions made there require flawless software.
If you always synchronize your systems you'll also blow them both out when you hit a software issue
and as most of the audience is painfully aware, they are not infrequent enough to count out of the equation.

So you need to keep track how to re-synchornize after an upgrade/failure/etc. With a large stream of
updated data on the system keeping the state difference becomes expensive quickly. (unless you are going
to re-sync everything after a failure, and then you'll be vulnerable until that's completed)

Pointers to perfect SAN solutions appreciated.

Pete

Gordon Cook wrote:

5

Non-flawless software (i.e., 100% of it) is usually dealt with storage
snapshots; if they are kept local or remote, that's a design choice that a
metro SAN can impact.

Doing backup/restores thru a metro SAN also has some advantages on
facilities with tight-controlled physical access; the bandwagon that moves
tapes to off-site storage still has more bandwidth than the fiber, but it
can also be used to smuggle <insert-your-favorite-paranoia-here>.

Rubens

6

Yeah, people with serious business continuity needs use real-time
replication for datacenter disasters, plus point-in-time copies to
protect against software error that often get replicated remotely as
well. That way, the DR servers can serve a purpose doing regression
testing, development work, integrity testing, etc, without exposing the
possibility of unrecoverability in the event of a disaster by disabling
the real-time replication to do testing.

The other interesting thing in terms of point-in-time replication is
what has come to be known as near-line storage. Big boxes of
inexpensive drives that are used to offload point-in-time copies from
expensive storage for long-term retrieval. This is someplace where
metro networks could come into play - having organizations use this
excess bandwidth to offload encrypted copies of various data sets to a
centralized point run by some neutral third-party. This would be a good
way to enforce data retention policies and offsite storage of data.
Like an Iron Mountain, only without all the irritation of trucks,
people, and magnetic tapes. Economies of scale, plus much better
recovery SLAs than having someone retrieve tapes out of a vault
somewhere.

Thanks,
Matt