Sorry, this is somewhat OT.
I'm looking for information on energy consumption vs percent utilization. In other words if your datacenter consumes 720 MWh per month, yet on average your servers are 98% underutilized, you are wasting a lot of energy (a hot topic these days). Does anyone here have any real data on this?
Grisha
Sorry, this is somewhat OT.
Also Sorry, but I think the question itself is completely flawed.
I'm looking for information on energy consumption vs percent utilization.
In other words if your datacenter consumes 720 MWh per month, yet on
average your servers are 98% underutilized, you are wasting a lot of
energy (a hot topic these days). Does anyone here have any real data on
this?
What does 98% underutilized mean?
What is the utilization of a device with fully built out RAM that is used
to 100%, when the CPU is used 2% only?
What is the utilization of a system, that uses two percent of the
memory and two percent of the available CPU time, when the policy
of the top secret organization owning this system requires, that the
application is running on a seperated machine?
Sure many machines might be (computing power wise) able to
handle Firewalling, Routing, Webserving, Database Serving, Mailserving and
storing accounting data, but still there might be very good reasons to
seperate these on different machines.
If you take points like policy requirement (see above:
an application might by policy utilize a machine to 100%), different types
of resources, failover etc. into account, you might end up
with different numbers then just looking at the CPU (and I
have the feeling that is what you did or were intending to do).
Actually I think nobody does calculate "real" utilization,
as there are a lot of soft factors to be taken into account.
Nils
Which means you have to make sure the revenue generated by those 98%
underutilized servers covers your powerbill and other expenses,
preferrably leaving some headroom for a healthy profit margin.
As long as that's the case there's no real waste of energy, the services
people run on their servers are supposed to be worth the energy and
other costs, whether they physically fully utilize their power or not.
Cheers,
Actually I think nobody does calculate "real" utilization,
as there are a lot of soft factors to be taken into account.
Electrical usage for a datacenter is pretty consistent throughout a month, even as measured by a sum of days. The utilization of the systems inside of it are almost anything but consistent... even during boot up it would be nearly impossible to determine the instantaneous necessary power draw.
Separately, deploying applications to clusters of machines where the cluster is dynamically resized [more machines are turned on/off] depending on load is a non-trivial function and outside the operational experience/need of most customers.
But even assuming you could do that, the best approximation I could imagine for an Internet data center would be something akin to its network traffic graph [assumption being that network load amongst a stable set of customers is proportionate to the processing power required to produce it... even if an individual customer uses much more CPU power to do that at a specific time quanta]. Basically, if you use 1Mb/s at noon on Monday, and 1.2Mb/s at noon on Tuesday with the same customer set, you can probably estimate that your system's load is 20% higher than it was on Monday. Assuming you aren't operating at either the very low extreme or very high extreme. At least that would be my thought.
If all applications were designed to virtualized ala mainframe style, this clustering concept might work to dynamically redeploy resources... However the mainframes themselves are inherently not-smooth-stepped in terms of their power/cpu curves, so its probably a dead issue in that regard.
Deepak Jain
AiNET
Erik Haagsman wrote:
Which means you have to make sure the revenue generated by those 98% underutilized servers covers your powerbill and other expenses, preferrably leaving some headroom for a healthy profit margin. As
long as that's the case there's no real waste of energy, the services
people run on their servers are supposed to be worth the energy and other costs, whether they physically fully utilize their power or
not.
Yet there are a lot of clusters which are designed for peak load, which will waste energy during non-peak hours. Developing an in-house system for shutting down power to excess servers in a cluster might increase the healthy profit margin.
-Jack
Hello,
I've done quite a bit of studyin power usage and such in datacenters over the last year or so.
I'm looking for information on energy consumption vs percent utilization. In other words if your datacenter consumes 720 MWh per month, yet on average your servers are 98% underutilized, you are wasting a lot of energy (a hot topic these days). Does anyone here have any real data on this?
I've never done a study on power used vs. CPU utilization, but my guess is that the heat generated from a PC remains fairly constant -- in the grand scheme of things -- no matter what your utilization is.
I say this, because, with a CPU being idle of 100% utilized, they still are grossly inefficient, on the order of less than 10% in all cases (ie, 1 watt in returns at least .9 watts of heat, no matter loading of the CPU).
-- Alex Rubenstein, AR97, K2AHR, alex@nac.net, latency, Al Reuben --
-- Net Access Corporation, 800-NET-ME-36, http://www.nac.net --
Remember in your analysis to include premature hardware failure due to too many
power cycles...
A server can *easily* "on average" be running at only 20-30% of capacity,
simply because requests arrive at essentially random times - so you have to
deal with the case where "average" over a minute is 20% of capacity for 600
hits (10/sec), but some individual seconds only have 1 hit, and others have 50
(at which point you're running with the meter spiked).
Time-of-day issues also get involved - you may need to have enough iron to
handle the peak load at 2PM, but be sitting mostly idle at 2AM. Unfortunately,
I've seen very few rack-mount boxes that support partial power-down to save
energy - if it's got 2 Xeon processors and 2G of memory, both CPUs and all the
memory cards are hot all the time...
There's also latency issues - if some CPUs on a node or some nodes in a cluster
are powered down, there is a timing lag between when you start firing them up
and when they're ready to go - so you need to walk the very fine line between
"too short a spike powers stuff up needlessly" (very bad for the hardware), and
"too much dampening means you get bottlenecked while waiting for spin-up".
(Been there, done that - there's a 1200-node cluster across the hall, and
there's no really good/easy way to ramp up all 1200 for big jobs and power down
800 nodes if there's only 400-nodes worth of work handy. So we end up leaving
it all fired up and let the node's "idle loop" be "good enough")..
If it was as easy as all that, we'd all be doing it already.. 
Sorry, this is somewhat OT.
I'm looking for information on energy consumption vs percent utilization.
In other words if your datacenter consumes 720 MWh per month, yet on
average your servers are 98% underutilized, you are wasting a lot of
energy (a hot topic these days). Does anyone here have any real data on
this?Grisha
It is only waste is the P & L statement is showing no profit.
Thats an insane statement.
Are you saying, "You are only wasting money on things if you aren't profitable" ?
/action shakes head.
It's more or less the truth though. Only on rare occasions, such as the
cluster/fail-over scenario given, can you actually supply less power to
certain machines, and power use largely unrelated to their actual
utilisation. Keep an eye on your UPS load during peak hours and you'll
see the load rising when traffic and server utilisation rises, but
compared to the baseline power needed to feed servers these fluctuations
are peanuts.
You supply a server with enough power to run...how is this waste
exactly...? If anyone is wasting anything, it's perhaps hardware
manufacturers that don't design efficiently enough, but power that you
provide and that's used (and paid for) by your customers is not wasted
IMO.
Cheers,
Erik
It's more or less the truth though.
I think the comment was outside of the scope of the original discussion. It seemed to me that:
It is only waste is the P & L statement is showing no profit.
inferred that any business practice is OK, as long as your are profitable. It is that concept that I felt was insane.
-- Alex Rubenstein, AR97, K2AHR, alex@nac.net, latency, Al Reuben --
-- Net Access Corporation, 800-NET-ME-36, http://www.nac.net --
Thats an insane statement.
Are you saying, "You are only wasting money on things if you aren't
profitable" ?/action shakes head.
No, I am not but my statement did sure sound like that was what I was
saying.
I do think it is apples or oranges comparing CPU % to total power used and
coming up
with a wasted factor. My colo needs X amps/hour just to run at idle, I don't
call this waste.
It is the cost of doing business. Power factor causes losses. So you need
enough customers to cover
this and other expenses.
I guess we need a definition of waste here.
I would say the the heat produced by pulling all the amp/hrs is waste. It
could be possible to harvest
this and reuse it elsewhere.
So, just because you are profitable does not mean there is no waste but it
also depends on how you
classify waste. Also, do the methods to avoid this waste justify (pay for
over time) their use.
James H. Edwards
Routing and Security Administrator
At the Santa Fe Office: Internet at Cyber Mesa
jamesh@cybermesa.com noc@cybermesa.com
http://www.cybermesa.com/ContactCM
(505) 795-7101
Alex Rubenstein wrote:
I'm looking for information on energy consumption vs percent utilization. In other words if your datacenter consumes 720 MWh per month, yet on average your servers are 98% underutilized, you are wasting a lot of energy (a hot topic these days). Does anyone here have any real data on this?
I've never done a study on power used vs. CPU utilization, but my guess is that the heat generated from a PC remains fairly constant -- in the grand scheme of things -- no matter what your utilization is.
You should be able to pick up simple current / wattage meter from local hardware store for $20 or so. That will tell you that on a modern dual-CPU machine the power consumption at idle CPU is about 60% of peak. The rest is consumed by drives, fans, RAM, etc. As wattage the difference is 100-120W (50-60W per cpu)
All modern operating systems do moderate job of saving CPU wattage when they are idle (BSD's, Linux, MACOS X, WinXP, etc.)
Pete