Data center capacity is the aggregate capacity of all data center assets. These assets extend beyond the physical and virtual systems of an IT infrastructure to the equipment that powers and regulates the actual data center climate and environment. Capacity is not easy to manufacture, nor does it comes cheaply. Today, IT must rethink data center capacity management from a “what’s required now” mentality to a far more strategic vision of dynamic capacity allocation. Forced to juggle its priorities between business demands and increasingly complex data center provisioning strategies, IT is even more hampered by a flat or only marginally growing budget.

Under such tight conditions, data centers have no room for capacity mismanagement. Overprovisioning servers is an unacceptable money drain on the organization, and capacity shortages can cause performance degradation or even service disruption. Because business advantage and competitiveness depend so much on both an enhanced technology infrastructure and the optimized capacity to run it, the data center must shift pace and become more agile—ready to anticipate urgent new business demands and embrace new technologies while remaining within its current cost structure.

Achieving data center agility requires optimized capacity based on a unified view of data center performance starting at the individual asset level. Comprehensive asset visibility, analysis and control are the prerequisites for every agile data center.

Data Center Performance Management (DCPM): The Business and Data Center Agility Imperative

Data center performance management (DCPM) takes a unified approach to benchmarking and forecasting—against dynamic business needs—of the aggregate performance of data center assets. This unified approach is highlighted by Forrester Research analyst Jean-Pierre Garbani in his 2011 article “If you don’t manage everything, you don’t manage anything,” offering a clear message: failure to monitor one element can lead to the failure of the entire system. Specifically, Garbani draws attention to the failed design of an early-generation Citroen 2CV gas gauge as an analogy to explain why IT should focus on all components of the IT infrastructure: the minimalist approach of the highly efficient car used a dipstick, as opposed to a dashboard gas gauge, that often left drivers stranded when they forgot to check the dipstick for the fuel level in the tank. This is an example of “a great means of transportation [that] failed regularly for lack of instrumentation.”

Garbani emphasizes that application performance management (APM) must take a unified approach to managing IT infrastructure—hardware, software, virtual and physical—and ensure that all components perform to keep an application up and running in an optimized fashion.

Data center performance management takes this APM model one step further, measuring not only IT asset utilization and response time but also environmental metrics such as power utilization and peak demand against business objectives. Why? Failure of one underlying component supporting the IT infrastructure (e.g., maxed-out power capacity) could disrupt business-critical applications and processes. With the data center being at the core of every business-critical application, IT shortens the path to agility by taking a holistic approach to managing data center performance—by accurately analyzing forecasting and planning for system and environmental variables so as to ensure uptime and available capacity of every ongoing and upcoming project.

DCPM requires three foundational capabilities to deliver capacity, performance and agility: visibility, analysis and control.

1. Visibility. Broad, deep and continuous monitoring unites comprehensive asset visibility with performance trends to give IT both “in the moment” and historical performance intelligence about all the data center resources that support its business-critical applications. Myriad application performance and enterprise management software exist today, but they provide only a partial view of their attributes. What IT needs is a “manager of managers” approach to monitoring. This approach uses the integration of physical and virtual systems along with any enterprise/IT asset management and building management systems already in place for a unified view of the data center.
2. Analysis. Continuous performance analysis correlates individual asset KPIs—such as computational utilization, memory, storage, network and power—and compares them against each asset’s capacity limit to determine available headroom, or it analyzes the historical maximum/minimum/average capacity for performance comparison. These measurements bring critical insights into each component’s current operating capacity, as well as highlighting potential bottlenecks and wasted resources that demand attention. Using DCPM, IT can pinpoint which applications are the major consumers of each resource and drill down into the root causes of resource contention or capacity shortages. IT can also investigate virtual-machine memory allocation as well as storage oversubscription or peak power demands of the physical environment by analyzing deviations from planned efficiencies. This analysis is a continuous, iterative process that must begin with baseline measurements of workloads, systems and equipment. Having a historical perspective on consumption, utilization and costs, IT will be able to analyze variances and take action to improve data center performance through accurate demand forecasts and what-if planning.
3. Control. Capacity forecasting and what-if planning gives IT hands-on control to ensure data center agility. DCPM functions can accurately portray current resource usage and available capacity for new applications on the basis of a specific asset’s historical resource consumption and its ongoing utilization patterns. In addition, continuous capacity planning enables IT to forecast which physical or virtual assets can effectively serve specific applications or workloads, giving IT the ability to proactively shift workloads on the fly to more suitably assets and better meet the dynamic needs of the business.

In addition to forecasting asset utilization and capacity, DCPM provides what-if scenarios to help IT evaluate cost/performance tradeoffs for multiple hardware options and service deployment strategies: virtualization, cloud computing, server consolidation, hardware refresh and so on. Approaching this type of planning manually, using spreadsheets or a hodgepodge of planning tools, requires “an army of elves” to map, track and analyze change and its related impact. With DCPM, IT can automate data center capacity planning by focusing on the important objectives: providing accurate data center capacity forecasts and appropriate data center capacity to meet business needs.[1]

DCPM With Six Sigma

By combining a DCPM platform with the Six Sigma DMAIC best-practice framework for capacity planning, IT and data center professionals can achieve continuous data center resource optimization via the following steps:

1. Define data center goals: IT initializes the capacity planning process by setting the overall data center SLA and key metrics against business goals, followed by identifying future requirements and trends.

2. Measure data center baseline: Through its extensive library of connectors and integration with enterprise management software, a DCPM platform delivers comprehensive data center infrastructure metrics for baseline measurements. Where asset connectivity is not directly available, virtual meters and inference engines could fill in the gaps, providing highly accurate inferred metrics. The resultant baseline measurements allow IT to compare actual data center performance with their overall performance targets, as well as to spotlight any variance from that baseline over time.

3. Analyze data center variances: A DCPM platform can provide rich and complete history by way of correlating data across multiple data centers. Using in-memory data storage and query techniques, time-series precalculations and various other performance optimizations, a DCPM can present root-cause analysis of these variances and help in the formulation of corrective actions.

4. Improve data center infrastructure: A DCPM can use what-if scenarios for assessing new workloads and application deployment strategies, evaluating new equipment, or determining capacity for VM placement. These scenarios not only help IT evaluate and select the best scenario but also accurately forecast and plan for the capacity needed to support new application rollouts or increased demand.

4a. Forecast utilization and capacity: A DCPM platform uses historical consumption and utilization patterns of assets to predict their future demand and available capacity.

4b. Plan with what-if scenarios: A DCPM platform should auto-populate key parameters with smart default values (which are calculated from real utilization, consumption and cost data), allowing IT to predict operating impact on the basis of asset changes.

4c. Select best scenario: A DCPM platform would allow IT to compare, choose and save desired scenarios.

4d. Execute on the selected plan: A DCPM platform routes the selected plan to service desk tickets (in HP, BMC, IBM Tivoli and so on).

4e. Check actual performance against plan: A DCPM platform compares baseline performance against the selected plan, providing IT with intelligent insights leading to corrective actions.

5. Control data center operations: A DCPM platform should provide real-time monitoring for new variances, enabling IT to repeat plan-do-check-act steps continuously to correct variances.

5a. Check variance against plan: A DCPM platform alerts and reports on threshold crossing.

5b. Analyze root causes: A DCPM platform drills down to details of failure and determines what to fix.

5c. Act to resolve problems: A DCPM platform should route issues to service desk tickets for problem resolution.

Through this intelligent capacity planning process, IT/data center professionals can see whether available physical and virtual IT infrastructure, power, and space capacity exist to support the upcoming application rollout. The capacity forecast and performance prediction reduce risk by enabling IT to analyze cascading impacts of upcoming projects before installing any systems.

DCPM is at the heart of the agile data center. With clear insight into downstream resource requirements, IT can focus on implementing the necessary changes to operate—continuously—at optimum capacity, meeting the requirements that support an agile business.

About the Author

Bob Ertl is senior director of product management at Sentilla Corporation (www.sentilla.com), headquartered in Redwood City, Calif. Before joining Sentilla in 2011, Bob worked in product management roles at Oracle, Hyperion Solutions and Brio Software. He is available for questions and comments at bob.ertl@sentilla.com.

Preparedness for unexpected power events requires the ability to rapidly identify the individual servers at risk of power overload or failure. A variety of proactive energy management best practices can not only provide insights into the power patterns leading up to problematic events, but can offer remedial controls that avoid equipment failures and service disruptions.

Best Practice: Gaining Real-Time Visibility

Dealing with power surges requires a full understanding of your nominal data center power and thermal conditions. Unfortunately, many facilities and IT teams have only minimal monitoring in place, often focusing solely on return air temperature at the air-conditioning units.

The first step toward efficient energy management is to take advantage of all the power and thermal data provided by today’s hardware. This includes real-time server inlet temperatures and power consumption data from rack servers, blade servers, and the power-distribution units (PDUs) and uninterrupted power supplies (UPSs) related to those servers. Data center energy monitoring solutions are available for aggregating this hardware data and for providing views of conditions at the individual server or rack level or for user-defined groups of devices.

Unlike predictive models that are based on static data sets, real-time energy monitoring solutions can uncover hot spots and computer-area air handler (CRAH) failures early, when proactive actions can be taken.

By aggregating server inlet temperatures, an energy monitoring solution can help data center managers create real-time thermal maps of the data center. The solutions can also feed data into logs to be used for trending analysis as well as in-depth airflow studies for improving thermal profiles and for avoiding over- or undercooling. With adequate granularity and accuracy, an energy monitoring solution makes it possible to fine-tune power and cooling systems, instead of necessitating designs to accommodate the worst-case or spike conditions.

Best Practice: Shifting From Reactive to Proactive Energy Management

Accurate, real-time power and thermal usage data also makes it possible to set thresholds and alerts, and it introduce controls that enforce policies for optimized service and efficiencies. Real-time server data provides immediate feedback about power and thermal conditions that can affect server performance and ultimately end-user services.

Proactively identifying hot spots before they reach critical levels allows data center managers to take preventative actions and also creates a foundation for the following:

• Managing and billing for services based on actual energy use
• Automating actions relating to power management in order to minimize the impact on IT or facilities teams
• Integrating data center energy management with other data center and facilities management consoles.

Best Practice: Non-Invasive Monitoring

To avoid affecting the servers and end-user services, data center managers should look for energy management solutions that support agentless operation. Advanced solutions facilitate integration, with full support for Web Services Description Language (WSDL) APIs, and they can coexist with other applications on the designated host server or virtual machine.

Today’s regulated data centers also require that an energy management solution offer APIs designed for secure communications with managed nodes.

Best Practice: Holistic Energy Optimization

Real-time monitoring provides a solid foundation for energy controls, and state-of-the-art energy management systems provide enable dynamic adjustment of the internal power states of data center servers. The control functions support the optimal balance of server performance and power—and keep power under the cap to avoid spikes that would otherwise exceed equipment limits or energy budgets.

Intelligent aggregation of data center power and thermal data can be used to drive optimal power management policies across servers and storage area networks. In real-world use cases, intelligent energy management solutions are producing 20–40 percent reductions in energy waste.

These increases in efficiency ameliorate the conditions that may lead to power spikes, and they also enable other high-value benefits including prolonged business continuity (by up to 25 percent) when a power outage occurs. Power can also be allocated on a priority basis during an outage, giving maximum protection to business-critical services.

Intelligent power management for servers can also dramatically increase rack density without exceeding existing rack-level power caps. Some companies are also using intelligent energy management approaches to introduce power-based metering and energy cost charge-backs to motivate conservation and more fairly assign costs to organizational units.

Best Practice: Decreasing Data Center Power Without Affecting Performance

A crude energy management solution might mitigate power surges by simply capping the power consumption of individual servers or groups of servers. Because performance is directly tied to power, an intelligent energy management solution dynamically balances power and performance in accordance with the priorities set by the particular business.

The features required for fine-tuning power in relation to server performance include real-time monitoring of actual power consumption and the ability to maintain maximum performance by dynamically adjusting the processor operating frequencies. This requires a tightly integrated solution that can interact with the server operating system or hypervisor using threshold alerts.

Field tests of state-of-the-art energy management solutions have proven the efficacy of an intelligent approach for lowering server power consumption by as much as 20 percent without reducing performance. At BMW Group,[1] for example, a proof-of-concept exercise determined that an energy management solution could lower consumption by 18 percent and increase server efficiency by approximately 19 percent.

Similarly, by adjusting the performance levels, data center managers can more dramatically lower power to mitigate periods of power surges or to adjust server allocations on the basis of workloads and priorities.

Conclusions

Today, the motivations for avoiding power spikes include improving the reliability of data center services and curbing runaway energy costs. In the future, energy management will likely become more critical with the consumerization of IT, cloud computing and other trends that put increased service—and, correspondingly, energy—demands on the data center.

Bottom line, intelligent energy management is a critical first step to gaining control of the fastest-increasing operating cost for the data center. Plus, it puts a data center on a transition path towards more comprehensive IT asset management. Besides avoiding power spikes, energy management solutions provide in-depth knowledge for data center “right-sizing” and accurate equipment scheduling to meet workload demands.

Power data can also contribute to more-efficient cooling and air-flow designs and to space analysis for site expansion studies. Power is at the heart of optimized resource balancing in the data center; as such, the intelligent monitoring and management of power typically yields significant ROI for best-in-class energy management technology.

About the Author

Jeffrey S. Klaus is the director of Data Center Manager (DCM) Solutions at Intel Corporation.

The Answer to the Question: Maybe

Although we might call one data center “energy efficient” and another “energy inefficient,” these are really just relative terms. No facility is 100% efficient; rather, efficiency falls into a range. Every data center design, however, still leaves room for efficiency improvements. And at each end of the spectrum (very inefficient or very efficient), one can easily imagine design alterations that are either very much worth the cost, or the opposite. For instance, putting in a little more effort to arrange server racks in a manner that optimizes airflow and creates warm and cold aisles will no doubt provide a fast return on a minimal investment. On the other hand, for an already highly efficient design, investing heavily in a system or configuration to gain a miniscule improvement in energy efficiency can easily be a waste of money—at least in terms of ever receiving a return comparable to the investment.

For most data centers (which fall between these extremes), the question of ROI on building for energy efficiency is much less clear. The closer a company comes to one extreme or the other tends to clarify the matter, but in the middle range, the question must be answered case by case. In other words, building for energy efficiency may be worth it, depending on the particular situation.

The Middle of the Road: Energy Efficiency Best Practices

The above qualifications notwithstanding, in some sense building for energy efficiency is always worth it—as long as the design doesn’t go to the extreme. For many companies, this middle ground involves building a facility that conforms to tried-and-true measures that other companies have used successfully to improve efficiency in a manner that provides a return on the investment. These measures are often called best practices, and if you follow trends in data center design, you’ll probably recognize many of them. Here are a few that are generally recognized as worthwhile investments to reduce energy consumption and thus provide ROI over the long term (and in some cases the short term).

• Hot aisle/cold aisle containment. Larger temperature differentials improve cooling efficiency, thus reducing power consumption. This measure requires little by way of infrastructure—it mostly requires careful planning of facility layout—and thus can provide a fast return on investment.

• Free cooling. Although free cooling is not entirely free, it does avoid much of the operational expense of mechanical cooling methods. Many data centers (depending on the temperature and humidity requirements of their IT equipment) can use free cooling most of the year, if not year round, yielding tremendous savings and a fast return on investment. Furthermore, facilities designed to rely heavily on free cooling also have reduced infrastructure needs for mechanical cooling, potentially balancing the any investment in greater efficiency.

• High-efficiency UPS systems. All power consumed by IT equipment will first go through some form of uninterruptible power supply. UPS inefficiency, therefore, can be a tremendous waste of energy (and thus money). Depending on the lifespan and maintenance requirements of a high-efficiency UPS, the investment in efficiency can thus quickly provide returns, especially if the efficiency improvement is more than a few percent.

• Efficient servers. Although higher-efficiency IT equipment doesn’t help your PUE (it can actually hurt it), it nonetheless improves overall efficiency and reduces energy consumption. The question in this case, however, is greater efficiency relative to what? Servers sporting processors manufactured in a cutting-edge 22nm process may be worth the investment relative to those with processors built in 90nm, but the choice is seldom so black and white. In between is a range, and one choice may be better than another. Given, however, that the lifetime operational (energy) costs of a server now exceed its capital cost, choosing servers with greater efficiency is typically going to be the best bet—although some careful thought may be required to maximize the returns.

• Virtualization. Virtualization software requires an investment, but virtualized infrastructure can often reduce the required hardware needed to achieve a certain capacity. It can thus decrease both operational costs in the long term, and it may balance capital cost adjustments from the start.

• Automated lighting systems. If lights are left on in unoccupied rooms, you’re losing money. An automated lighting system can eliminate much of this loss by automatically shutting off lights when they are not needed. Although the energy savings may not be as dramatic as those of some other efficiency measures, this step can still provide a return on the infrastructure investment.

• “Recycling” waste heat. A data center is more or less a big heater (although it does a little more than that). So why use energy to separately generate heat for nearby office spaces when the data center produces plenty? Depending on the situation, a data center may provide enough heat to eliminate any need for separate heat generation. The question is whether the necessary infrastructure is worth the investment—and that depends on a variety of factors.

• Computational fluid dynamics (CFD) analysis. CFD enables “previewing” the temperature and airflow dynamics of a data center design. This information can be used to eliminate hot spots and other anomalies, thus reducing the cooling needs of a facility and saving money. The catch is that CFD can be expensive, but the potential returns are significant.

Choosing the Right Course of Action

Building for energy efficiency is worth the investment, but you can also go overboard and end up wasting money. The key is choosing the right combination of energy efficiency design elements to maximize the return on the investment—and this combination will vary from data center to data center. Differences in the size of the facility, the climate, the purpose (for instance, a cloud data center versus a colocation facility) and the characteristics of the surrounding campus can all affect design choices. In effect, then, building for energy efficiency may or may not be worth the investment, but building wisely for energy efficiency is always worth the investment. Industry best practices for efficiency are an excellent place to start, and most data center designs will benefit from at least a subset of these measures—particularly those such as the use of free cooling, hot aisle/cold aisle containment and virtualization.

As energy prices rise, the threshold for a worthwhile investment in efficiency also becomes more reachable. But the benefits of energy efficiency go beyond just financial returns. An efficient design also gives a company public relations points, and overall industry movement in the direction of greater efficiency helps hold off government agencies, which invariably have the itch to impose new regulations (which are almost always costly). Furthermore, energy efficiency is simply a matter of corporate responsibility—particularly when it costs little to nothing (or even provides a return) in the long term.

Photo courtesy of Brooks Elliot

What Your Certification Means

Whether you’ve earned an Energy Star, LEED, or some other certification, all you really have is an independent statement that your data center was at one time up to par according to a set of standards. The very next day after receiving the certification—as far as anyone else knows—your facility may well have thrown all the energy efficiency measures out the window. Of course, when it comes to energy efficiency, that would make little business sense: lower energy consumption means lower operating costs. Furthermore, implementing energy-efficient practices often requires supporting infrastructure with its own capital costs, and not employing that infrastructure wastes the opportunity to recoup costs and even gain a return on the investment.

Maintaining Energy Efficiency Certifications

Recertification may or may not be required by your certifying organization, but as certifications age, they mean less both to you and to your customers. Hence, keeping your facility up to standards is important, both to keeping costs down and to maintaining a PR edge. The following are several tips to consider for maintaining your certification.

• Don’t make the certification your main goal. This may sound a little odd, but it’s an important point, psychologically. An energy efficiency certification is like kudos for a good job: your focus should be on doing a good job, not getting the kudos. If you focus on pursuing strong efficiency practices, getting certified will be that much easier. You may still need to take a few extra steps to meet guidelines of the certifying agency, but chances are you’ll already be most of the way there.

• Work toward the best efficiency you can achieve, not just the bare minimum for certification. This is an extension of the above point. The certification shouldn’t be your primary focus, nor should its minimum standards be all that you aim for. If you can improve efficiency in a cost-effective manner—even if it’s above and beyond the certification’s guidelines—pursue it by all means. In some cases, this approach may over time help you work toward a higher level of recognition, as in the case of the various LEED levels (Certified, Silver, Gold and Platinum).

• Keep track of changes in the certification standards. If you choose to (or are required to) recertify, you can avoid hassles by keeping an eye on the standard as changes are made. The conditions for certification may remain the same from year to year, or they may change in major or minor ways. In particular, for example, be on the lookout for changes to definitions: if the precise definition of data center space changes (as in the Energy Star program), then your task of achieving certification may become easier or harder, depending on your facility. Here, especially, is where aiming for better efficiency rather than just maintaining minimum standards is helpful—it helps you stay ahead of changes in certification standards, particularly if they become more stringent.

• Recertify as often as is practical. Your certification only says that your facility was up to standards at the time it was certified. The less recent the certification, the less meaning it has, especially if you’ve made major changes in the meantime. But certifications also cost money and don’t really do anything—of themselves—to improve productivity or efficiency, so some balance is required. Some standards may require periodic re-examination to maintain certification. Certainly you want to recertify within such windows to stay up to date. Either way, have a plan for preparation so that you’re not caught off guard when recertification time arrives.

• Make energy efficiency an ongoing project, not just a task for the couple months before recertification time. By creating an atmosphere in which energy efficiency is a perpetual concern for the data center, you will avoid the need to “cram” right before recertification. This is in many ways an extension of focusing on energy efficiency and not the certification for it, but it is important to note that the focus should be continual and not just at certain times. Of course, you have a business to run, and your sole concern isn’t just improving efficiency, but that doesn’t mean you should ignore efficiency for long stretches of time. Remember, the more you do to maintain and improve efficiency on a continual basis, the less you will need to do when recertification inspections loom.

• Stay informed about new energy efficiency technologies—and choose wisely among them. One way to maintain and improve efficiency is to know what new helps are out there. Of course, not all of them will be affordable, and many will likely be impractical, but by keeping informed, you can research and implement those measures that can provide a good return on your investment. Again, the more you do in the interim between recertifications to improve efficiency, the smoother the recertification process will be.

• Budget for energy efficiency. Fitting energy efficiency practices and investments into a budget that leaves no room for them is obviously difficult, particularly in a tight economy. So, as much as possible, make room in your budget for energy efficiency. Explain to the C-suite how better efficiency means lower operating costs and, eventually, a return on infrastructure investments. If energy efficiency is a priority in your data center (and it should be to at least some extent), then show it by investing appropriately in it.

• Be honest about the meaning of certifications. Remember that a certification is like a grade—it’s just one person or organization’s evaluation of your efforts. You can “cheat” on the test, and you can “work the system” to make things look better than they really are. Energy efficiency provides a variety of benefits, and the ability to achieve a certification is only one of them. Prioritize appropriately: aim for energy efficiency first, then for the certification (not vice versa).

Conclusions

What specific steps you need to take to maintain your energy efficiency certification depend on the standards you must meet and on the state of your data center. The tips above focus broadly on establishing the proper focus needed to make recertification a smoother and less stressful process. The main point is that if you prioritize energy efficiency in your data center operations, then recertification shouldn’t pose any hassles.

Author contact

Photo courtesy of Sam Beebe, Ecotrust

PUE and What It Means

Power usage effectiveness (PUE) is simply the ratio of total data center energy consumption to the amount of energy consumed by the IT equipment alone. Since the data center’s primary purpose is to supply IT resources to a company or consumers, an ideal PUE is 1.0: all energy consumed by the data center goes to the IT equipment. That would mean no energy is used or lost in the power distribution system (including cables, uninterruptible power supplies, voltage conversion stages and so on), security system, lighting, heating/cooling or any other non-IT portion of the data center. Obviously, then, a PUE of 1.0 is unattainable, since it requires 100% efficiency.

Of course, simply looking at the definition of PUE leaves some grey areas. What if a company has solar panels (or some other on-site power generation infrastructure) at its data center campus, for instance? Does that power count toward the total power consumed by the facility? Should peripheral power drains like lighting and security really be included in the PUE calculation? The Green Grid, the organization that developed the PUE metric, provides some guidance on how it is to be properly calculated, but unscrupulous companies can easily “cook the books” to make their facilities look more efficient than they really are.

Thus, PUE measurements should always be taken with a grain of salt, particularly when no details are provided regarding how they are calculated. If you ever see a PUE of exactly 1.0 or less, you can be certain you’re being lied to. But what about a PUE of 1.001? Believe it only if you think that only 0.1% of every watt delivered to the data center’s servers is lost in power distribution inefficiencies—to say nothing of lighting or anything else happening in the facility. In other words, don’t believe it. But for a company honestly seeking to improve its energy efficiency (not just its public image), PUE can be a helpful metric. Here’s some ways to manage it.

What You Don’t Measure Won’t Help You

To know your PUE, you must measure it. Unfortunately, looking at the power rating labels on equipment just isn’t enough—most equipment, including IT systems and cooling infrastructure, varies in its power consumption throughout the day and over the course of the year. The following are some considerations to aid in PUE measurement.

• Measure IT power as close to the source as possible. Unless you’re using zero-resistance cables (believe me, you’re not), then the farther you go from a server, the higher the power consumption reading for that server—and the lower your PUE. That may be good for marketing, but it’s not good for an honest PUE assessment. When you take measurements close to the IT equipment, you’re getting a more accurate reading of the IT power consumption. That may hurt your PUE calculation a little, but it gives you a more honest assessment of your data center’s efficiency.
• Don’t cut corners on total power consumption. Assuming you don’t do any on-site power generation, the best way to calculate your total power consumption is to just look at your energy bill. See what the power company says you’re consuming, and use that number. (Yes, I suppose the power company could be wrong, and it may not hurt to check once in a while, but this approach reduces the temptation to fudge the numbers.) Don’t cut out security, lighting and other needful systems—but if you must, make note of it if you report your PUE publicly. Of course, you’ll need to do some actual measurement of your own to calculate PUE on a finer-grain time scale, but the point is this: don’t pick and choose systems and then say “these don’t really count.”
• Measure often and regularly. The closer you can get to a continuous PUE measurement, the more information you will obtain to help you improve efficiency. PUE varies daily depending on workload, environmental conditions (outside temperature) and other factors. Furthermore, it also varies over the year, largely for similar reasons. To get the best sense of your data center’s efficiency, measure PUE as often as possible in a regular manner—don’t skip certain times of day or year (another way to fudge the numbers).

Improve Your PUE

Once you are able to measure your PUE, you can work on improving it. (Even if you don’t care about PUE, you can still improve your data center’s efficiency; but if you want numerical data to back you up, here’s some steps to take.)

• Hot/cold air isolation. One of the best things you can do in terms of maintaining good airflow in the data center is to isolate hot air from cold air. A larger temperature differential improves cooling efficiency. How you implement this isolation depends in large part on your cooling approach and infrastructure and your equipment layout.
• Raise the temperature. Many data centers operate their facilities at temperatures much lower than the maximum recommended by ASHRAE. By raising the thermostat, you automatically save cooling costs.
• Use free cooling when possible. Why use mechanical cooling when outside air is often cool enough? Free cooling (although not entirely free) can slash power consumption by cooling infrastructure tremendously—and thereby significantly lower PUE.
• Focus on power distribution. Using high-efficiency UPS systems and eliminating unnecessary voltage conversion stages reduces power loss. (And since this power loss becomes heat, it also reduces the cooling load, meaning even more savings.) After cooling, power distribution is the primary leech on your efficiency.
• Implement controlled lighting systems. Employees may not be able to function in the dark, but servers can. So don’t waste money and power on lighting when no one’s in the facility. Consider installing a lighting control system that shuts the lights off automatically at certain times or under certain conditions (such as when no one is in the facility).

Don’t Stop at PUE

PUE provides some indication of the efficiency of your data center, but it is not the end-all be-all of efficiency metrics. Imagine you went into your facility today and replaced all the servers with less efficient models: your PUE would improve! By the same token, installing more-efficient servers hurts your PUE. Don’t ignore IT efficiency just for the sake of PUE (you are billed by your utility according to how many watts you consume, not according to your PUE). You may even want to measure and seek to improve your data center according to other metrics as well—and numerous ones are out there. Yes, you want to minimize the amount of power your facility consumes in peripheral tasks (power distribution, cooling, lighting and so on), but you also want to minimize the amount of power your IT equipment consumes. In other words, the ideal facility should accomplish the maximum amount of work using the minimum amount of power. And numerous benchmarks are available to measure performance per watt—although these metrics also can be manipulated and must be used with care, just like PUE. So, PUE is helpful, but it isn’t the end of the story.

Conclusions

You can take steps to obtain useful and accurate measurements of your data center’s PUE, and once you do, you can seek to minimize it through a variety of steps. But don’t stop there—PUE is useful, but it doesn’t deal with all aspects of data center efficiency. IT equipment—the biggest energy consumer for data centers with a PUE below 2.0—can also always stand to see greater efficiency. Supplementing PUE with other metrics can help avoid a myopic view of energy efficiency, enabling companies to garner the maximum savings from their efficiency improvements.

Author contact

Photo courtesy of mastermaq

Apple’s Solar Plant

According to DatacenterDynamics (“Apple confirms solar farm at Maiden data center”), Apple’s Maiden campus will include a 100-acre solar array with a peak output of 20MW, producing about 42 million kWh per year. Doing a little math, the yearly output translates to an average output of a little less than 5MW. (If you double that to very roughly account for nighttime hours, the array will provide an average of between 9MW and 10MW during daylight hours.) According to the U.K.’s Register (“Apple slaps mega-solar panel field on new ENORMO data centre”), the Maiden data center will consume on the order of 50MW to 60MW; some wilder estimates even put it as high as 100MW. In either event, the solar array—as huge as it is—will only cover a maximum of about 10% of this consumption.

Nevertheless, even though the solar array accounts for a relatively small percentage of the facility’s expected power needs, it is still significant and will no doubt provide some insight into the feasibility of the sun as a practical large-scale energy source. Apple’s addition of the solar array in North Carolina no doubt scores it some marketing points by enabling it to boast of concern for the environment—and there’s no reason to think the company wants to “destroy the planet.” But ultimately, making money is what drives business, so Apple is no doubt exploring the potential of solar power as a means of lowering its energy costs. After all, sunlight is free. The infrastructure has an associated cost, as well as maintenance expenses, but the investment offers significant potential returns in the long term.

An important question remains, however: does solar energy really have the potential to meet the data center industry’s (or world’s) power needs?

Solar Power’s Potential

Let’s do some more math. Electromagnetic solar radiation (all frequencies, including visible light) constitutes about 1,350 watts per square meter—the solar constant. This number simply means that if you held up a solar panel that is one square meter in area, you could theoretically get 1350 watts of power—enough to power about thirteen 100-watt light bulbs. But this number is the solar incidence just outside the atmosphere; what’s received at ground level is less, owing to losses as the radiation propagates through air (these losses include scattering, such as off clouds, as well as absorption/heating). Atmospheric factors affecting the solar constant are manifold, but let’s assume about a 25% average loss, bringing the potential power to about 1,000 watts per square meter. To account for nighttime hours, cut that power number in half (and that’s being generous): 500 watts per square meter. Now, assume a solar cell can turn half of what it receives into usable power: we’re now down to 250 watts per square meter. Thus, a solar panel measuring one meter by one meter (about 10 square feet) would—assuming 12 hours of full daylight and a system to ensure the panel is always directed toward the sun—provide enough power to on average run two or three light bulbs continuously. Not bad, but not great.

Using this math, a 50MW data center would require roughly 50 acres of solar panels. Not 50 acres of land—the required land to support that much solar panel area would need to be much more, particularly if it is flat. Calculating numbers for the case of trying to tackle a large portion of the overall power consumption of the entire world is left as an exercise for the reader.

Solar Power: Not as Ideal as It Seems

Aside from the infrastructure aspects (such as manufacturing and maintaining all the solar panels and power distribution equipment to collect and supply all this power), there is the question of whether the disturbance of large portions of land, and the creatures that inhabit it, to create mega solar farms is warranted.

A blog article by James Hamilton offers a similar perspective. Hamilton notes that according to at least one model, Apple’s 20MW array “has an average output of 15.8% which yields 3.2MW.” So, needless to say, complete reliance on solar power would consume much more land area (again, Apple’s solar array will consume about 100 acres); as such, this approach isn’t practical for facilities located in  urban areas or where weather conditions hamper efficiency. “I’m personally not crazy about clearing 171 acres in order to supply only 4% of the power at this facility. There are many ways to radically reduce aggregate data center environmental impact without as much land consumption,” Hamilton said in his blog.

For those concerned about carbon emissions, note that clearing land for a solar array also involves destruction of trees, which convert carbon dioxide into oxygen. The energy produced by the solar plant may outweigh the emissions of a coal plant for a similar amount of energy, but an exchange is made: solar power is not purely beneficial in this case.

Some market observers cite falling prices of solar infrastructure as an indication that it will become a more widespread and economically feasible means of supplying power. A GigaOM article (“5 reasons why Apple is embracing clean power for its data center”) notes that “Prices of solar panels and cells have plummeted recently… That’s bad news for the solar manufacturers — and has led to a wave of solar maker bankruptcies — but that’s good news for companies, utilities and home owners that are buying solar panels.”

The problem, however, is that if prices are so low that companies cannot sustain their manufacturing operations, then prices are being manipulated. Look no further than Solyndra for evidence—a highly subsidized company (to the tune of more than $500 million) went under, bringing to light the U.S. federal government’s manipulation of the solar industry. Hamilton’s blog article hits this point head on: “Looking more deeply at the Solar Array at Apple Maiden, the panels are built by SunPower.  Sunpower is reportedly carrying $820m in debt and has received a $1.2B federal government loan guarantee. The panels are built on taxpayer guarantees and installed using tax payer funded tax incentives.”

Given the government’s current fiscal status—as well as its obvious penchant for manipulating markets with floods of money (from taxes or the printing press)—the economic feasibility of solar power cannot be judged on the basis of recent cost trends.

Conclusions

So, is Apple just pursuing the best interests of the environment, or is it pursuing its own financial interests? Probably a mix of both, and that’s perfectly understandable. The company’s solar farm at its Maiden data center site, however, will likely turn out to be more of a marketing gimmick than a true proof of concept for workable large-scale solar power.  Reliance on the sun for energy will likely expand, but it seems most beneficial for small-scale supplementation (e.g., residential use) rather than a large-scale energy solution. For data centers—and, in particular, those in urban areas—solar power (especially private solar farms) is practically a non-starter as a significant source of energy.

Author contact

Photo courtesy of h080

]]> http://www.datacenterjournal.com/facilities/apples-solar-farm-stewardship-or-marketing/feed/ 7 http://www.datacenterjournal.com/facilities/computational-fluid-dynamics-cfd-for-data-centers/ http://www.datacenterjournal.com/facilities/computational-fluid-dynamics-cfd-for-data-centers/#comments Tue, 20 Mar 2012 14:59:41 +0000 Jeff Clark http://www.datacenterjournal.com/?p=4697 Data centers must be cooled, but designing a new facility or changing an existing one to maximize cooling efficiency can be a mammoth task. Any number of different design strategy or floor layout variations can affect the results, thereby changing efficiency, creating hotspots or altering the amount of infrastructure required for the design. Computational fluid dynamics (CFD) offers a method of evaluating new designs or alterations to existing designs before they are implemented. As with any simulation tool, however, maximizing the benefits requires a careful balance of budget, effort and understanding of the process.

What Is CFD?

Air, like water, is a fluid that moves in response to temperature and pressure gradients. Obstructions of various types (walls, floors and ceilings, server racks and so on) change the way the air flows, and as the number and complexity of these obstructions increases, so does the complexity of the analysis of how air flows. Add to that heat sinks (air conditioners, for instance) and heat sources (servers) and you have a very complicated problem that you can’t analyze using pencil and paper.

If your data center relies mainly on air to cool IT equipment, you’ve probably struggled with “hot spots” and other air flow problems as you try to ensure that your servers don’t overheat. In an existing data center, identifying regions that are insufficiently cooled is as simple as installing temperature sensors and monitoring the data. But what if you want to know what cooling problems might crop up in a new design or a reconfiguration of an existing data center? That’s where CFD comes in.

CFD takes a computer model of a data center and, using numerical techniques, calculates a steady-state airflow “map” of the facility, revealing locations that are insufficiently cooled or that are contributing to cooling inefficiency. Thus, CFD provides insights into challenges that might be posed by a new data center design, a rearrangement of a facility’s layout or another major change.

So, When Do We Start?

On its face, CFD sounds like an indubitably helpful tool, but as you might expect, there are qualifications. Obtaining a CFD analysis—whether on your own computers using purchased software or by way of an analysis firm—costs money, and the potential benefits should not be outweighed by the investment. Here are some essential considerations if you’re looking into CFD as a method of refining your data center design.

• Outsourcing or do-it-yourself—The first thing you must decide is whether you want to tackle the analysis yourself or hire a professional consultancy to do it for you. CFD involves a steep learning curve that may cost you heavily in terms of both time and results. Numerical techniques are fraught with pitfalls that can greatly affect the accuracy of results, and unless you plan to use CFD extensively (i.e., more than just once), you are likely better served by hiring a professional to do the work for you.
• Budget—Depending on whether you’ve decided on a consultancy or an in-house analysis, your costs will vary. Obviously, a consultancy will charge you for its expertise as well as to amortize its own costs (software, computers and so on), and you receive only results for a single design. (Some consultancies will work with you to some extent to optimize the design or to otherwise perform several analyses, but ultimately, you are paying to get just one set of results.) For an in-house analysis, the obvious cost is the software, which can vary depending on number of licenses and so on. Furthermore, you can either buy the software outright or, in some cases, use a cloud-based “pay-as-you-go” model. Less obvious costs are computer usage and employee time—the learning curve has its own price!
• Return on investment—CFD is not a solution for everybody. If you just want to find hot spots in your existing data center, you don’t need CFD, you need data center monitoring equipment. CFD, like any computational (and, particularly, numerical) technique, can suffer from any number of inaccuracies, and it is best used as a predictive tool when actual measurements are impractical. Be sure that investing in a CFD analysis—whether in-house or through a consultancy—has enough potential returns to justify the costs.

Some Pitfalls of CFD

If you’re convinced CFD is right for your data center project, you must be aware of several pitfalls that can crop up in a CFD analysis. First, you’re dealing with a numerical technique, not an analytical one (i.e., CFD doesn’t find the “equation of your data center”—it breaks your data center into tiny pieces and analyzes each piece in light of the pieces around it). Furthermore, the accuracy of the results is limited by the accuracy of the model you supply: every unfilled cable hole or obstruction in an aisle can affect airflow, so the more details you provide, the better your chances of receiving helpful results.

Second, if you’re using software on your own, you shouldn’t just trust everything the software tells you. One of your most important tasks is to gain some insight into how the algorithm calculates results, as this will help you interpret the analysis correctly. (Avoiding such “extra work” is one of the advantages of hiring a professional CFD firm.) Details of your model that seem perfectly reasonable to you could cause the software to return strange results, simply because of quirks in the algorithm.

Third, CFD is attempting to solve an extremely complex problem in a manner that simply cannot take into account every variable, so regardless of whether you perform the analysis in house or hire a consultancy, you should take the results with a grain of salt. This is not to say that CFD analysis is inaccurate, but not every detail of the analysis will correspond with the reality of your newly built facility.

Warnings Aside

As data centers consume more energy (which is converted to heat that must be removed from the facility) and energy prices continue to rise, the potential return on a CFD analysis increases. Long-term energy savings can greatly outweigh the costs of the analysis, whether through a consultancy or via the in-house route. For instance, if you are able to identify potential hot spots and take steps to eliminate them in your design, you can reduce the required cooling infrastructure, saving capital cost. Furthermore, your facility can operate at a higher temperature, saving operating costs. And when your data center operates at a higher temperature, you increase your opportunities for free cooling, which bypass traditional mechanical cooling methods and save even more cost. The key to successfully using CFD is understanding the purpose and limitations of the technology as well as your own company’s needs.

Author contact

Photo courtesy of Rob Bulmahn

One approach here is to take a more modular approach to data center design. The initial adopters of this strategy five years ago were Amazon and Google, and they are usually the reference for data center best practices in current deployment methodology. For companies evaluating where modular data center assets can help their own strategies, there are both lessons to be learned and pitfalls to avoid.

The growth of cloud computing is turning traditional thinking about data center design on its head. When looking at a data center refresh project, the first step is to consolidate and take this opportunity to deploy new IT equipment. Today’s Intel-based servers require much less the electricity for the same amount of CPU power compared with older servers. Making an investment in new hardware will therefore reduce your equipment power bill and cut your server cooling costs. Virtualizing these servers at the same time will remove up to 60 percent of the physical machines in the data center, too, with the equivalent power and cooling cost reduction.

Alongside these steps, there are also opportunities to reduce the power consumed at the network level as well. Investing in new networking equipment to take advantage of Fibre Channel over Ethernet can help consolidate your LAN and SAN. This will reduce 80 percent of your server cabling costs and cut your support costs in half with one infrastructure.

How you choose to deploy this equipment matters, and this is where some cloud and virtualization deployments fail to realize their full potential around energy efficiency. If you carry out deployments in the traditional way, you will have 68 percent of the cold air bypassing the IT equipment, according to research by ASHRAE. This means that you are unnecessarily paying double for your cooling to achieve the needed result.

Taking a modular design approach can solve this problem. Preconfigured cabinets that are implemented and approved by the manufacturers themselves are designed to have the best possible PUE and need minimal maintenance. Products in this category cover everything from a single cabinet preconfigured for a specific switch or server deployment; preconfigured cabling from simple patch cables through to complete end-to-end data center patching; and complete preconfigured switch, server and storage assets loaded with a virtualization platform. Examples here range from a single cabinet such as the NetApp FlexPod up to nine-cabinet 6000 Virtual Machine stacks from VCE; these can even be loaded into shipping containers for full “data center in a box” installations.

Managers should therefore look for the appropriate-size preconfigured solution that best suits their requirements, which means filling cabinets and packing as much density as you can into the available space. This approach follows the hot aisle/cold aisle principles but also involves using containment. You are able to add doors and a roof to either the hot aisle for “hot aisle containment” or the cold aisle for “cold aisle containment.” You can also use chimney doors to create a vertical exhaust system to contain the hot air. It doesn’t matter which containment you use, as independent testing of all three solutions has been carried out for the same IT load. When properly configured, they all increase efficiency by 23 percent, leading to reduced cooling expenditures.

To get the best out of virtualization you will need to provision for spare capacity. If you have reduced 200 servers down to 20, having some spare capacity deployed will allow for dynamic provisioning. This will also help the data center team plan ahead sufficiently, but without needing to add more resources that require power and cooling but do not deliver business results.

A Cooler Data Center Than You Think?

Cloud computing implementations are being sold on their flexibility and ability to scale up and down rapidly according to business demand. But the reality from the physical data center side is that power and cooling resources will be consumed whether the equipment is working hard or not.

The solution to this paradox is to use cooling that is closely coupled with the IT equipment deployment. By using inline cooling systems from the likes of Stulz, these units are deployed in the cabinet rows and can mix with any of the containment regimes that might be deployed.

The biggest variable here is due to virtualization. The virtual machine manager that the customer has in place will move the virtual machine load to the most appropriate resource in real time according to business rules and demand for services by other virtual machines. The inline cooling units will monitor the changing load and will adjust the airflow to suit. Similarly, the network equipment can respond to computing load in the same way using the IEEE 802.3AZ Power Down While Idle standard. This will vary the switches’ power requirements up and down in line with IT demand, reducing the overall power requirements for the switches themselves.

Previous cooling systems rely on fixed torque motors to run the fans for cooling, while the Stulz units use variable torque motors. The significance of this difference is that in the old systems, reducing the load by 20 percent drops the power consumption by the same amount—20 percent. With variable torque motors, the fan speed can be varied. Consequently, if the IT load requirement drops by 20 percent, the power requirement for the cooling system drops by 50 percent.

Making the Move to Cloud—A Physical IT Perspective

To make the move to deploying cloud within an existing data center, the first step should be to create some space, and then use modular deployments to create a virtualized environment that will support any private cloud installation. The business can then install and run its application environments in parallel while any testing and configuration work is carried out.

This is a good test to see if support staff can maintain the new way of delivering services. Virtualization products are tried and tested, and had been commercially running for over five years, so if you are happy with the installation, you are now able to turn off the old systems and run completely on the new module. Once running on the new module, you can decommission the old application servers and create more space in the data center.

The Uptime Institute found from a survey of its user database that if your business is growing at seven percent per annum, your database will have doubled its switching, server and storage capacity within 10 years. If your business is growing at 10 per cent per annum, however, this doubling capacity will have occurred in seven years. This information on business growth can therefore help with long-term data center planning.

Data center managers have been realizing that they must move from the traditional IT delivery model to keep up with the pace of change within their organizations. Running behind the curve because they have previously routinely built bespoke and piecemeal solutions deployed in 90 days is not seen as good enough anymore. The new breed of data center managers are anticipating business requirements and building up their skills around mapping these needs to service-level agreement frameworks and management delivery options. IT staff can no longer compete with the external providers if they remain too specialized; it is better to be a generalist and be able to manage the delivery of services using modular and preconfigured data center units that can be deployed ahead of business demand, rather than to meet it.

About the Author

David Palmer-Stevens is Systems Integrator Manager EMEA at Panduit.

Photo courtesy of walkinguphills

Why Water?

One of the main draws of liquid cooling is the greater ability of liquids (relative to air) to capture and hold heat—it takes more heat to warm a certain amount of water to a given temperature than to warm the same amount of air to the same temperature. Thus, a smaller amount of water can accomplish the same heat capture and removal as a relatively large amount of air, enabling a targeted cooling strategy (the entire data center need not be flooded to keep it sufficiently cool). And with the rising cost of energy and the growing power appetite of data centers, the greater energy efficiency of water is a temptation to companies.

For high-density implementations, air cooling is often simply insufficient—particularly when a whole-room cooling approach is used. In such cases, water (or, more generally, liquid) cooling offers an alternative in which not only can greater cooling efficiency be applied, it can be applied only where it is needed. That is, why cool an entire data center room when you can just cool, say, a particular high-density cabinet? And when individual cabinets are kept cool, they can also be placed much closer together, since air flow is less of a concern. Thus, liquid cooling can also enable more-efficient use of precious floor space.

Of course, no solution is ideal. Liquid cooling has its drawbacks, just as air does, and they’re worth noting. But it’s helpful to first review some liquid cooling solutions that are now in use in data centers.

Liquid Cooling Solutions

Liquids can serve as a medium for transporting heat in a number of different ways, ranging  from broader cooling of the entire computer room to targeted cooling of particular racks/cabinets or even particular servers. The most basic option is to use water just as the means of moving heat from the computer room to the outside environment. In a whole-room approach to cooling, computer room air handlers (CRAHs) use chilled water to provide the necessary cooling; the water then moves the collected heat through pipes out of the building, where it is released to the environment (in part through evaporation, which is helpful but also can consume large amounts of water). This cooling approach is similar to the use of computer room air conditioners (CRACs).

A more targeted approach involves supplying cool water to the rack or cabinet. In the case of enclosed cabinets, only the space surrounding the servers need be cooled—the remainder of the room is largely unexposed to the heat produced by the IT equipment. This approach is similar to the whole-room case using CRAHs, except that only small spaces (the interiors of the cabinets) are cooled.

Cooling can be targeted even more directly by integrating the liquid system into the servers themselves. For instance, Asetek provides a system in which the CPU is cooled by a self-contained liquid apparatus inside the server. Essentially, it’s a miniature version of the larger data center cooling system, except the interior of the server is the “inside” and the exterior is the “environment.” Of course, the heat must still be removed from the rack or cabinet—which can be handled by any of the above systems.

At the extreme end of liquid cooling are submersion-based systems, where servers are actually immersed in a dielectric (non-conducting) liquid. For example, Green Revolution Cooling offers an immersion system, CarnotJet, that uses refined mineral oil as the coolant. Servers are placed directly in the liquid, yielding the benefits of water cooling without the hassles of containment, since mineral oil is non-conducting and won’t short out electronics. Using this system requires several fairly simple modifications, including use of a special coating on hard drives (since they cannot function in liquid) and removal of fans.

Going a step further, some solutions even use liquid inside the server only, avoiding the need for vats of liquid. In either case, however, the heat must still be removed from the server or liquid vat.

Now for the Downsides

Depending on the particular implementation, liquid cooling obviously poses some risks and other downsides. Air is convenient, partially because of its ubiquity; no one worries about an air leak. A water leak, on the other hand—particularly in a data center—is a potential disaster. Moving water requires its own infrastructure, and although moving air does as well, a leak from an air duct is much less problematic than a leak from a water pipe. Furthermore, water pipes can produce condensation, which can be a problem even in a system with no leaks. And with more-stringent requirements on infrastructure comes greater cost: infrastructure for liquid cooling requires greater capital expenses compared with air-based systems.

Another concern is water consumption. Evaporative cooling converts liquid water to water vapor, meaning the data center must have a steady supply. Furthermore, blowdown (liquid water released to the environment) can be problematic—not because it is polluted (it shouldn’t be), but simply because it’s warm. When warm water drains into a river or stream, for instance, it can damage the existing ecosystem.

Proponents of liquid cooling approaches cite the energy efficiency improvements that their systems can provide—estimates range as high as a 50% cut in total data center energy consumption. These numbers, of course, depend on the situation (where the particular data center starts and what type of system it installs), but the returns only begin after the infrastructure has been paid off. Also, given the greater infrastructure needs of liquid cooling, maintenance may be more of a concern. Furthermore, in cases where water is consumed by the cooling process, some energy efficiency comes at the cost of a high water bill.

Liquid Cooling to Stay

In some cases, particularly in lower-density data centers, air cooling may be the best option, if for no other reason than it is simpler and lower in cost to implement. Questions linger regarding at what point liquid cooling becomes financially beneficial (“Data Center Myth Disproved—Water Cooling Cost-effective Below 6kW/rack”). For high-density configurations, however, liquid may be the only viable option, and as high-power computing grows, so will an emphasis on liquid cooling. Air cooling simply has too many benefits—many of which center on its simplicity—to expect that liquid cooling will one day be the only cooling approach. Nevertheless, liquid/water cooling has an established position in the data center (particularly the high-density data center) that will grow over time. The only question is how much of the market will implement some form of liquid cooling, and which types of liquid cooling solutions will be the most prevalent.

Author contact

Photo courtesy of Jayashreee

The data center is at the heart of the enterprise—a complex, interconnected array of equipment, software and data that drives the business and powers every aspect of the operations. Yet change is inevitable, and as the business grows, companies may discover mission-critical reasons why such a move makes sense. Some of the reasons companies decide to relocate a data center include the following.

Business reasons—Businesses are increasingly dependent on their IT infrastructures to drive mission-critical operations throughout the enterprise, from high-volume financial transactions to web-based, global commerce to always-on services. For many businesses, that means significant cost savings can be realized through data center consolidation and right-sizing. For others, a merger or acquisition drives the need for broad-scale integration, and a spike in the demand for data storage. Increasing regulatory requirements also play a significant role, as companies struggle to meet compliance, archiving, data management and security requirements. And every company needs a business continuity plan that includes disaster recovery, backup and remote operations. For some companies, the reasons for data center migration are simple: market success has resulted in explosive growth that has rendered the current facility obsolete.

Technology reasons—The continual evolution of infrastructure technology is transforming the modern data center. Acres of individual servers and miles of cable are being replaced by high-density clusters of rack-based equipment, reducing the need for physical space. Advances in network bandwidth are making it possible to store and access mission-critical applications and data remotely. At the same time, more processes, documents and data are being digitized, increasing the need for more-advanced data center configurations. Regardless of what drives data center migration, the goal is to minimize the downtime of business-critical applications and systems while making the move.

Data center migration projects typically involve a broad spectrum of internal and external stakeholders. Each stakeholder views the data center migration project uniquely, on the basis of his or her charter. Typically, the CFO views it as a cost, while the CIO considers it a business challenge. The data center manager perceives it as a logistical nightmare, the systems administrator views it as a technical challenge, and the business units see it as a potential outage and a threat to the revenue stream.

Most importantly, it represents a potential inconvenience to customers, and perhaps even a reason for them to go elsewhere. It is therefore important and necessary to address each key stakeholder’s concerns and work with them to design and implement a plan with the least impact on customers.

Most organizations make significant investments in their new data center facilities, leading to a state-of-the-art physical plant. High security and redundancy of the facility and utilities are common. A frequent oversight, however, is carrying over poor processes, procedures, architecture and documentation into the new site. To achieve the desired availability of applications and data, the maturity level of the IT infrastructure and processes must meet or exceed the design criteria of the facility.

Determine Scope Through Organizational Readiness Analysis

To understand the scope of preparations and investment required for a smooth migration, an organization must first evaluate its readiness to undertake the initiative. The maturity of an organization’s IT infrastructure processes, procedures and documentation has a direct correlation to the complexity of the undertaking, and the level of complexity is a major factor in an initiative’s cost and risk to the business.

Organizations with well-documented, active asset management, disaster recovery, monitoring and management, and change control programs have the essential elements required to successfully undertake a data center migration. They will not have to invest in the discovery, validation or development of information and processes in order to prepare.

Conversely, gaps in these processes and documentation must be addressed before or in conjunction with the project. Failure to address gaps will introduce a high degree of risk to the project and could lead to outages that harm the business.

Steps to a Successful Data Center Migration

Carry Out a Readiness Assessment

Performing a best-practices checkup for infrastructure management provides a baseline of the organization’s readiness to undertake this initiative. The objective is to evaluate the accuracy and completeness of processes, procedures and documentation. Focus areas include the following:

Support structure—Are problem management, notification and escalation processes current and documented?

Service-level agreements—Do they exist? Are they documented? Are they current?

Documentation—Do the five basic documents (configuration, startup, shutdown, backup and recovery) exist for each asset? Is there a central repository? Is there a document control system? Is the documentation current?

Asset management—Does a current system exist that reflects all assets and related portfolio information?

Maintenance contracts—Are these consolidated into a single data source, preferably the asset management system? Do the maintenance contracts reflect service levels proportionate to criticality and usage of the assets? Are contract expirations proactively managed?

Financial management—Does all information related to environment lifecycle costs exist in a central repository (asset management system)? Does a total cost of ownership (TCO) model exist for each asset?

Change control—Is there an actively managed process that tracks and audits all changes to the environment, including facilities, hardware, software, applications and data structures?

Architecture—Is the IT architecture well defined and documented? Is the architecture team involved in the design and validation of initiatives?

Capacity planning—Does an automated system exist to track the usage baseline and deltas in the environment at a component level?

Performance management—Does an automated system exist to track the baseline and deltas of the environment’s performance to a component level?

Monitoring and management—Does an automated system exist to track the availability and service levels of the IT environment? Are support and escalation procedures automated and current?

Business initiatives—Is there an overall perspective on the parallel initiatives that will be undertaken by IT and the business during the life of the data center migration project? Are the impacts and resource requirements understood and documented?

Stakeholder management—Have the basic requirements and value proposition for the data center migration project been communicated to the business and internal/external partners? Has a communication plan been established and implemented?

Resource availability—Is there a commitment of resources from each of the stakeholder groups in direct relation to the project timeline?

Industry regulations—Are the compliance ramifications of the project understood and overseen by a certified organization?

Logistics—Have the decisions related to the location of the destination facility been made? Is there a strategy for the location of assets by class and by facility?

Migration project—Has the project executive defined the basic initiative timeline? Is there a dedicated project manager? Does a corporate project management office (PMO) exist, and has this initiative been registered with the PMO?

Disaster recovery plans—Do current validated plans exist for each environment? Because a data center migration is essentially a managed disaster-recovery event for which the IT environment will be re-established at a different location, disaster recovery is the most pertinent area to the success of the project. A thorough disaster recovery plan provides key information about the interrelationships between the infrastructure and the business, the criticality of applications and data, and the mechanisms to mitigate risk.

On the basis of the project timeline, a determination must be made for each gap area on whether to implement a long-term or interim solution.

Assess the Environment

This phase of the project involves gathering, combining and correlating information about assets and their use in support of the business. Analogous to a disaster recovery plan, this step baselines the environment and begins the process of asset classification. Each asset must be identified, and the portfolio of information regarding its use and interrelationship to the whole environment must be established and documented. The output of this phase is the asset repository that reflects the current inventory, technical and business interrelationships, and supporting asset life cycle information. Best practices include automated asset discovery and tracking, and the use of an industry standard repository such as a configuration management database (CMDB) that is capable of providing a comprehensive view of all aspects of each asset.

Design, Validate and Plan the Project

Building on the assessment, each asset must be correlated to the business function it supports. This step parallels the disaster recovery process of defining recovery groups; for the sake of this project, these groups will be referred to as “move groups.” Each move group represents a consolidated collection of assets that support a key business function or IT support function.

Each move group is analyzed for its criticality to the business and assigned a corresponding ranking. The disaster recovery plan for each move group is consulted, along with the technical architecture employed for availability and recovery. The result is a migration methodology tailored for each move group according to the service level agreement, risk mitigation capabilities that currently exist and an approved business case for additional investment required to support availability or limit risk during the migration.

The output of this project phase will be an overall project plan that includes detailed task plans, time budgets, and resource and contingency plans. A migration calendar should detail the timing of move events in relation to business initiatives and cycles. A communication plan and command center structure should be documented and validated with all stakeholders.

Implement the Plan

This phase is where the detailed analysis and planning pays off. Each stakeholder should understand his or her role and tasks. Decisions regarding contingencies and timelines have been established. The command center coordinates the activities, tracks and communicates progress, and performs problem management and escalation coordination. Successes and failures are documented and used after the migration to improve the process for subsequent events.

Manage the Environment Post-Migration

On completion of the data center migration, it is imperative to take one additional step: the incorporation of knowledge, updated processes, procedures and documentation into the normal support structure of the IT infrastructure. The migration project will have validated or generated current information about the IT infrastructure. As change is constant in information technology, this information will have a limited shelf life. In the normal course of business, these processes, procedures and documentation all too often become a low priority compared with the demands of the business on IT organizations. Quickly incorporating this information and implementing a process to continually refresh it will achieve a far greater long-term result than solely the migration of assets.

The Benefits of a Successful Data Center Migration

The benefits of carefully planned and executed data center migration go well beyond what meets the eye of the user or customer. Done correctly, the end result is not only a seamless transition for the business, but also the creation of a set of business continuity disciplines that can validate or provide groundwork for disaster recovery and business continuity planning—as well as IT and physical security, asset management, systems documentation, change control, operating standards and processes, capacity planning, maintenance and license management, service and operating level agreements, business alignment, and data center facility management.

In other words, successful data center migration can completely transform the overall operating environment—its processes, procedures, documentation and personnel—in a way that has significant, lasting benefits for an organization’s disaster recovery readiness as well as day-to-day operational efficiencies.

Data Center Migration Checklist

Stage 1: Planning and Design

• Define the scope and size of the project
• Plan, agree on and allocate resources and budget
• Agree on key determining factors, limitations (network, security, etc.)
• Undertake a risk analysis and complete due diligence activities
• Create project plan identifying the critical path and key resources
• Complete inventories of existing systems and interdependencies
• Identify future system requirements (pipeline for growth)
• Create a step-by-step decommissioning and equipment rebuilding plan, including health and safety procedures
• Determine interim equipment requirements to keep systems operational during the migration
• Devise a contingency plan to include illness, accidents and damage to equipment
• Define connectivity requirements and allocate adequate time for new connections
• LAN design/WAN detailed diagram
• Consider your company’s operational requirements and plan the move to cause minimal disruption
• Undertake a “dry run” to test if the plan is achievable

Stage 2: Pre-Migration

• Check for readiness of racks, power circuits, cage and biometric reader
• Ensure back-up systems are in place and operational
• Data backup (before migration)
• Check that transportation and buildings can cope with equipment size and weight
• Check for common surveillance camera covering new customer cages

Stage 3: Migration

• Review deployment teams tasks and timelines
• Plan new space configuration and reference racks grid
• Install and test new data cabling (certification to be issued)
• Ensure all identified processes and procedures are followed
• Checking on connectivity between hardware equipment
• Test network and application components
• Arrange for M&E engineers, hardware and software experts to be on call
• Advise users of changes, and provide contact points for issues

Stage 4: Post-Migration

• Post implementation review
• Rebalance air-flow systems
• Check electricity quality
• Closely observe all building and equipment monitoring systems
• Test security systems
• Detailed diagrams rack/network/ patching/LL connectivity
• Infrastructure deployment complete signoff from the customer

About the Author

Jayabalan Subramanian is the Chief Technology Officer and co-founder of Netmagic Solutions Pvt. Ltd. He is recognized in the Industry as an expert on Internetworking and has consulted with leading organizations including BAFL, BPL, Bharti, Worldtel and Hathway Internet, among others.

Photo courtesy of s_w_ellis.