The growing demand for Internet services, the social-media explosion and constant collection of “big data” has put increased pressure on data center growth. Data centers must guarantee uninterrupted service and control costs when upgrading capabilities for higher bandwidth and increased storage capacity.
Major Internet companies including Google and Amazon have been architecting and designing their own data center equipment. They develop custom networking and software solutions that are specific to their needs. Other companies including Netflix and Shutterfly use a different approach, employing semicustom storage hardware. As demand accelerates for storage and bandwidth, these “hyperscale” data centers are at the forefront of adopting very different business models for how they grow. These changes include working with new companies in untraditional ways to meet the complex management and logistics of upgrading their large-scale rack-based systems. Hyperscale does not necessarily mean very large data centers, but the architecture that enables rapid scaling or expansion without changing the hardware components and basic infrastructure of the network.
Expanding an existing data center infrastructure is a different challenge compared with architecting and building a center from the ground up. An existing data center needs to be operational 24/7, and hardware expansion needs to take place seamlessly without interrupting services. To deal with the challenge of adding capacity, some of the more advanced data center architectures are based on a “modular” approach. The advantage is the ability to scale, in real time, without changing storage or storage-server elements. Using this approach, expanding the bandwidth or storage capacity of an existing infrastructure is driven more by operational, supply-chain, test-engineering and logistics challenges than by design or architectural decisions. Nevertheless, as the following case example shows, increasing data center capacity in real time has its own complex problems that require operational precision to maintain uninterrupted 24/7 service of the existing infrastructure.
Case Example: Upgrading Data Center Capacity
Upgrading a company’s data center capacity required the seamless installation of large-scale rack systems, with 24/7 uninterrupted operation. The additional hardware was not new or a custom design. The expansion would be based on existing hardware, with software images developed by the company that owned and operated the data center. That company chose a partner that was more traditionally known as an electronics contract manufacturer. This contract manufacturer, Sanmina, had developed considerable experience with storage and storage-server integration and system design. Part of the benefit to the data center operator was significant cost savings, as well as access to a large contract manufacturer’s footprint of facilities in Europe and Asia.
The system build and installation had to ensure the continuous operation of the company’s existing data center infrastructure, often within a short installation window. The company needed to add several hundred new data-center racks each quarter. The project required precise and effective planning along with a flawless execution—every quarter.
This program was executed in four major steps:
- Develop a modular and highly scalable test system
- Implement an efficient manufacturing process
- Create a custom supply-chain-management process
- Design a flexible logistics system
Modular and Highly Scalable Test System
Developing a modular and highly scalable test system in a facility where a large number of racks can be tested in parallel was essential. This effort was possible because the contract manufacturer had a large production facility with extensive test expertise. Custom software image generation and Nagios management software was used to automate the process. Furthermore, consistent communication with the data center company provided testing status in real time, enhancing product deployment. The result was faster project startup and higher confidence in the test and burn-in process.
Efficient Manufacturing Process
Sanmina used its experience with BTO/CTO (build and configure to order) of complex computing and telecommunications hardware to optimize tightly scheduled quarterly production of the rack systems. An efficient power and air-conditioning system was designed to accommodate the peak demands of 72 racks powered at the same time. This approach enabled the parallel testing of all racks, reducing overall system test time. A process for the configuration and placement of each rack in the system was developed. It specified their exact placement in the data center and enabled an efficient and timely installation. Furthermore, suppliers were on site during manufacture and test at the integration facility to enable immediate replacement of failing components.
Custom Supply-Chain-Management Process
To manage the short time frame for assembly, test and delivery, a custom supply-chain solution was implemented. It included flexible tools to manage the program across three different time zones. Detailed tracking of component delivery was implemented to ensure on time arrival. Daily supply-chain status updates were monitored to identify potential delivery issues. An effective process of forecasting replacement parts enabled them to be purchased at the same time as components for new builds. This approach accommodated very short server-component life cycles, eliminated the risk of parts becoming obsolete and minimized potential excess material exposure for the data center company.
Flexible Logistics System
A logistics system was designed specifically for this data center customer. It included the capability of transporting racked systems through different climate extremes at various times of the year. A careful selection of freight providers was made to ensure air-ride suspension and climate control. Doing so was essential to eliminating the risk of failure owing to extremes of temperature and humidity. A detailed labeling system for each crate was put in place. This process helped designate specific racks for delivery within a four-hour window onto the data center floor. The restriction of the four-hour window was to ensure uninterrupted service of the data center’s redundant systems. Reusable packaging was designed and tested for the racks to provide robust protection. This packaging design also enabled very efficient delivery and unpacking by eliminating the need for special handling equipment and tools to unload the systems in the data center.
By developing the facility, processes and tools to configure and test 72 racked systems in parallel, Sanmina was able to deliver 200 systems in a two-week time window each quarter. Most importantly, the data center company was able to reduce the failure rate and optimize overall system cost. Increasing the test coverage of the rack systems, along with a comprehensive burn-in process, significantly reduced early-life failures, eliminated all dead-on-arrival events and reduced the total cost.
These were some of the benefits of adopting a new business model for data center expansion. Especially important were the test-script and test-system development, burn-in, and logistics experience that the contract manufacturer brought to this engagement. Its expertise and global footprint enabled a new business model that significantly reduced costs for the data center. As the IT industry matures and evolves, this model may prove to be very effective for other high-growth data centers and cloud-computing deployments.
About the Author
Gelston Howell is Senior Vice President of Sanmina Corporation. Sanmina is a leading integrated-manufacturing solutions provider serving the fastest-growing segments of the global electronics manufacturing services (EMS) market. Recognized as a technology leader, Sanmina provides end-to-end manufacturing solutions, delivering superior quality and support to OEMs primarily in the computing and storage, communications networks, defense and aerospace, industrial, medical, multimedia, automotive, and clean-technology sectors. It has facilities strategically located in key regions throughout the world. More information regarding the company is available at Sanmina.com.