Data center build-outs continue to grow, driven by the increase in bandwidth demands and changes in network architectures. As a result, time-to-market needs for the data center production environment continue to compress. In addition to these demands, data center managers are struggling to balance space constraints, rapid deployment timelines and upgrade scenarios given their staffing levels and funding.
As presented by the Dell’Oro Group at the Ethernet Alliance Forum in 2014, 10GbE switching is the current majority of data center switch revenue, with 40GbE and 100GbE switch revenue beginning to show growth. Bandwidth demands will continue to develop, as the majority of server-port bandwidth currently operates at 10GbE but is moving to 25GbE and 40GbE. This increase in demand at the server level is driving the need for 100GbE switch connectivity.
Adoption of network architectures such as spine and leaf as well as the use of technologies such as parallel optics are driving not only bandwidth demand, but also the scale of the network, requiring a greater fiber count for the cabling infrastructure. For spine-and-leaf architectures, each leaf switch in the network connects using a mesh to every spine switch in the network to provide increased resiliency. As a result, when this architecture is deployed in data centers, fiber counts can multiply very quickly compared with traditional three-layer distribution architectures. In addition, parallel optics uses eight fibers per port instead of the traditional two fibers per port of duplex applications, such as GbE and 10GbE. Therefore, when deploying parallel technologies, the fiber counts in your data center structured cabling will increase regardless of whether you’re using spine-and-leaf or traditional three-layer architectures.
Although bandwidth demand may come from different sources—such as support for external clients operating in a cloud-based-services data center, and internal lines of business for a traditional data center—the need to offer services more quickly is critical. To support this effort, the infrastructure must be capable of efficient deployment, offering rapid transition into production.
Impact on Structured Cabling
Traditional structured cabling deployments in the data center are based on infrastructure designs using pre-terminated MTP assemblies ranging from 12 to 144 fibers. To support the growing bandwidth needs as well as changing technologies and architectures, fiber-count requirements are also increasing, with spaces in the data center requiring connectivity to support demands of up to 288 fibers in a single run—and even fiber counts of 576 fibers in high-density areas.
As these types of deployments become necessary, it’s important to continue considering the impact of the application, environment and installation methods required to provide this high-fiber-count connectivity. Multiple design options enable implementation of high-fiber-count cabling and connectivity, each supporting scalability and helping the data center bring services to market quickly.
As the data center environment evolves, the supporting cables and connectivity must change as well. Figure 3 shows where connectivity is required in a data center. High-fiber-count solutions are commonly needed between the main distribution areas (MDAs) of multiple computer rooms, or data halls, in a data center building. Within a computer room or data hall, high-fiber-count cabling can also be deployed between the MDA and the server rows as well as between the MDA and the data center core switching racks.
Designing the optical-cabling infrastructure for a data center requires consideration of many factors, including network architecture and physical planning of the white space, or data center production area. The optical cabling can be deployed in a manner that mimics the network architecture layout. For example, a common practice in many data center designs is the use of top-of-rack (ToR) switch architectures. One option in cabling for this type of architecture is to install dedicated low-fiber-count optical cables to the ToR switch in each cabinet. Alternatively, the optical cabling can be deployed in a middle-of-row (MoR) or end-of-row (EoR) topology, using patch cords to support connectivity from the MoR or EoR structured cabling to the end equipment. This approach enables consolidation of the optical cabling, making more-efficient use of rack space and providing pathway-space savings.
In many legacy data center designs, links requiring more than 12 fibers consist of multiple 12-fiber trunks. As fiber needs continue to grow, the fiber count of the cabling deployed in a given pathway has increased; but using a traditional approach of multiple low-fiber-count cables to accomplish this task is a challenge for both pathway-space utilization and cable management. To address these challenges, many data center cabling designs use MTP trunks with up to 144 fibers. In data centers requiring link deployments with more than 144 fibers, multiple runs of a 144-fiber cable assembly are typically installed to achieve the total desired fiber count. For example, if a link requires 288 fibers from the main distribution area of the data center to another location, two 144-fiber trunk cables would be installed. The use of multiple cables can fill the available pathway space quickly, reducing the physical space capacity for future growth. An improved approach would include installation of a single high-fiber-count trunk (e.g., 288 fibers) in place of the multiple lower-fiber-count trunk cables. Doing so reduces day-one space requirements, leaving room to grow, as well as reducing the quantity of trunks, cutting the deployment time.
Figure 4 depicts the space savings across three deployment scenarios in a 12-inch x 6-inch cable tray with a 50 percent fill ratio:
- 4,440 total fibers using 370 x 12-fiber MTP-MTP Edge trunks
- 13,680 total fibers using 95 x 144-fiber MTP-MTP Edge trunks
- 16,128 total fibers using 56 x 288-fiber MTP-MTP Edge trunks
To meet the need for high-fiber-count cable and connectivity solutions, various implementation options are available. Depending on the application as well as deployment considerations, each solution has a target placement in the data center.
MTP connectivity is an important component of the solutions recommended in high-fiber-count environments. The MTP footprint enables the lowest total cost of ownership for the implementation of or future transition to 40/100/200/400GbE networks using parallel optical technologies. Additionally, installation of cabling with MTP connectivity allows the deployment of the optical-fiber terminations 12 fibers at a time rather than individual termination of single fiber strands.
These MTP terminations can then either break out into individual ports using MTP-LC modules or serve directly as MTP interfaces. Given variations in infrastructure design, cabling environments and pathway types, MTP connectivity in backbone cabling can employ multiple methods. Below are two possibilities:
- Cables that are factory terminated on both ends using MTP connectors (MTP trunk assemblies)—see Figure 5.
- Cables that are factory terminated on one end using MTP connectors and then field terminated at the blunt cable end of an MTP pigtail trunk—see Figure 5.
MTP-MTP Trunk Assemblies
MTP trunk assemblies are used where the entire fiber count is being landed at a single location at each end of the link—for example, between the MDA and the server rows or between the MDA and the core switching racks in a computer room or data hall, as Figure 6 shows. Additionally, high-fiber-count MTP-MTP trunks also appear between MDAs of multiple computer rooms or data halls where open tray is the pathway.
MTP Pigtail Trunks
The application for MTP pigtail trunks has two primary use cases. One is for environments where the pathway won’t allow for a pre-terminated end with pulling grip to fit through—for example, a small conduit space (see Figure 7). For instance, this approach is common when needing to provide connectivity between MDAs of multiple computer rooms or data halls. Additionally, a deployment using pigtail trunks can be useful when the exact pathway or route is not fully known, preventing exact length measurement before ordering of the assembly.
MTP pigtail trunks can be terminated in multiple ways. Figure 8 depicts the recommended solution set to maintain MTP connectivity for the full link. Field terminating a high-fiber-count MTP pigtail trunk directly with MTP splice-on connectors is recommended when the assembly is landing at a single cabinet or location.
An alternative to terminating the MTP pigtail trunk with MTP splice-on connectors is to splice each leg of the MTP pigtail trunk to individual 12-fiber MTP pigtail assemblies, or to another MTP pigtail trunk. This splicing can be done in splice trays installed in a separate wall or rack-mount splice housing.
Planning and installing a data center cabling infrastructure for actual and future needs is a complicated task; but a simple, fast and easy implementation is within reach. Choosing the best solution will depend on several factors:
- Application environment: inside or between computer rooms or data halls
- Design requirements: traditional three-layer or spine-and-leaf architecture
- Installation needs: speed of deployment, quality control, pathway type, fiber count, cost, plan for or measure cable routes
- Future proofing: Transition path and future-technology support—for example, parallel optics
By using high-fiber-count trunks as your backbone cabling, you can combine maximum density with faster installation, lower pathway congestion and greater efficiency while delivering the bandwidth to meet today’s needs providing a simpler transition path to 40GbE/100GbE/200GbE and beyond.
About the Authors
Jennifer Cline is data center market development manager and Luis Abreu is a senior systems engineer for Corning Optical Communications.