Non-Volatile Memory Express (NVMe)—which boasts lower latencies than legacy protocols—is poised to transform the industry as the need for low latency becomes even more critical for large data centers. Although the NVMe conversation largely revolves around performance advantages, another issue deserves equal attention: how your cable infrastructure plays into the situation.
According to the “NVMe over Fibre Channel” chapter of IBM Redbooks, “The next wave of all flash arrays will provide flash media connectivity to NVMe over Fibre Channel, providing reduced latency and performance gains. To avoid storage response latency, flash media connected to NVMe over Fibre Channel allows direct communication from the host to the storage. This is done by using the Component Interconnect Express (PCIe) peripheral commands, bypassing the Small Computer System Interface (SCSI) translation layer.”
As enterprise data centers continue to upgrade to Gen6 Fibre Channel, with its faster speeds, the quality of the fiber becomes more and more important—especially in products implementing NVMe over Fabrics. With each subsequent generation of Fibre Channel and other SAN protocols, the distance limit and insertion-loss budget are shrinking. Any company exploring NVMe technology must ensure its cable plant will support the speed necessary to use NVMe to its full potential.
If you’re thinking about deploying NVMe—or if you’re wondering whether it’s right for you—keep these tips in mind.
NVMe Isn’t for Everyone
Small companies, and those that are minimally dependent on the latency of their data, may choose to phase in NVMe flash-based storage. Companies that offer home security cameras, for example, typically store the videos—which capture any motion around a house—and make the footage available for users to retrieve whenever they choose. Such companies are probably less concerned about the latency of their data, since it enters a sort of cold storage.
What type of companies would benefit? Examples include insurance, health-care and finance facilities, where fast transactions and information retrieval are crucial. If speed is unnecessary, however, a major investment in NVMe is probably unnecessary as well.
Redesign Your Cable Infrastructure at the Same Time as Your NVMe Transition
All speed upgrades are in the director-class switches—that’s where your initial investment should begin. This approach will drive updates to your storage and high-speed Fiber Channel networks. Afterwards, focus on the server environment.
When building out your infrastructure to support an upgrade to a new switch, keep in mind the next-generation speeds that you must soon address. A good rule of thumb is to plan at least a generation ahead so you don’t have to discard what you just put in—especially considering that a new generation comes out about every 18 to 24 months.
Unfortunately, if you ignore the cabling infrastructure when you perform the switch and NVMe transition, you could face extending refresh time lines and additional costs by having to reconfigure network cabling after installation.
Place Switch Gear Close to Your Central Patching Location
Configuring your switch gear near the central patching location (CPL) allows you to account for the cabling distance in your overall switch-to-server or switch-to-NVMe device. That distance begins at the physical switch and goes to the CPL. Then, you must account for a jumper between the two to reach the device that the switch is supporting—and that’s where distances come into play.
How far away is that end device from that switch? If you move your switch gear as close as possible to the patch bay, the answer is a given—it’s as small a distance as possible in that environment. At that point, you mainly need to concern yourself with how far away the CPL is from the server or storage.
Although Fibre Channel networks are ready for use with NVME over Fibre Channel, addressing distance limitations of those networks is necessary to minimize latency and keep your Gen6 and NVMe performance at its best. It can be easy to take the cabling for granted because it has always worked, but that’s not the case with NVMe and its next-generation speeds—your cabling must be as much of a consideration as the equipment itself.
Think of it this way: The cost of cabling is roughly 10 times less than that of new equipment. When you consider the total cost of your project, it’s a minor expense with major implications—it’s your cable infrastructure, after all, that will ensure your Gen6 equipment performs to its intended potential. Gen6 Fibre Channel is ready for NVMe and offers the ideal connectivity for high performance only if you’ve updated your cable infrastructure to support it.
About the Authors
Rodney Jacks is technical-solutions manager for Data Center Systems (DCS), which designs, manufactures and installs fiber connectivity solutions. In this role, he’s responsible for cabling solutions in data centers.
Todd Wheeler is technical-account manager for DCS, where he is responsible for the technical standards concerning Fibre Channel specifications. He is also a member of INCITS T11.