Any change in business processes will have an effect. The juggernaut of digital transformation brings the potential for incredible benefits, but they come at the expense of traditional storage architectures. Digitized businesses generate and receive data in such volumes that the cost to scale out using legacy storage solutions is unsustainable.
It would also take too long to scale linearly to meet storage demands. In addition, even adding multiple servers can’t meet the need. Vertical storage architecture contains bottlenecks that slow performance to an unacceptable level.
Fortunately, linear scaling with hardware isn’t the only option. Software-defined storage (SDS) decouples the programming that controls storage-related tasks from the physical storage hardware, dramatically reducing costs associated with hardware. Fewer, less expensive servers can be used to improve both capacity and performance. Administration becomes simpler as well as more flexible and efficient. SDS enables users to allocate and share storage assets across all workloads.
With all the benefits SDS offers, it has enjoyed widespread adoption. Gartner recently reported that by 2020, anywhere from 70 to 80 percent of unstructured data will be stored and managed on lower-cost hardware supported by software-defined storage.
Today’s Storage Needs File Systems
File systems are available in most—80 percent—of storage solutions on the market. That’s because 80 percent of data is unstructured. Although it’s widely understood that unstructured data is best managed with a file system, many SDS offerings for some reason focus solely on block or object storage. Few offerings focus on file systems or do them well. Without a file system overlaying this data, the data becomes very difficult to manage.
Each type of storage exists for a purpose:
- There’s a huge amount of interest in object storage. It’s used for machine-to-machine/IoT transactions and other applications that require extreme scalability, but it isn’t that much better than block when managing data.
- The basic foundation of storage, block is used for storing virtual machines or databases, but you need files as well to deal with all the unstructured data.
- Necessary but often overlooked, file system storage isn’t as hyped as object, but it’s the best at handling unstructured data.
Since unstructured data is so prominent, it’s understood that file systems are an important element in modern storage. Some software-defined-storage providers claim to provide a file system with their offerings, but these file systems are usually based on Samba and exclude some features most Windows users are familiar with.
Samba is a freeware module that enables support for SMBs and allows end users to access and use files on the company’s intranet or network. Many in need of a file system have gone this route. But providing file services through Samba, which is open source, often means going without needed features.
This is no small matter. Features are extremely important. It’s not just the file system that organizations need in order to deal with unstructured data; file-related features are also necessary. They include the following:
- Quota: This feature helps you monitor the amount of storage being used. You can set a soft-limit quota that will warn you when part of a file system is close to reaching its storage limit but still allow data to be saved. If you set up a hard-limit quota, no new data can be saved after the quota is reached.
- Snapshot: A snapshot is a read-only copy of the contents of a file system or independent file set, taken at a single point in time. When a snapshot of an independent file set is taken, all files and nested dependent file sets will be included.
- Tiering: By using a policy, you create a filter that designates a specific file type to a particular tier. The policy enables you to designate where a specific file is to be placed as well as if and when the file will be moved between file-system pools. You can define both file placement and migration policies. Tiered storage is more efficient and boosts performance.
- Retention: Automatically create a single folder or a hierarchy of folders on file servers, to be deleted according to assigned policies.
Know Before You Go
Clearly, file systems are essential for managing unstructured data—the majority of your data pool. What’s unclear is why most SDS providers omit the robust file system they know you need. Open-source solutions get an A for effort but can’t deliver the features necessary to make file-system storage fully functional. A unified model that includes object, block and file-system storage serves organizations best. In a world where storage needs are increasing exponentially, it’s important to choose wisely, so use the information above to ask the right questions before you buy.
About the Author
Stefan Bernbo is the founder and CEO of Compuverde. For 20 years, Stefan has designed and built numerous enterprise-scale data-storage solutions that are cost effective for storing huge data sets. From 2004 to 2010, Stefan worked in this field for Storegate, the wide-reaching Internet-based storage solution for consumer and business markets with the highest availability and scalability requirements. Previously, Stefan has worked with system and software architecture on several projects with Swedish giant Ericsson, the world-leading provider of telecommunications equipment and services to mobile- and fixed-network operators.