Blockchain is best known as the underlying technology behind cryptocurrencies such as Bitcoin and Ethereum, but savvy technologists have recognized that it has even more-valuable applications beyond creating a new digital form of money. At its core, a blockchain is a cryptographic algorithm that can securely verify contracts between parties and create an immutable public record, all without needing an intermediary to guarantee trust. Visionaries have pointed to the supply chain, global finance and real estate as industries that could put this technology to good use.
Add one more area to that list: enterprise storage.
Storage capacity has long been a painful management experience for CIOs. Running out of capacity can bring the entire enterprise to a halt, so it’s a top priority to ensure there’s plenty of storage to handle current and future needs. Unfortunately, IT must therefore buy far more capacity than it can currently use in order to meet those future demands.
Despite the essentially limitless storage capacity available in the public cloud, the cloud can’t entirely solve the enterprise capacity challenge. For data that must be immediately available, the inherent lag in connecting to a public cloud may remove cloud storage as an option. Additionally, cloud storage is expensive, especially if you’re dealing with massive amounts of data that must always be available. Blockchain technology can play a central role in solving the storage-capacity problem.
If you’re in your 30s or 40s, you probably remember the more innocent days of the early Internet, when nearly everyone had the SETI application running on their desktop. That app used individuals’ excess computing power to crunch through mountains of data to find evidence of extraterrestrial life. Today, existing technology allows enterprises to do the same thing with storage. If it needs additional capacity, IT would be able to securely purchase storage from nearby organizations that have more than they need. Conversely, companies with too much capacity could monetize it by selling storage to their neighbors.
So imagine a world that offers a local storage marketplace as a community grid composed of storage providers and consumers. As organizations need to reclaim their excess capacity, the marketplace seamlessly enables it, while the consumers’ data moves to the most appropriate location automatically on the basis of performance, policies and SLAs.
It may sound crazy, but it’s similar to a distributed-consumption model that everyone, from individuals to enterprises, already uses daily: the electrical grid. The grid enables electric companies to buy and sell excess power to one another. No one thinks twice about where their power was originally generated, whether from a Canadian hydroelectric dam or a wind farm off the coast of Cape Cod.
Distributed storage would likely be far faster than cloud storage because IT would be buying capacity from nearby neighbors rather than from a source hundreds of miles away. And, depending on demand, costs could also be lower. After all, excess storage that’s just sitting unused still incurs costs for electricity, cooling and maintenance; as long as the risk of selling unused capacity is low, enterprises would be unlikely to sell additional storage in a distributed system for anywhere close to what the public cloud charges.
This peer-to-peer distributed-storage model requires three components: distribution, security and a transaction marketplace. Distribution is the easiest component to address, as high-performance distribution across a multinode environment is readily realizable through peer-to-peer (P2P) content distribution. Many organizations such as Microsoft already use this type of content delivery to lessen the stress on central distribution hubs. Distribution can further be accelerated by chunking the files or objects into standard or adaptive file sizes as appropriate.
The second requirement is security, and in this context, it encompasses three essential attributes: confidentiality, availability and integrity. Confidentiality is achievable through established techniques for encryption and data protection to ensure the data is protected at the source, in motion and at rest. With appropriate encryption and effective key management, data components are completely protected from prying eyes.
Availability involves ensuring that each piece of data is securely stored in multiple, redundant locations. SLAs may dictate the level of redundancy, but a minimum threshold should ensure that data is not lost by nodes going offline or storage providers reclaiming their capacity.
Finally, integrity is enforceable by implementing strong user-access controls as well as employing checksums and hash tables that are shared across the community storage grid. Doing so prevents data tampering, as it’s securely distributed, stored and retrieved across the grid.
The third requirement is the marketplace that tracks all capacity buying and selling. Each transaction must be immutable so that payments can be effectively managed to provide benefits to both the seller and the consumer.
The Role of Blockchain
Blockchain technology not only provides additional security, but it also offers an elegant approach to the challenge of immutability, ensuring that 1) a contract exists between the storage owner and writer of data and 2) the data has been written to storage. As data is segmented and distributed across the grid, blockchain and distributed-ledger technology can ensure that each data action and movement appears in the ledger, so nothing occurs outside of the known ledger. Most importantly, the blockchain creates an immutable record that cannot be changed after the fact, creating the necessary trust a marketplace requires.
Blockchain technology also improves on the availability model because when a data provider reclaims storage capacity, it triggers the marketplace to copy the relevant data to an alternative data provider and provide confidence that the data is redundantly stored. This feature is particularly useful in the cryptocurrency world because a blockchain provides a secure ledger of ownership and contracts. In the world of distributed storage, instead of creating a ledger of cryptocurrency transactions, the blockchain records data movement and ownership. There’s no need for a third party to secure trust. The blockchain secures trust all by itself.
Of course, this state of affairs begs the question: if storage capacity is such a headache, and if the technology exists to enable peer-to-peer distributed enterprise storage, why isn’t it in use already? The answer is simple, because it all comes down to trust. People remain sensitive about storing data on platforms they don’t control. AWS has been publicly available for more than a dozen years, and still many CIOs balk at sending anything remotely mission critical to the cloud. They won’t be any happier about storing their data in environments that belong to nearby businesses.
If one or more of the startups working to create a peer-to-peer enterprise storage market succeeds, the result would transform a large chunk of the data-storage industry, making enterprise storage even more efficient and nimble. Organizations could make a great deal more progress in their move from a capital-investment storage model to a consumption model. As a result, on-premises enterprise storage sales would likely feel the impact because the need to buy excess storage capacity so far in advance would decline.
Cloud storage, however, is unlikely to be heavily affected. The cloud will always be able to do some things better because of its scale and capacity. Plus, though buying and selling storage with other nearby businesses would reduce latency, the amount of cryptography required to power the marketplace, both in encryption for security and blockchain, would require additional raw power from computing resources.
Given that the technical barriers should be easy to surmount, because the underlying solutions to create these markets largely already exist, I believe we’ll see the first peer-to-peer distributed enterprise storage markets appear in the next few years. The economic and operational advantages are simply too great for CIOs to ignore.
About the Author
Danny Allan is vice president of product strategy for Veeam Software.