What is a cloud storage gateway?

Before going further, lets take a step back and address what for some may be a fundemental quesiton of what is a cloud storage gateway?

Cloud services such as storage are accessed via some type of network, either the public Internet or a private connection. The type of cloud service being accessed (figure 1) will decide what is needed. For example, some services can be accessed using a standard Web browser, while others must plug-in or add-on modules. Some cloud services may need downloading an application, agent, or other tool for accessing the cloud service or resources, while others give an on-site or on-premises appliance or gateway.

Figure 1: Accessing and using clouds (From Cloud and Virtual Data Storage Networking (CRC Press))

Cloud access software and gateways or appliances are used for making cloud storage accessible to local applications. The gateways, as well as enabling cloud access, provide replication, snapshots, and other storage services functionality. Cloud access gateways or server-based software include tools from BAE, Citrix, Gladinet, Mezeo, Nasuni, Openstack, Twinstrata and Zadara among others. In addition to cloud gateway appliances or cloud points of presence (cpops), access to public services is also supported via various software tools. Many data protection tools including backup/restore, archiving, replication, and other applications have added (or are planning to add) support for access to various public services such as Amazon, Goggle, Iron Mountain, Microsoft, Nirvanix, or Rack space among several others.

Some of the tools have added native support for one or more of the cloud services leveraging various applicaiotn programming interfaces (APIs), while other tools or applications rely on third-party access gateway appliances or a combination of native and appliances. Another option for accessing cloud resources is to use tools (Figure 2) supplied by the service provider, which may be their own, from a third-party partner, or open source, as well as using their APIs to customize your own tools.

Figure 2: Cloud access tools (From Cloud and Virtual Data Storage Networking (CRC Press))

For example, I can use my Amazon S3 or Rackspace storage accounts using their web and other provided tools for basic functionality. However, for doing backups and restores, I use the tools provided by the service provider, which then deal with two different cloud storage services. The tool presents an interface for defining what to back up, protect, and restore, as well as enabling shared (public or private) storage devices and network drives. In addition to providing an interface (Figure 2), the tool also speaks specific API and protocols of the different services, including PUT (create or update a container), POST (update header or Meta data), LIST (retrieve information), HEAD (metadata information access), GET (retrieve data from a container), and DELETE (remove container) functions. Note that the real behavior and API functionality will vary by service provider. The importance of mentioning the above example is that when you look at some cloud storage services providers, you will see mention of PUT, POST, LIST, HEAD, GET, and DELETE operations as well as services such as capacity and availability. Some services will include an unlimited number of operations, while others will have fees for doing updates, listing, or retrieving your data in addition to  basic storage fees. By being aware of cloud primitive functions such as PUT or POST and GET or LIST, you can have a better idea of what they are used for as well as how they play into evaluating different services, pricing, and services plans.

Depending on the type of cloud service, various protocols or interfaces may be used, including iSCSI, NAS NFS, HTTP or HTTPs, FTP, REST, SOAP, and Bit Torrent, and APIs and PaaS mechanisms including .NET or SQL database commands, in addition to XM, JSON, or other formatted data. VMs can be moved to a cloud service using file transfer tools or upload capabilities of the provider. For example, a VM such as a VMDK or VHD  is prepared locally in your environment and then uploaded to a cloud provider for execution. Cloud services may give an access program or utility that allows you to configure when, where, and how data will be protected, similar to other backup or archive tools.

Some traditional backup or archive tools have added direct or via third party support for accessing IaaS cloud storage services such as Amazon, Rack space, and others. Third-party access appliance or gateways enable existing tools to read and write data to a cloud environment by presenting a standard interface such as NAS (NFS and/or CIFS) or iSCSI (Block) that gets mapped to the back-end cloud service format. For example, if you subscribe to Amazon S3, storage is allocated as objects and various tools are used to use or utilize. The cloud access software or appliance understands how to communicate with the IaaS  storage APIs and abstracts those from how they are used. Access software tools or gateways, in addition to translating or mapping between cloud APIs, formats your applications including security with encryption, bandwidth optimization, and data footprint reduction such as compression and de-duplication. Other functionality include reporting, management tools that support various interfaces, protocols and standards including SNMP or SNIA, Storage Management Initiative Specification (SMIS), and Cloud Data Management Initiative (CDMI).

First impression: Interesting, good move Amazon, I was ready to install and start testing it today

The good news here is that Amazon is taking steps to make it easier for your existing applications and IT environments to use and leverage clouds for private and hybrid adoption models with both an Amazon branded and managed services, technology and associated tools.

This means leveraging your existing Amazon accounts to simplify procurement, management, ongoing billing as well as leveraging their infrastructure. As a standalone gateway appliance (e.g. it does not have to be bundled as part of a specific backup, archive, replication or other data management tool), the idea is that you can insert the technology into your existing data center between your servers and storage to begin sending a copy of data off to Amazon S3. In addition to sending data to S3, the integrated functionality with other AWS services should make it easier to integrated with Elastic Cloud Compute (EC2) and Elastic Block storage (EBS) capabilities including snapshots for data protection.

Thus my first impression of AWS storage gateway at a high level view is good and interesting resulting in looking a bit deeper resulting in a second impression.

Second impression: Hmm, what does it really do and require, time to slow down and do more home work

Digging deeper and going through the various publicly available material (note can only comment or discuss on what is announced or publicly available) results in a second impression of wanting and needing to dig deeper based on some of caveats. Now granted and in fairness to Amazon, this is of course a beta release and hence while on first impression it can be easy to miss the notice that it is in fact a beta so keep in mind things can and hopefully will change.

Pricing aside, which means as with any cloud or managed storage service, you will want to do a cost analysis model just as you would for procuring physical storage, look into the cost of monthly gateway fee along with its associated physical service running VMware ESXi configuration that you will need to supply. Chances are that if you are an average sized SMB, you have a physical machine (PM) laying around that you can throw a copy of ESXi on to if you dont already have room for some more VMs on an existing one.

You will also need to assess the costs for using the S3 storage including space capacity charges, access and other fees as well as charges for doing snapshots or using other functionality. Again these are not unique to Amazon or their cloud gateway and should be best practices for any service or solution that you are considering. Amazon makes it easy by the way to see their base pricing for different tiers of availability, geographic locations and optional fees.

Speaking of accessing the cloud, and cloud conversations, you will also want to keep in mind what your networking bandwidth service requirements will be to move data to Amazon that might not already be doing so.

Another thing to consider with the AWS storage gateway is that it does not replace your local storage (that is unless you move your applications to Amazon EC2 and EBS), rather makes a copy of what every you save locally to a remote Amazon S3 storage pool. This can be good for high availability (HA), business continuance (BC), disaster recovery (DR) and compliance among other data management needs. However in your cost model you also need to keep in mind that you are not replacing your local storage, you are adding to it via the cloud which should be seen as complimenting and enhancing your private now to be hybrid environment.

Walking the cloud data protection talk

FWIW, I leverage a similar model where I use a service (Jungle Disk) where critical copies of my data get sent to that service which in turn places copies at Rack space (Jungledisks parent) and Amazon S3. What data goes to where depends on different policies that I have established. I also have local backup copies as well as master gold disaster copy stored in a secure offsite location. The idea is that when needed, I can get a good copy restored from my cloud providers quickly regardless of where I am if the local copy is not good. On the other hand, experience has already demonstrated that without sufficient network bandwidth services, if I need to bring back 100s of GBytes or TBytes of data quickly, Im going to be better off bring back onsite my master gold copy, then applying fewer, smaller updates from the cloud service. In other words, the technologies compliment each other.

By the way, a lesson learned here is that once my first copy is made which have data footprint reduction (DFR) techniques applied (e.g. compress, de dupe, optimized, etc), later copies occur very fast. However subsequent restores of those large files or volumes also takes longer to retrieve from the cloud vs. sending up changed versions. Thus be aware of backup vs. restore times, something of which will apply to any cloud provider and can be mitigated by appliances that do local caching. However also keep in mind that if a disaster occurs, will your local appliance be affected and its cache rendered useless.

Getting back to AWS storage gateway and my second impression is that at first it sounded great.

However then I realized it only supports iSCSI and FWIW, nothing wrong with iSCSI, I like it and recommend using it where applicable, even though Im not using it. I would like to have seen a NAS (either NFS and/or CIFS) support for a gateway making it easier for in my scenario different applications, servers and systems to use and leverage the AWS services, something that I can do with my other gateways provided via different software tools. Granted for those environments that already are using iSCSI for your servers that will be using AWS storage gateway, then this is a non issue while for others it is a consideration including cost (time) to factor in to prepare your environment for using the ability.

Depending on the amount of storage you have in your environment, the next item that caught my eye may or may not be an issue that the iSCSI gateway supports up to 1TB volumes and up to 12 of them hence a largest capacity of 12TB under management. This can be gotten around by using multiple gateways however the increased complexity balanced to the benefit the functionality is something to consider.

Third impression: Dig deeper, learn more, address various questions

This leads up to my third impression the need to dig deeper into what AWS storage gateway can and cannot do for various environments. I can see where it can be a fit for some environments while for others at least in its beta version will be a non starter. In the meantime, do your homework, look around at other options which ironically by having Amazon launching a gateway service may reinvigorate the market place of some of the standalone or embedded cloud gateway solution providers.

What is needed for using AWS storage gateway

In addition to having an S3 account, you will need to acquire for a monthly fee the storage gateway appliance which is software installed into a VMware ESXi hypervisor virtual machine (VM). The requirements are VMware ESXi hypervisor (v4.1) on a physical machine (PM) with at least 7.5GB of RAM and four (4) virtual processors assigned to the appliance VM along with 75GB of disk space for the Open Virtual Alliance (OVA) image installation and data. You will also need to have an proper sized network connection to Amazon. You will also need iSCSI initiators on either Windows server 2008, Windows 7 or Red Hat Enterprise Linux.

Note that the AWS storage gateway beta is optimized for block write sizes greater than 4Kbytes and warns that smaller IO sizes can cause overhead resulting in lost storage space. This is a consideration for systems that have not yet changed your file systems and volumes to use the larger allocation sizes.

Some closing thoughts, tips and comments:

• Congratulations to Amazon for introducing and launching an AWS branded storage gateway.
• Amazon brings trust the value of trust to a cloud relationship.
• Initially I was excited about the idea of using a gateway that any of may systems could use my S3 storage pools with vs. using gateway access functions that are part of different tools such as my backup software or via Amazon web tools. Likewise I was excited by the idea of having an easy to install and use gateway that would allow me to grow in a cost effective way.
• Keep in mind that this solution or at least in its beta version DOES NOT replace your existing iSCSI based storage needs, instead it compliments what you already have.
• I hope Amazon listens carefully to what they customers and prospects want vs. need to evolve the functionality.
• This announcement should reinvigorate some of the cloud appliance vendors as well as those who have embedded functionality to Amazon and other providers.
• Keep bandwidth services and optimization in mind both for sending data as well as for when retrieving during a disaster or small file restore.
• In concept, the AWS storage gateway is not all that different than appliances that do snapshots and other local and remote data protection such as those from Actifio, EMC (Recoverpoint), Falconstor or dedicated gateways such as those from Nasuni among others.
• Here is a link to added AWS storage gateways frequently asked questions (FAQs).
• If the AWS were available with a NAS interface, I would probably be activating it this afternoon even with some of their other requirements and cost aside.
• Im still formulating my fourth impression which is going to take some time, perhaps if I can get Amazon to help sell more of my books so that I can get some money to afford to test the entire solution leveraging my existing S3, EC2 and EBS accounts I might do so in the future, otherwise for now, will continue to research.
• Learn more about the AWS storage gateway beta, check out this free Amazon web cast on February 23, 2012.

Learn more abut cloud based data protection, data footprint reduction, cloud gateways, access and management, check out my book Cloud and Virtual Data Storage Networking (CRC Press) which is of course available on Amazon Kindle as well as via hard cover print copy also available at Amazon.com.

Ok, nuff said for now, I need to get back to some other things while thinking about this all some more.

Cheers gs

For those of you in the Netherlands, or elsewhere in Europe, I’m going to be doing a two-day seminar for storage professionals along with for those involved in strategy, architecture and related data infrastructure topics on May 7 and 8. On May 9, I will be doing a deep dive companion seminar. You can learn more about these seminars being organized by Brouwer Consultancy in Nijkerk Netherlands by visiting their site here which includes agenda and related information.

Watch for more events, seminars, webinars and virtual trade shows by visiting the StorageIO events page.

Drop me a note if you would like to schedule or arrange for a seminar or event near you.

Ok, nuff said for now, see you out and about

Cheers
Gs

While I was at SNW, had the honor of being a guest on Calvin Zito (aka @HPStorageguy) pod cast that can be found on his Around the Storage Block Blog or by clicking here.

Check out our conversations about clouds, related topics and more from a practical perspective cutting through the hype and fud.

Oh, if you are interested in Cloud and Virtual Data Storage Networking, click here to learn more about the book, or here to find various downloads and associated presentations, and here to see some upcoming events, activities and venues both in the U.S. and in Europe.

Ok, nuff said for now.

Cheers
Gs

For a certain generation of IBM faithful or followers the recently announced PureFlex and PureApplication systems might give a sense of DejaVu perhaps even causing some to wonder if they just woke up from a long Rip Van Winkle type nap.

Yet for another generation who may not yet be future IBM followers, fans, partners or customers, there could be a sense of something new and revolutionary with the PureFlex and PureApplication systems (twitter @ibmpuresystems).

In between those two groups, exist others who are either scratching their heads or reinvigorated with enthusiasm to get out and be able to discuss opportunities around little data (traditional and transactional) and big data, servers, virtualized, converged infrastructure, dynamic data centers, private clouds, ITaaS, SaaS and AaaS, PaaS, IaaS and other related themes or buzzword bingo topics.

Let us dig a little deeper and look at some So What types of questions and industry trends perspectives comments around what IBM has announced.

So what did IBM announce?
IBM announced PureSystems including:

• PureFlex systems, products and technologies
• PureApplication systems
• PureSystems Centre

You can think of IBM PureSystems and Flex Systems Products and technology as a:

• Private cloud or turnkey solution bundle solution
• Platform deploying public or hybrid clouds
• Data center in a box or converged and dynamic system
• ITaaS or SaaS/AaaS or PaaS or IaaS or Cloud in a box
• Rackem stack and package them type solution

So what is an IBM PureFlex System and what is IBM using?
It is a factory integrated data and compute infrastructure in a cabinet combing cloud, virtualization, servers, data and storage networking capabilities. The IBM PureFlex system is comprised of various IBM and products and technologies (hardware, software and services) optimized with management across physical and virtual resources (servers, storage (V7000), networking, operating systems, hypervisors and tools).

PureFlex includes automation and optimization technologies along with what IBM is referring to as patterns of expertise or what you might relate to as templates. Support for various hypervisors and management integration along with application and operating system support by leveraging IBM xSeries (x86 such as Intel) and pSeries (Power7) based processors for compute. Storage is the IBM V7000 (here and here) with networking and connectivity via IBM and their partners. The solution is capable of supporting traditional, virtual and cloud deployment models as well as platform for deploying Infrastructure as a Service (IaaS) on a public, managed service provider (MSP), hosting or private basis.

Click here to view the next post in this series, ok nuff said for now.

Here are some links to learn more:
Various IBM Redbooks and related content
The blame game: Does cloud storage result in data loss?
What do you need when its time to buy a new server?
2012 industry trends perspectives and commentary (predictions)
Convergence: People, Processes, Policies and Products
Buzzword Bingo and Acronym Update V2.011
The function of XaaS(X) Pick a letter
Hard product vs. soft product
Buzzword Bingo and Acronym Update V2.011
Part I: PureSystems, something old, something new, something from big blue
Part II: PureSystems, something old, something new, something from big blue
Part III: PureSystems, something old, something new, something from big blue
Part IV: PureSystems, something old, something new, something from big blue
Part V: PureSystems, something old, something new, something from big blue
Cloud and Virtual Data Storage Networking

Cheers
Gs

Robin Harris (aka @storagemojo) recently in a blog post asks a question and thinks solid state devices (SSDs) using SAS or SATA interface in traditional hard disk drive (HDD) form factors are a bad idea in storage arrays (e.g. storage systems or appliances). My opinion is that as with many things about storing, processing or moving binary digital data (e.g. 1s and 0s) the answer is not always clear. That is there may not be a right or wrong answer instead it depends on the situation, use or perhaps abuse scenario. For some applications or vendors, adding SSD packaged in HDD form factors to existing storage systems, arrays and appliances makes perfect sense, likewise for others it does not, thus it depends (more on that in a bit). While we are talking about SSD, Ed Haletky (aka @texiwill) recently asked a related question of Fix the App or Add Hardware, which could easily be morphed into a discussion of Fix the SSD, or Add Hardware. Hmmm, maybe a future post idea exists there.

Lets take a step back for a moment and look at the bigger picture of what prompts the question of what type of SSD to use where and when along as well as why various vendors want you to look at things a particular way. There are many options for using SSD that is packaged in various ways to meet diverse needs including here and here (see figure 1).

Figure 1: Various packaging and deployment options for SSD

The growing number of startup and established vendors with SSD enabled storage solutions vying to win your hearts, minds and budget is looking like the annual NCAA basketball tournament (aka March Madness and march metrics here and here). Some of vendors have or are adding SSD with SAS or SATA interfaces that plug into existing enclosures (drive slots). These SSDs have the same form factor of a 2.5 inch small form factor (SFF) or 3.5 inch HDDs with a SAS or SATA interface for physical and connectivity interoperability. Other vendors have added PCIe based SSD cards to their storage systems or appliances as a cache (read or read and write) or a target device similar to how these cards are installed in servers.

Simply adding SSD either in a drive form factor or as a PCIe card to a storage system or appliance is only part of a solution. Sure, the hardware should be faster than a traditional spinning HDD based solution. However, what differentiates the various approaches and solutions is what is done with the storage systems or appliances software (aka operating system, storage applications, management, firmware or micro code).

So are SSD based storage systems, arrays and appliances a bad idea?

If you are a startup or established vendor able to start from scratch with a clean sheet design not having to worry about interoperability and customer investment protection (technology, people skills, software tools, etc), then you would want to do something different. For example, leverage off the shelf components such as a PCIe flash SSD card in an industry standard server combined with your software for a solution. You could also use extra DRAM memory in those servers combined with PCIe flash SSD cards perhaps even with embedded HDDs for a backing or preservation medium.

Other approaches might use a mix of DRAM, PCIe flash cards, as either a cache or target combined with some drive form factor SSDs. In other words, there is no right or wrong approach; sure, there are different technical merits that have advantages for various applications or environments. Likewise, people have preferences particular for technology focused who tend to like one approach vs. another. Thus, we have many options to leverage, use or abuse.

In his post, Robin asks a good question of if nand flash SSD were being put into a new storage system, why not use the PCIe backplane vs. using nand flash on DIMM vs. using drive formats, all of which are different packaging options (Figure 1). Some startups have gone the all backplane approach, some have gone with the drive form factor, some have gone with a mix and some even using HDDs in the background. Likewise some traditional storage system and array vendors who support a mix of SSD and HDD drive form factor devices also leverage PCIe cards, either as a server-based cache (e.g. EMC VFCahe) or installed as a performance accelerator module (e.g. NetApp PAM) in their appliances.

While most vendors who put SSD drive form factor drives into their storage systems or appliances (or serves for that matter) use them as data targets for creating LUNs or file systems, others use them for internal functionality. By internal functionality I mean instead of the SSD appearing as another drive or target, they are used exclusively by the storage system or appliance for caching or similar purposes. On storage systems, this can be to increase the size of persistent cache such as EMC on the CLARiiON and VNX (e.g. FAST Cache). Another use is on backup or dedupe target appliances where SSDs are used to store dictionary, index or meta data repositories as opposed to being a general data pool.

Part two of this post looks at the benefits and caveats of SSD in storage arrays.

Here are some related links to learn more about SSD, where and when to use what:
Why SSD based arrays and storage appliances can be a good idea (Part II)
IT and storage economics 101, supply and demand
Researchers and marketers don’t agree on future of nand flash SSD
Speaking of speeding up business with SSD storage
EMC VFCache respinning SSD and intelligent caching (Part I)
EMC VFCache respinning SSD and intelligent caching (Part II)
SSD options for Virtual (and Physical) Environments: Part I Spinning up to speed on SSD
SSD options for Virtual (and Physical) Environments, Part II: The call to duty, SSD endurance
SSD options for Virtual (and Physical) Environments Part III: What type of SSD is best for you?

Ok, nuff said for now, check part II.

Cheers gs

The good news is that companies interested in following the trend toward smaller data centers can employ a wide range of techniques to shrink their computing footprint.  IT consolidation is best approached by a combination of three primary strategies, design-level, white space and grey space. By employing a combination of strategies in each area, data center managers can not only make the most out of available footprint, but also assure mission-critical systems always have the clean, reliable electricity they need to drive long-term success.

Part one of this series discussed design-level strategies for IT consolidation; the following will recommend six white space tactics for shrinking the size of your data center without shrinking its operating capacity, availability or efficiency.

 1.       Utilize virtualization: Virtualizing and consolidating servers enables data centers to conserve floor space by replacing large numbers of lightly utilized devices with a smaller number of heavily utilized devices.

Consolidating storage resources through virtualization can similarly reduce a data center’s footprint by enabling it to house more data in less hardware. Leading-edge power companies can now integrate seamlessly with virtualization management systems like vCenter of VMware or SCVMM of Microsoft to enable live migration of virtual machines in case of power events. This is a significant improvement over the legacy server shutdown capability that is no longer addressing the need for business continuity which is inherent of any fully-virtualized environment. UPSs and advanced power management systems integrate seamlessly with software from VMware, Microsoft and other manufacturers to coordinate automatic migration or shutdown and infrastructure management in a virtual environment.

2.       Deploy blade servers: Often used in conjunction with virtualization, blade servers are plug-and-play processing units with shared power feeds, power supplies, fans, cabling and storage. By compressing large amounts of computing capacity into small amounts of space, blade servers can dramatically reduce data center floor space requirements. They also enhance IT agility, since companies can simply plug in additional blades any time their processing needs grow.

3.       Leverage cloud computing: In an effort to lower overhead and increase IT efficiency, businesses around the world are rapidly adopting cloud computing solutions. Indeed, some 15 percent of all IT spending in 2011 will be tied to cloud services or infrastructure, according to analyst firm IDC.

Cloud computing can help organizations shrink their data centers in several ways. By offloading applications and infrastructure to externally hosted “public” cloud data centers that use the public Internet to exchange data, companies can reduce the number of servers they must own and maintain in their own facilities. Alternatively, businesses can deploy “private” cloud solutions that utilize the same basic technologies as the public cloud but reside on privately owned or leased servers. Through the sophisticated use of virtualization and automation, private clouds dramatically raise server utilization rates, thereby lowering floor space requirements.

4.       Leverage capacity planning and asset management tools: Deploying more server and power resources than is necessary wastes floor space, but determining exactly how much capacity is required can be difficult, especially if virtual servers or a private cloud infrastructure are used. Capacity planning and asset management tools can help size a data center optimally for current and near-term needs, saving money while trimming the physical footprint.

5.       Implement a passive cooling scheme: Today, most organizations dissipate data center heat by placing computer room air conditioning (CRAC) units around the periphery of their server floor. Unfortunately, CRAC-based cooling systems are often incapable of handling the greater power densities and temperatures associated with technologies such as virtualization and blade servers.

As a result, some companies are now investigating or implementing in-row liquid cooling systems, which take up floor space that could otherwise be dedicated to IT equipment. By utilizing a hot or cold aisle containment strategy instead of in-row cooling, data center managers can fit more server racks in their existing data halls while keeping operating temperatures at safe levels. One such strategy would be a passive cooling system, which can come in many different types; the most efficient enclosures are equipped with a sealed rear door and chimney that captures hot exhaust air from servers and vents it directly back into the return air ducts on CRAC units. The CRAC units then chill the exhaust air and re-circulate it. A properly designed passive cooling system can cost-effectively prevent even the densest, hottest server racks from overheating. Furthermore, some passive cooling schemes don’t require raised floors, enabling companies to conserve the space formerly dedicated to maintaining air flow beneath their server enclosures.

6.       Switch from disk-based storage to static storage technologies: Though they currently account for a small fraction of storage hardware sales, static storage systems are slowly gaining popularity among enterprise IT managers. Such devices save data on solid-state memory chips, much like a USB memory stick. Though generally more expensive than conventional memory technologies, static storage systems are also dramatically faster, more energy-efficient and more reliable, since they contain no moving parts. In addition, static storage devices tend to be more compact than disk-based storage systems, so deploying them can help businesses save storage-related floor space in their data center. Tape-based archiving systems are also being impacted by cloud providers offering low-cost archiving storage at their facilities, something that can lead to reduction of equipment on the data center floor and still provide some type of disaster-proofing with data stored in another location.

Closing Thoughts

Today’s IT and facilities managers face a difficult bind: Though computing needs are constantly escalating, so is the cost per square foot of data center space and the price of critical supporting resources such as electricity and water.

Fortunately, there are many ways to compact a data center while still meeting business requirements. Most of them, moreover, are proven and cost-effective options for companies of almost any size. By following the white space recommendations outlined above, and the design-level strategies discussed in part one of this series, you can get more done in less space and position your company to meet its IT requirements with maximum cost effectiveness. Stay tuned for part three of the series, grey space consolidation strategies, for a complete guide to getting the most out of your physical footprint.

When it comes to protecting data, it may not be a headline news disaster such as earthquake, fire, flood, hurricane or act of man, rather something as simply accidentally overwriting a file, not to mention virus or other more likely to occur problems. Depending upon who you ask, some will say backup or saving data is more important while others will standby that it is recovery or restoration that matter. Without one the other is not practical, they need each other and both need to be done as well as tested to make sure they work.

Just the other day I needed to restore a file that I accidentally overwrote and as luck would have it, my local bad copy had also just overwrote my local backup. However I was able to go and pull an earlier version from my cloud provider which gave a good opportunity to test and try some different things. In the course of testing, I did find some things that have since been updated as well as found some things to optimize for the future.

My opinion is that if not used properly including ignoring best practices, any form of data storage medium or media as well as software could result or be blamed for data loss. For some people they have lost data as a result of using cloud storage services just as other people have lost data or access to information on other storage mediums and solutions. For example, data has been lost on cloud, tape, Hard Disk Drives (HDDs), Solid State Devices (SSD), Hybrid HDDs (HHDD), RAID and non RAID, local and remote and even optical based storage systems large and small. In some cases, there have been errors or problems with the medium or media, in other cases storage systems have lost access to, or lost data due to hardware, firmware, software, or configuration including due to human error among other issues.

Now is the time to start thinking about modernizing data protection, and that means more than simply swapping out media. Data protection modernization the past several years has been focused on treating the symptoms of downstream problems at the target or destination. This has involved swapping out or moving media around, applying data footprint reduction (DFR) techniques downstream to give near term tactical relief as has been the cause with backup, restore, BC and DR for many years. The focus is starting to expand to how to discuss the source of the problem with is an expanding data footprint upstream or at the source using different data footprint reduction tools and techniques. This also means using different metrics including keeping performance and response time in perspective as part of reduction rates vs. ratios while leveraging different techniques and tools from the data footprint reduction tool box. In other words, its time to stop swapping out media like changing tires that keep going flat on a car, find and fix the problem, change the way data is protected (and when) to cut the impact down stream.

Here is a link to a free download of chapter 5 (Data Protection: Backup/Restore and Business Continuance / Disaster Recovery) from my new book Cloud and Virtual Data Storage Networking (CRC Press).

Additional related links to read more and sources of information:

Choosing the Right Local/Cloud Hybrid Backup for SMBs
E2E Awareness and insight for IT environments
Poll: What Do You Think of IT Clouds?
Convergence: People, Processes, Policies and Products
What do VARs and Clouds as well as MSPs have in common?
Industry adoption vs. industry deployment, is there a difference?
Cloud conversations: Loss of data access vs. data loss
Clouds and Data Loss: Time for CDP (Commonsense Data Protection)?
Clouds are like Electricity: Dont be scared
Wit and wisdom for BC and DR
Criteria for choosing the right business continuity or disaster recovery consultant
Local and Cloud Hybrid Backup for SMBs
Is cloud disaster recovery appropriate for SMBs?
Laptop data protection: A major headache with many cures
Disaster recovery in the cloud explained
Backup in the cloud: Large enterprises wary, others climbing on board
Cloud and Virtual Data Storage Networking (CRC Press, 2011)
Enterprise Systems Backup and Recovery: A Corporate Insurance Policy

Take a few minutes out of your busy schedule and check to see if your backups and data protection are working, as well as make sure to test restoration and recovery to avoid an April fools type surprise. One last thing, you might want to check out the data storage prayer while you are at it.

Ok, nuff said for now.

Cheers gs

These findings mimic what we’re seeing at NGD Europe as multinational companies are looking for public, private and hybrid cloud computing solutions and the appropriate computing and storage environments for Big Data.  Other factors driving the demand for data center space include an improved economic outlook and a resurgence of interest from those organizations that had deferred expansion plans due to the economic uncertainty of years past.

An overwhelming 92 percent of respondents shared they will definitely or probably expand in 2012-2013 – the highest percentage ever recorded in the survey’s six-year history.  Of those with definitive plans to expand in 2012, nearly 40 percent expect to expand in three or more locations and approximately half plan to expand in Europe. Moreover, nearly 80 percent say they intend to use a partner (wholesale data center provider) for one or all of their projects and 41 percent cited plans to use a containerized module as part of their expansion.

According to the Global Containerized Data Center Market 2010-2014 by TechNavio, the market for containerized solutions will grow at a GAGR of 5.6 percent.  Key trends driving this market expansion include the demand by organizations worldwide for better ways to manage the complex data center infrastructure and the need for faster data center deployments.  Compute-hungry cloud computing providers searching for exceptional space to securely house and power large-scale containerized computing installations, look no further.  NGD Europe’s 750,000 square feet multi-tier carrier-neutral mega data center has an entire fit-for-purpose top floor available for immediate occupancy.

With a massive 180 MVA on tap – from 100 per cent renewable energy sources - the self-contained 140,000 square feet (net technical services) reinforced floor area offers all the space, power, cooling and high-speed communication links necessary for meeting the most demanding containerized computing requirements.  An on-site sub-station connected directly to the national grid ensures full control over power supply and high resilience, while the largely pillar-free area allows for maximum space utilization including stacking of containers up to three high. Roll-on, roll-off elevators and removable wall panels make for easy and rapid container installation and de-installation.

Large cloud providers are demanding bigger, better and more secure containerized computing facilities in Europe.  The NGD container floor offers the ultimate in accommodation for a very select number of large cloud providers searching for exceptional space, power and security – all from under one roof.  As with our popular custom-built data halls, we are able to offer container tenants highly competitive rates which are substantially lower than London-area facilities

Without going into any details, lets just say it is a popular myth just like saying that there is no money in hardware or that all software and people services are pure profit. Ok, lets leave sleeping dogs lay where rest (at least for now).

Why will some storage vendors make more margin off of HDD when everybody is supposed to be adopting or deploying solid state devices (SSD). Or Hybrid Hard Disk Drives (HHDD) in the case of workstation, desktop or laptops? Simple, SSD adoption (and deployment) is still growing and a lot of demand generator incentives available. Likewise HDD demand continues to be strong and with supplies affected, economics 101 says that some will raise their prices, manage their expenses, make more profits which can be used to help fund or stimulate increased SSD or other initiatives.

Storage, IT and general Economics 101

Economics 101 or basics introduces the concept of supply and demand along with revenue minus costs = profits or margin. If there is no demand yet a supply of a product exists then techniques such as discounting, bundling or other forms of adding value to incentivize customers to make a purchase. Bundling can include offering some other product, service or offering that could be as simple as an extended warranty to motivate sellers. Beyond discounts, coupons, two for one, future buying credits, gift cards or memberships for frequent buyers (or flyers) are other forms of stimulating sales activity.

Likewise if there is a supply or competition for a given market of a product or alternative, vendors or those selling the products including value added resellers (VARS) may sacrifice margin (profits) to meet revenue as well as unit shipped (e.g. expand their customer and installed base footprint) goals.

Currently in the IT industry and specifically around data storage even with increased and growing adoption and demand deployment around SSD, there is also a large supply in different categories. For example there are several fabrication facilities (FABs) that produce the silicon dies (e.g. chips) that form nand flash SSD memories including Intel, Micron, the joint Intel and Micron Fab (IMF) and Samsung. Even with continued strong demand growth, the various FABs seem to have enough capacity at least for now. Likewise manufactures of SSD drive form factor products with SAS or SATA interfaces for attaching to existing servers, storage or appliances including Intel, Micron, Samsung, Seagate, STEC and SANdisk among others seem to be able to meet demand. Even PCIe SSD card vendors have come under pressure of supply and demand. For example the high flying startup FusionIO recently saw its margins affected due to competition which includes Adaptec, LSI, Texas Memory Systems (TMS) and soon EMC among others. In the SSD appliance and storage system space there are even more vendors with what amounts to about one every month or so coming out of stealth. Needless to say there will be some shakeout in the not so distant future.

On the other hand, if there is a demand however limited supply, assuming that the market will support it, prices can be increased from what discounts had applied. Assuming that costs are kept inline any subsequent increase in average selling price (ASP) minus costs should result in higher margins.

Another variation is if there is strong demand and shortage of supply such as what is occurring with hard disk drives (HDD) due to recent flooding in Thailand, not only prices increase, there can also be changes to warranties or other services and incentives. Note some of HDD manufactures such as Western Digital were more affected by the flooding than Seagate. Likewise the Thailand flooding was not limited to just HDD having also affected other electronic chip and component suppliers. Even though HDDs have been declared dead by many in the SSD camps along with their supporters, record number of HDDs are produced every year. Note that economics 101 also tells us that even though more devices are produced and sold, that may not show a profit based on their cost and price. Like the CPU processor chips produced by AMD, Broadcom, IBM and Intel among others that are high volume, with varying margins, the HDD and nand flash SSD market is also high volume with different margins.

As an example, Seagate recently announced strong profits due to a number of factors even though enterprise drive supply and shipments were down while desktop drives were up. Given that many industry pundits have proclaimed a disaster for those involved with HDDs due to the shortage, they forgot about economics 101 (supply and demand). Sure marketing 101 says that HDDs are dead and if there is a shortage then more people will buy SSDs however that also assumes that people are a) ready to buy more SSDs (e.g. demand) and b) vendors or manufactures have supply and c) that those same vendors or manufactures are willing to give up margin while reducing costs to boost profits.

Note that costs typically include selling, general and administrative, cost of goods, manufacturing, transportation and shipping, insurance, research and development among others. If it has been awhile since you looked at one, take a few minutes sometime to look at public companies and their quarterly securities exchange commission (SEC) financial filings. Those public filing documents are a treasure trove of information for those who sift through them and where many reporters, analysts and researchers find information for what they are working or speculating on. These documents show total sales, costs, profits and losses among other things. Something that vendors may not show in these public filings which means you have to look or read between the lines or get the information elsewhere is how many units were actually shipped or the ASP to get an idea of the amount of discounting that is occurring. Likewise sales and marketing expenses often get lumped into or under general selling and administration (SGA). A fun or interesting metric is to look at the percentage of SGA dollars spent per revenue and profits.

What I find interesting is to get an estimate of what it is costing an organization to do or sustain a given level of revenue and margin. For example, while some larger vendors may seem to spend more on selling and marketing, on a percentage basis, they can easily be out spent by smaller startups. Granted the larger vendor may be spending more actually dollars however those are spread out over a larger sales and revenue basis.

What does this all mean?

Look at multiple metrics that have both a future trend or forecast as well as trailing or historical perspective view. Look at both percentages as well as dollar amounts as well as both revenue and margin while keeping units or number of devices (or copies) sold also into perspective. For example its interesting to know if a vendors sales were down 10% (or up) quarter over quarter, or versus the same quarter a year ago or year over year. It is also interesting to keep the margin in perspective along with SGA costs in addition to cost of product acquired for sale. Also important is to get a gauge of if sales were down, yet margins are up, how many devices or copies were sold to get a gauge on expanding footprint which could also be a sign of future annuity (follow up sales opportunities). What Im watching is over the next couple of quarters is to see how some vendors leverage the Thailand flooding and HDD as well as other electronic component supply shortages to meet demand by managing discounts, costs and other items that contribute to enhanced margins.

Rest assured there is a lot more to IT and storage economics, including advanced topics such as Return on Investment (ROI) or Return on Innovation (The new ROI) and Total Cost of Ownership (TCO) among others that maybe we will discuss in the future.

Ok, nuff fun for now, lets get back to work.

Cheers gs

Blogs, social media networks, machine sensors and location-based tools are generating a whole new universe of unstructured data that must be stored, captured, analyzed and processed.  This coupled with the large amounts of structured data from cloud and mobile computing has given rise to the era of Big Data – or, the idea that datasets grow so large they become awkward to work with using traditional database management tools.  The market demand for new tools, processes and procedures that enable organization to create, manipulate, and manage very large data sets and storage facilities is so great, IDC predicts the Big Data technology and services market to reach $16.9 billion by 2015.  This represents a compound annual growth rate (CAGR) of 40% or about seven times that of the overall information and communications technology (ICT) market according to IDC.

To take advantage of the data-driven discovery and decision-making that comes with the promise of real-time predictive intelligence, IT leaders must begin prepping themselves and their organizations now to support Big Data.  IT managers must be prepared for changes to both server and storage architectures that are extensible and scalable.  Once siloed, disparate systems for payroll, customer management, marketing etc., are coming together to create a system of systems and the necessity for more storage and abundant, uninterrupted power has never been so great.

State-of-the-art data center facilities like NGD Europe, are well equipped to handle the demands of Big Data and its precursor technologies including virtualization, supercomputing and cloud-based computing.  The built-to-suit, Tier 3 facility has a private distribution network providing up to 180 MVA on tap, sourced directly from its own on-site substation that connects directly to the national grid.  This provides the data center with full control over its entire power supply ensuring a clean, stable and maintainable power infrastructure that’s sourced entirely from renewable energy sources.  and a variety of on-site high-speed, low latency carrier interconnects.

Oracle’s research report, The Next Generation Data Center Index Cycle II, found global businesses have been caught off-guard by the recent explosion in data volumes and are looking to buy external data center capacity to cope. Between 2010 and 2011 the report found there was a rise from 40 to 60 percent in the number of businesses using external data centers and citing Big Data as the biggest IT challenge facing businesses over the next two years.

Data visualization created by IBM

For those global organizations looking for a data center partner that offers highly secure and cost-effective space, housing up to 19,000 racks that can be arranged into self-contained data halls of various sizes – all with independent services, resilient power and cooling systems – look no further. NGD’s environmentally friendly, high-performance technology infrastructure and abundant power supply supports Big Data at big cost savings compared to its London-area counterparts.  Plus, the added safety and security bonuses a rural data center facility provides, while still being in proximity of an international hub with major airports only a two-hour drive away.