When searching for a new in-memory database system, it can be overwhelming to sift through the options if you don’t have a process to follow. Without in-memory databases, companies have to limit the number of users that can interact with its data, leading to fewer analyses, less automation and less optimization of business systems.
First and foremost, it’s critical that organizations identify the business applications that absolutely must benefit from performance to provide better service and products and/or optimize operations. From there, the business intelligence (BI) must clearly specify the criteria for a database and define a complete set of benchmark requirements. Knowing this information up front is imperative before evaluating the offerings from various vendors. Beyond setting the criteria, companies should also establish a solid team, including representatives from different business units, to research the possible solutions. It should comprise members from the following:
- The BI team to check the analytical capabilities and performance of the system, and to translate the departmental requirements into technical ones.
- The IT department to research how easy it is to integrate the system into the existing ecosystem and IT landscape.
- External consultants or analysts who can research and advise on a short list of vendors that could be suitable for addressing the specific requirements.
Once you have your team in place, the top five things you should consider when conducting research on in-memory databases are the following:
- General system architecture: One of the first things to keep in mind is that every solution is different, and true analytic performance can only be guaranteed through a tight integration of in-memory computing, as opposed to just adding a cache. Through an integrated in-memory computing approach, users can run larger and more-complex analytic workloads, as well as use the database for a wider range of use cases.
- Costs and scalability: The next thing companies should inquire about when researching systems are the software acquisition or licensing costs, as well as hardware costs. Furthermore, users should ask themselves whether the solution they are considering is a scalable massively parallel (MPP) system to which they can easily add more servers. If not, companies should prepare themselves for any added costs as requirements and data volumes continue to grow.
- Integration: Next, users need to investigate whether the solution is mature enough to handle complex analytic workloads. Does it support the commonly used drivers and interfaces? Does it integrate with the most widely used extract, transform, load (ETL) and business intelligence (BI) tools, too? As analytic ecosystems adapt over time, users need to ensure that the database will still be compatible in the future.
- Vendor maturity and proven customer success: Users should also find out about the vendor behind the product and inquire about the levels of ongoing support that it offers. Reaching out to existing customers to discuss the system’s real-life advantages as well as any shortcomings is also beneficial if they are willing to share any insight or if the vendor provides customer references. It’s not enough to use a technically solid solution; users need to assure themselves that they can depend on the vendor and its customer ecosystem for ongoing support.
- Simplicity: Finally, anyone considering an in-memory database needs to ask whether the solution is easy to install and operate, or whether an army of database administrators is required to tune, design and implement the database as well as ETL processes. The more automated the solution, the fewer the hurdles users will have to jump over to find value in their analytics and BI projects.
Companies should also be prepared for challenges that come with looking for an in-memory database system, however, as solutions differ considerably. Since there are different kinds of architectures for various systems, each with its own advantages/disadvantages, consider the following factors:
- Pure query performance: Since in-memory alone is not the solution for every problem, users should run a series of performance tests using a wide range of queries to determine the best way to gain analytic performance.
- Memory consumption: Users need to ask themselves how much hardware the database really needs to function. Also, does the whole database need to reside in RAM or is it an intelligence hybrid solution where only the “hot data” needs to be kept in cache? This characteristic can vary greatly and influence the final buying decision.
- Scalability: Can the solution grow in line with your needs and still deliver performance?
- Ease of use: What is the administration overhead to use the system?
- Operational limitations: Is the system ACID compliant? Does it support general drivers and interfaces? Does it offer failover capabilities?
- Level of support offered by the company: Does the company offer support? Is the company professional in its dealings? Is it dedicated to making you successful?
Overall performance and simplicity are two of the main driving factors in achieving customer satisfaction with an in-memory database. Only with superior performance can companies run the most complex analyses in near real time and concentrate on their analytics instead of their database setup. Also, the simplicity of using the product allows users to focus on analytics and finding value in the data instead of having to “make the database work” through tuning or designing queries.
When researching a new in-memory database system, I recommend that all potential buyers use the checklist above to ensure they have considered all of the main features they want in a new system, but also take the opportunity to simplify their IT infrastructure and use an in-memory database that can process much bigger workloads with much fewer hardware resources. By offloading certain applications and processes that are causing the biggest headaches to in-memory systems, organizations can avoid expensive upgrades for legacy systems, traditional databases and hardware appliances. When choosing in-memory database systems, users will find that these products scale extremely well, run far more analyses and allow many more users to analyze the data concurrently. This capability ultimately leads to less automation and less optimization of business processes, allowing companies to focus on creating a data-driven business.
About the Author
Mathias Golombek is the Chief Technology Officer at EXASOL , provider of the world’s fastest database. EXASOL is the first company to combine in-memory with columnar storage and massively parallel processing into its DBMS, EXASolution, giving it unrivaled speed and performance that has never been beaten, according to the independent TPC-H Benchmark. Leading global companies using EXASolution to run their businesses faster and smarter include Adidas Group, GfK, IMS Health, MyThings, Olympus, Sony Music and Xing.