My colleagues and I sat with a division president of one of the largest financial services firms in the world about one year ago, when we heard the executive ask her direct reports, “How many customers do we have that are eligible for our new product line?” The assembled executives sat silently for a few moments, looking side to side at one another, before one spoke up: “We don’t really know. We have been asking this question for months and have been told that it will take us six months and $5 million to integrate, correlate and analyze the data to get us this answer!” Sound incredible? Sound far-fetched? Well, then, welcome to the real world!
Corporate executives across industries are faced with this and similar issues every day—how do we get answers to critical business questions in a reasonable amount of time? What should be simple, intuitive and obvious is instead often astonishingly complex and counterintuitive. Why is this so, and why has it become the case?
The Proliferation of Data
The past few decades have seen an astonishing proliferation in the amount of data that organizations generate, maintain and store. Historically, the costs of maintaining and storing data have been high, but as costs have fallen dramatically, it has become possible for nearly anyone within the organization to build and maintain copies of critical business data, resulting in proliferation of the volume and varieties of data that businesses manage. Hence, we have reached the Democratization of Data, where the rate of data growth is far outstripping Moore’s law—more than 100% annually and accelerating.
Writing as experts who have each worked with data for three decades, we expected the problem of capturing, organizing and maintaining business data would get better as new technology and business processes for managing data were developed. Instead, the challenge and problem has become worse—much worse. In addition to traditional customer and operational account and transactional information, companies are now witnessing an astonishing growth in new types of data: unstructured data such as documents and pictures, sensor data that monitors and measures movement of individuals and equipment (think GPS), and social-media data such as Twitter and Facebook messages.
But the biggest challenge results from the ability of traditional-business analysts working in a company to very simply and easily create and maintain their own version of data. Why be dependent on centralized production systems when you can formulate and ask your own questions more rapidly and more cost effectively?
Making Better Decisions, Faster
If time is money, then the ability to accelerate the time that it takes to ask and obtain answers to critical business questions is priceless. Organizations have been grappling with the issue of developing systems to improve executive decision making for several decades now, and it has gone under different names at different times: decision support systems (DSS), executive information systems (EIS), business intelligence (BI) and various forms of analytics. The latest incarnation in the evolution of this process of gaining business insight from data is big data. Yet, although big data may be a new term that encapsulates many of the efforts to manage and learn from data that have been underway for decades, there are aspects to big data that are new and even revolutionary in the breakthroughs they provide.
Our recent survey of CIOs and senior industry executives highlighted the commitment of large organizations to using big data to “improve their data-driven decision-making,” and most importantly, to improve their ability to make better decisions, faster.
The real breakthrough that big data offers these executives is the ability to accelerate the time it takes for a business manager to answer a question. The business metric is known as time-to-answer (TTA); it enables business managers, through big data approaches, to answer critical business questions in seconds rather than days, days rather than weeks, and weeks rather than months. This improvement happens for two reasons:
- Business managers can now ask questions of all of their data, something that has not been possible in the past. This is because the new big data technologies make it possible and affordable for business managers to load and access the full set of available information. This approach is commonly known as “load and go.”
- Business managers can ask more questions, and can ask new questions as they go, because big data technologies defer the need for data engineering, which has been an obstacle to business managers’ ability to ask questions. The net result is that business managers can now ask more questions without waiting and, as a result, get faster answers because they are largely freed from dependency on a technical team or IT organization. This enables greater self-service.
Rather than wait 6 months, and spend $5 million to understand which customers are eligible for a new product, like the executive in the opening example, companies are putting in place today the capabilities to answer this question, and many other questions quickly and at minimal cost.
This process is at the heart of big data initiatives across industries and firms.
Answering Critical Business Questions Using Big Data
- How many customers do we have?
- What are sales for our product lines?
- What happens if we redefine customer segments or product lines—how do these numbers change?
Business executives and senior managers are asking these questions each day, and surprisingly, they are precisely the kind of fundamental business questions that companies are using big data approaches and capabilities to address. As simple as these questions may appear, there are complexities to each question that belie a simple answer. What constitutes a customer or product line can vary depending on the perspective of the questioner. Is a customer an individual, a household, a direct client or an indirect client?
For this reason the need to ask questions from fresh perspectives becomes critical, and it is why the need to act quickly becomes essential. Big data approaches enable business managers to ask more questions, looking at more data, without requiring technical know-how or depending on an IT group. This strategy reduces time and cost, and although some data engineering is simply deferred to later, the power and the beauty of big data is that it promises business managers the ability to ask more questions and to iterate through these questions more rapidly—in short, the ability to get fast answers to business questions.
About the Author
Randy Bean is Managing Partner at NewVantage Partners.
Photo courtesy of Kevin Krejci