Big Data

Turning a tidal wave of information into a competitive edge

As the strategic, operational and financial challenges facing utilities become more complex, these companies are finding it increasingly difficult to make informed, data-driven decisions in a timely manner. The ability to quickly consolidate, analyze and distribute business-critical information to key decision makers is the foundation of an effective data-driven culture. Such a capability could enable utilities to drive improvements in operations, reliability and customer service – leading to more attractive returns on investment.

The digital universe of information stored by utilities in the United States is expanding by an estimated 40 percent per year. This means that from 2013 to 2020 alone, it is estimated that the amount of data available to utilities will increase tenfold. Utilities that can successfully draw insights from the deluge of information can turn their data into assets at a time when they face threats to their core business.

Yet many utilities are historically ill equipped to tackle such enormous amounts of data. While utilities often use large customer and operational datasets, this information frequently is scattered across numerous information systems and organizational departments. Resource-constrained IT organizations are generally responsible for maintaining underlying systems, and these technical resources are often segregated from the subject matter experts who can put the information to use in improving operations.


The real value of stored data, of course, is in linking information across disparate systems and databases to generate targeted insights that drive value. For utilities, this means using their stored data to understand customer behavior more holistically across channels and interactions and to disaggregate complex operational issues, such as outage restoration.

Too often, however, companies try to aggregate their data and simply end up with larger, more confusing and unmanageable data sets. The key to creating value from stored big data is to develop projects with well-defined objectives that focus on addressing questions, and then to build targeted data sets to answer those questions. This requires the involvement of business-line employees and managers charged with the development of business solutions (not just IT professionals) to ensure the creation of a usable system for gathering and analyzing data. Finally, the analytics made possible by the project must be both backward- and forward-looking, providing a better understanding of past actions and a greater ability to predict likely wins.

Harnessing the power of big data in this way can significantly boost a utility’s performance. Recently, we helped a utility improve customer satisfaction and deliver higher service levels more economically by developing insights that were only accessible after linking more than 20 different systems. Many utilities face these challenges, since customers are demanding more services across multiple touch points and communication channels, while cost pressures continue to rise. Marketing departments also struggle with matching up the right customers with the right offering and what programs should be directed to which customer segments at what time.

By “connecting the dots” through a directed, collaborative process that involved both business line units and IT, our client obtained a holistic view of its customers, across multiple interactions, different departments and different channels for the first time in its history. This view enabled the client not only to improve customer satisfaction, but alsoto unearth a number of multimillion dollar cost savings opportunities: The company identified a segment of high-value (and likely receptive) customers who had not yet been moved to electronic billing – representing millions of dollars in annual savings – and then produced the customer-level detail needed for the marketing department to effectively target them.


The right subject matter experts (typically business or operational users, not just IT) need to be involved intimately in the design of the analytic scope and implementation of targeted databases. In too many cases, we have seen big data decisions driven by IT, with limited involvement of end users. Without business users engaged early in the project, the end result will lack the structure, clarity and ease of use critical to timely, effective decision making. Throughout the development and implementation process, business users should continue to use two considerations as guideposts: “What problems will this project address?” and “What outputs do you need to solve this problem?”

Big data issues do not arise from a lack of information, but from an abundance of inputs. Deciding which inputs are necessary is critical to initial planning and to enable quick identification of scope creep. In addition, understanding where potential data linkages can be made between systems, even if these links are not perfect, allows business users to pose questions that previously could not be answered.

The scope of big data projects tends to grow as people attempt to encompass all possible information. This can lead to an overly complex, hard-to-use IT “solution” focused on the mechanics of data storage, rather than the end uses of the information. This risk must be recognized and addressed, as the inclusion of multiple data sources increases the complexity of project implementation as well as the effort required for ongoing maintenance, all while generating limited value for the organization.


Utilities should develop analytics using big data that allow them not just to react to problems – but also to steer clear of them. Making use of big data is like driving a car at night: The headlights represent predictive analytics and the rearview mirror represents reactive analytics. You need both to drive sensibly, although the choice of how much to rely on each will depend on the situation. Similarly, for utilities, both predictive and reactive information is necessary, but the right blend depends on the company and the task. (See Exhibit 1.)

The problem is that many utilities right now are driving blind, or nearly so. Utility “data marts” tend to be geared toward rearview metrics, and the IT function generates reports that track past performance, such as monthly trends or year-over-year comparisons. But a utility can make use of its stored data to develop predictive analytics, which can allow users to quickly test “what-if” scenarios and identify or improve opportunities that complement existing systems. Rearview analyses can then be used to monitor performance against new opportunity areas and tweak future opportunity identification.

This blended approach requires an iterative process to determine how the different pieces of information can work together. Reactive methods can draw on past lessons to influence future decisions, while predictive methods can send the utility in new directions, tempered by reactive input.

In summary, ensuring real insights into complex business questions requires analytics that put business solutions – not data or systems – at the forefront. The right decision-support tools can empower users – with help from IT – to continue evolving analyses and reports as business issues change. Utilities do not need to cross an ocean to find opportunities to improve returns. They just need the right tools to unleash the power of the tide of data coming their way.


Big Data