10 years ago, few executives talked about large language models or artificial neural networks. Today, however, many companies are investing massive amounts of capital, time, and energy to build advanced analytic capabilities.

And yet, often these investments have failed to deliver the results anticipated. It could be declining market share, and our clients are asking us “why?”

My name is Frank Roberts, and I'm in the Energy and Natural Resources practice at Oliver Wyman. 20 years ago, you could have found me in a lab with pipette in hand, but after earning my PhD in Organic Chemistry, I retired as a bench chemist to move into consulting. I still get a chance, however, to stay close to the field by working with companies in the chemicals industry.

Growing up, playing hockey in the Midwest, I was well familiar with the adage of skating to where the puck is going, not to where it is. I’ve heard the analogy used in business, particularly when it comes to advanced analytics. It can be a powerful analogy, but oftentimes results in companies losing sight of the bigger picture in dogged pursuit of the puck.

Advanced analytics and AI seem like a great way to better understand where the puck is going, and they can be, but while this may be provocative, I see two major pitfalls for companies that heavily rely on using models to drive decision making.

First, a prediction is not a strategy.

Predictions and models are important inputs into building winning strategy but are not sufficient alone. We've seen clients’ businesses best succeed when they marry data and analytics with a qualitative understanding of the businesses’ capabilities, assets, and culture that create a defensible market position.

Increasingly, however, businesses have become too dependent on analytics to predict the future and to use these predictions to set strategy. Practically, this might mean that a business creates a new product or expands into a new customer segment based on the output of a model or market intelligence. But continually using predictions to dictate direction can have the unintended consequence of diluting the company's core value proposition. Said differently, companies need to be careful about chasing trends that can turn focus away from understanding and protecting durable, differentiated market positions.

An out of industry example here comes from Apple. Steve Jobs returned to Apple in the late 1990s to find a sprawling, confusing product lineup, in part from individual category leaders chasing perceived industry trends. This unintended complexity ultimately pulled Apple away from one of its core tenants – a simple consumer experience. So, while Apple over the last 20 years has still used data to better understand how its products could be improved, and made bets on the direction of the market, like not having a physical keyboard on mobile devices, Apple has grounded its product development on a coherent strategy that is based on more than just data, modeling, and prediction.

Second, focusing too much on skating to where the puck is predicted to go may mean that you are unprepared to win the puck wherever it does go. That you cannot predict the future remains a truism, and advanced analytics hasn't and won't change this.

The 2008 financial crash, COVID-19, and the war in Ukraine were and are industry shaping events that were not captured in a single model. When working with clients, we advise building capabilities that allow them to sense the market and that provide flexibility to respond to changes.

It can be easy to be lulled into a false sense of security by a predictive model and say, rely on a single supplier of a critical raw material because costs are forecast to be “stable” and the supply chain is “robust.”

We've seen clients avoid the missteps I've just described by first making the key assumptions in a model explicit. Assumptions cannot solely live as a footnote in small font at the bottom of a page.

Second, by defining the operational and strategic implications of boundary outcomes in a model. You'll never get your forecast completely right, so what happens when you don't?

And finally, third, consistently tying initiatives back to the organization's underlying value proposition.

I'm Frank Roberts, and this is my take on advanced analytics.

This transcript has been edited for clarity