From the Startup Frontline: The Perils of Pilots

Editor's Note: This article was originally published on Oliver Wyman Insurtech.

Oliver Wyman Insurtech caught up with Oliver Wyman Partner, Jim Fields, who’s been fully engaged with a healthcare startup, Practicing Wisely®, which evaluates physician performance on evidence-based treatment measures. We thought that the lessons he’s learned as Practicing Wisely went into the marketplace would be applicable to many insurtechs in the same position.

Armed with the pitch and the product, Jim set off on a path familiar to many at this stage: securing a pilot program at a larger organization. The benefits of pilots are incontestable: a real-life test of the concept at a potential client that provides the magical “foot in the door.” The pilot introduces the idea to the client, proves the value of the offering, and in theory, provides a win-win for both parties.  

Not surprisingly, there’s a lot of “eagerness” involved in the pilot process, on both sides.  And it’s precisely in that eagerness that the peril lies.

Once you’re in, you’ve caught the tiger by the tail.

On the side of the startup, the eagerness to get something started can mean you don’t want to create any hurdles for someone saying “yes.”  Sometimes there’s even a “let’s get started and see how things work out” attitude. On the side of the potential pilot organization, there’s the promise of something new, of an outside perspective on a familiar issue, typically at a price point that’s comfortable (either low or sometimes even cost-free).

But, smiling ruefully, Jim adds, “Once you’re in, you’ve caught the tiger by the tail.” The initial eagerness can easily lead to what Jim calls “death by pilot.” You may get the pilot, only to find out that your collaborator doesn’t have the buying authority to do something bigger. Or the pilot simply lingers in limbo as the client doesn’t have clear decision processes or a strong leader who can help the organization change its ingrained approach. In some cases, there is demonstrated value but the pilot doesn’t really go anywhere. Time passes and you’re really no farther than where you began.

Jim’s lesson learned from the experience of the last year? Remember that “large organizations move slowly, and startups cannot move slowly.” Driving change of any magnitude in a large organization is hard; when you’re taking on big challenges, like physician performance and actionable care measures, it’s even harder.

So, should startups scratch pilots entirely? No. From his experience structuring pilots, some that “worked famously” and some that “went no-where,” Jim counsels that both sides come into the discussion with explicit objectives and embrace the notion of “bounding” the pilot. Jim recommends:

• Define specific and near-term milestones and outcomes, and don’t be afraid to make the overall project smaller as a result (a smaller pilot with achievable milestones accelerates the pace for expanding the relationship and builds momentum)

• Create a joint, clear map of stakeholders (and decision makers) on both sides

• Require that a champion with buying authority be involved – and make an update to them a contracted milestone

• Articulate what the client’s obligation is (and what the startup’s obligation is)

• Define what success means in specific terms and if possible what next steps would be, following a successful pilot

Anytime you are doing something new, the market wants to see proof that it works.

“Everyone comes into a pilot with the best of intentions,” Jim points out. “But don’t expect they know how to follow through unless you map it out.”  And remember they may decide not to go through with it.  “A clear ‘yes’ is great,” Jim says, “But I’ve learned to appreciate a quick ‘no.’” The worst is an environment where you are matched up with weak leaders from the client and you get bounced around with lots of compliments but no one with the power to do anything.

Jim shares that, “Thankfully the pilot phase of your growth agenda will come to an end one way or the other. Either you will fail to make the leap from pilot to program and need to cut bait or, you build a track record of successful clients that allow you to jump straight to deploying the offering without the need for a pilot.”

So, what about Practicing Wisely? Jim acknowledges, “We had our share of failed pilots early on. We didn’t clearly define the objectives or engage the right leaders. But we learned from those mistakes and we were more purposeful about our next round of pilots.” And how do you use those failures in a constructive way? “We learned from the early pilots to be more clear on the objectives, expectations and in picking the right partners…and those later pilots have been terrific proof-points,” observes Jim. “Where the clients have partnered with us in telling the story of the program’s impact, they have made Practicing Wisely a key part of how they ensure evidence-based and cost-effective care for their members and patients.”

So, the lesson is not to avoid pilots, but to do them wisely. “Anytime you are doing something new, the market wants to see proof that it works. That makes pilots critical to the success of a start-up. Having done pilots the wrong way, learning painful lessons, and ultimately putting those lessons into practice with better results, I hope we can accelerate the success of other start-ups.”

About Practicing Wisely

Practicing Wisely is a novel health start-up, incubated as a collaboration between Jim Fields of Oliver Wyman and Dr. Marty Makary of Johns Hopkins. Practicing Wisely addresses two significant healthcare problems in the US: Unnecessary care, which costs about $2.65 billion annually, and medical errors, which is the third leading cause of death. Over the past three years, it has grown it into an impactful analytic solution that has attracted leading healthcare customers including a national healthplan, Blue Cross Blue Shield plans, risk-based provider organizations, and notable health innovators.