Why innovation is so hard

Chances are, innovation doesn’t work where you work—or only works some of the time, mostly in spite of your organization’s system and processes. Why? Because you don’t understand what makes the innovation game so different from everything else you do at work—and you haven’t adjusted your playbook to accommodate these differences.

The good news is, you’re not alone. Too many of the best leaders still get a lot wrong when it comes to the innovation game. Getting on the path to mastery requires a deep understanding of what we don’t understand. Here are five truths about innovation that most of us get wrong—and how to get it right:  

Data gets you nowhere

Big data is the new big thing. Collecting and interpreting information to predict future possibilities is useful in many ways – hurricane warnings, consumer confidence ratings and disease outbreak forecasts come to mind. Given access to abundant computation power and a functional algorithm or two we can simulate how the various complex components of an event will interact and possibly even predict its timing.

But what about cases where the causes are not well understood and there is a lot of variability? Will consumers in Russia buy this new soft drink? Will this new molecule really cure the dreaded disease? Will this new security system keep us safe from hostile forces? The greater the magnitude and volatility of the forces that drive the future, the less likely you are to get it right.

You hold assumptions that the future will function like today. In 2001: A Space Odyssey, astronaut Dave calls Earth from Jupiter – from a phone booth. Even the brilliant Arthur C. Clarke missed a few things. Stop collecting excessive data. It’s a form of resistance because it stops you from running the meaningful experiments. Better to make a little, sell a little, and learn a lot.

Failure is inevitable

There are two kinds of parents at the playground – the “don’t do that” kind, and the “hurt, didn’t it?” kind. Which approach helps the kid learn faster? Ouch! That’s because all learning is developmental. Don’t believe it? Take out a piece of paper and draw a picture of your dog. Your friends and family will be able to tell you at what age you stopped learning to draw. Speak a foreign language or play an instrument and you get the point. No matter your age or professional status, if you haven’t done it before you will inevitably fail before you succeed.

So how do you fail without becoming a failure? Learn from venture capitalists and treasure hunters: hedge your bets. Diversify your projects. Take multiple shots on a single goal – small, wide and short. Fall in love with the problem; not your solution. Inevitably it won’t be quite right, at least right now. Fail early, often and out of sight.

Innovation happens from the outside in

The primary function of the operating rhythm of an organization is to maintain equilibrium. You march to beat of efficiency and quality. Variation is the enemy. The problem is that innovation is produced through variation. And each time you launch anything “abnormal,” the organizational antibodies attack it – hurdle rates, capital committees and passive aggressive human resourcing.

Revolutions start from the outside in. It’s the dissatisfied fringe that shake things up because for them the risk of doing something radically new and the reward of keeping the status quo is reversed. Think about the insolvent Apple of the late 1990’s. Think Gandhi and his march to the sea. Think Bob Dylan – “When you ain’t got nothin’ you got nothin’ to lose.”

Forget the middle of the organization where middle managers patrol the borders. Do what great doctors and researchers do. Take on the difficult case and the lost cause. The middle will avoid these like the plague because there is no antidote. This will give you some much needed operating room. Do your best to correct the problem or cure the disease and learn what works and doesn’t. Quickly adapt winning strategies, develop methods based on your conclusions and apply those everywhere. Soon miracles become replicable and scalable. Start at the outside of the bell curve and work your way back to the middle. There is no defense for success.

Innovation is a product of constructive conflict:

Spend an afternoon at a world-class biotech lab or drop in on a faculty meeting at top-flight university, and you can almost feel the intellectual pushing and shoving. At its worst it feels like politicians in Washington trying to destroy each other just to get the upper hand. At its best it feels like Michelangelo and the golden guild of Florentine craftsman wringing a masterpiece out of flawed slab of marble. The positive tension pulls the nascent creation like taffy until it morphs into something sublime. Too many leaders focus on alignment first. But alignment comes last, at the end.

So what if you don’t live in Cambridge or Tel Aviv or frequent artists colonies? Chances are you can enlist some folks who worship in weird ways, eat exotic fare and “think different” than you. Of course they’re annoying because they don’t want to do things the way you do them – the right way. But maybe they can be the antithesis to your thesis. You can’t have astounding synthesis, hybrid solutions, or the gee-whiz science fair winning gizmo without some antagonizing competition. If you don’t have a loyal opposition it might be time to create one.

Innovation happens in cycles, not straight lines:

If you’ve ever trekked up a winding mountain path you know that the vistas get more splendid as you climb higher. With each step, more and more is revealed. You get smarter as you go along and begin to see what you could not possibly have noticed from the ground. The tragic mistake of the novice climber is in believing that you can see over the next summit. Meanwhile, a sheer precipice may quickly put an end to even the best laid plans.

History is filled with stories of those who chose to believe in their strategy even when their senses and experience told them otherwise. The only real difference between the Donner Party and Lewis and Clark is that at some point the latter listened to unconventional guides, made dramatic adjustments to their plans and redrew their maps on the fly. And yes, luck always plays a part, but only for those who are open to it. Innovators who live to tell the tale of their great adventures, seldom move in straight lines.

Practice circling your innovation project - double back and reconsider what you may have overlooked. Build prototypes and see what works and doesn’t. Divine some simple rules of thumb, and only then start adding on with each cycle. Let the optimization wonks stick to the metrics. They seek alignment. You need to adjust your key indicators as you traverse the undiscovered country. You seek valuable variation. Be skeptical of portfolio and stage-gate processes that promise to accelerate innovation. These tools assume that you know everything you need to know at the start of the journey. Make enough laps and everything old becomes new again and vice versa. Don’t believe that once you take a higher point of view others will follow. Those who see the future first are always assumed to be delusional because in some ways they are.

Want to make innovation an everyday, everywhere capability in your company? Read, rate, and share the innovation case studies from the Innovating Innovation Challenge on the MIX.

Jeff DeGraff is Clinical Professor of Management and Organizations at the Ross School of Business at the University of Michigan. To learn more about Jeff and his work on innovation please visit www.jeffdegraff.com. You can follow Jeff on Twitter @JeffDeGraff and Facebook at DeanofInnovation.