For more than 20 years, I’ve seen consultants and other advice-sellers use an array of techniques to try to come up with a big new idea to take to market. Some of these ideas were total failures. A few were big successes. Most were in between, generating some market interest. And some of these spawned enough business leads to more than warrant the investment in time and money.
I’ll list these methods, from least to most successful:
- Shower inspiration. The light bulb goes off one weekend during a moment of solitude (it doesn’t have to be a shower but there's something about a steady stream of hot water that cleanses the soul). But the ideas that tend to emerge have absolutely no evidence. “Imagine if companies used cloud computing to capture, analyze and respond to every customer complaint logged in the last 20 years? Wouldn’t that lead to great things! …” Interesting but pure conjecture.
- Last project projection. Realizing they need more than an interesting idea, a team of professionals comes off a recent engagement ecstatic about what they helped their client achieve. The client is often ecstatic too. The team invented a whole new approach to addressing the client’s issue and is hungry to publish it. The only issue is that they have a data point of one. That won’t convince many other clients to be the next guinea pig.
- Colleague group grope. A group of professionals in a firm – a consultancy, for instance – meets for a day off-site and is charged with wrestling some emerging market issue to the ground. Since it’s an emerging issue, the group hasn't had much experience addressing it in their client work. But the thought is that a bunch of smart people can come up a great way to solve the problem. They invite a ghostwriter to sit in, listen, and eventually summarize what was said in a white paper. The group grope soon turns into ghostwriter water torture: draft after draft tries to capture and develop the core idea. Since there isn’t one – at least a novel idea built on substantial evidence – this becomes an exercise in writing futility. No one is happy with the drafts (including the writer) but no one is willing to say the obvious: The idea (not the way it was captured) is not very interesting and even worse, lacks the test of evidence: stories of companies that solved the issue a certain way and solved it well.
- Guru gumbo. I borrowed this term from an old colleague (Jay Michaud) from my days at the consulting firm Index Group. A firm convenes a group of external experts from different domains to tackle an emerging issue that none of them has addressed individually. Each expert speaks from his or her base of research and experience, which is substantial. But getting the experts to arrive on a joint solution is difficult because there’s nothing in it for anyone. Moreover, they’re more interested in impressing each other with their wit than in creating something new and exciting. (I’ve actually seen such groups convene. A tremendous expense, and – in every one I’ve witnessed -- a greate waste of money.) The result of the guru gumbo: soup that doesn’t taste very good.
- Survey extrapolation. A professional firm conducts a survey, possibly a lengthy one with hundreds of companies. But given the limitation of surveys – asking people to provide multiple choice answers to complex issues – it is difficult to get underneath the surface of any one issue. The survey gives the firm some good numbers on how many organizations are dealing with the issue but almost none of the important qualitative data necessary to understand just how these organizations are managing it. The result here is lots of numbers and little learning.
- Extensive project immersion. A team is convened over a day or two to debrief on the way they’ve handled a client issue, on which they have dozens of successful engagements. It’s just that they’ve never had a chance to compare notes. This can lead to very fertile discussions and “aha’s” from seeing patterns that weren’t viewable before. If the team does several such debriefs in an organized way and has a competent observer and writer taking notes, very good things can happen.
- Broad and deep case research. A team is given the permission to interview dozens of companies on the same issue over several months, and multiple executives per company. The team also has the liberty to meet over several weeks in an organized fashion to understand what they’ve learned. The team dynamics are good – rank doesn't make one opinion more valuable than another. The most creative and rigorous analysis rules the debate. I’ve seen big ideas come out of this type of thought leadership R&D. (Read more here about why I believe case study research is the foundation of thought leadership.) It isn’t easy but the results – a big idea that becomes a magnet for client attention -- can be substantial.
- Extensive case and quantitative research. Combine great case study research on a couple dozen companies with quantitative research on a couple of hundred and you have a truly deep base of data upon which to generate big ideas. This is rare but great when it happens.
These methods differ in two fundamental ways: the depth of research and proficiency of analysis. By “research,” I don’t necessarily mean how much a professional firm has studied some marketplace issue. The examples could be a professional firm’s client engagements – the consulting firm that has dozen of projects in designing Web self-service customer management systems, or the law firm defending numerous clients against class-action lawsuits. However, if a professional firm lacks deep experience on an issue, it needs to learn about the best way to address it through other means. The best one I’ve seen is conducting interviews with multiple executives at multiple companies that have addressed the issue – companies that solved it and others that did not (to begin to understand what accounts for success or failure).
Proficiency of analysis is rooted in rigor and creativity. Rigor includes the ability to ignore conventional wisdom and reject anything that doesn’t stand up to scrutiny. Rigor is also partly a function of time – how long a team has to study the data in front of it and think hard about what it means. Creativity is important for looking at data in entirely new ways. Given the license to reject conventional wisdom that is no longer valid, a team that is creative is far more likely to come up with a novel idea than one that isn't creative or isn't permitted to question the status quo.
If your firm wants to be the thought leader on a market issue, the most important decision is choosing an “R&D” approach that will get you there.