Measure Intangibles

Give Everyone a Voice


I had a role in investing quite large amounts of money- up to $50m a year- on behalf of my employers. So I have been up close and personal with the decision making around hundreds of millions of dollars. And at times, it was not pretty.

Some of the decisions were effectively made at far too early a stage. For example, getting an investment idea listed on a 3-year road map can start a train rolling that's very hard to stop later. People can become invested in the idea, and challenging it seems disloyal or argumentative.

But it's important to remember that such long term plans often result from a very brief process. Perhaps one person thought it was a good idea, and no real evaluation was done of costs, risks and benefits.

So before you commit real money, resources and opportunity cost to something, you need to apply more rigour to investigating it. And in my experience that means consulting a much wider range of stakeholders.

My view has strong support in the 2004 book Wisdom of Crowds by James Surowiecki. A trivial example of his gives the idea: the US TV game show "Who Wants to Be a Millionaire?" offered contestants three lifelines to help answer its progressively harder questions;

  • take away two wrong answers (giving you a 50% chance to guess right)
  • Phone a Friend (expert judgment)
  • Ask the Audience.

Turns out, Phone your Friendly Expert got the right answer around 65% of the time, but the majority vote of Ask the Audience (random crowds of people with nothing better to do on a weekday afternoon than sit in a TV studio) picked the right answer 91 percent of the time.
There are many other examples, independent of Surowiecki; for example see the crowd experiment by the US National Public Radio .

Thing is, you have quite an audience among your colleagues. And they already know something about the specialist subject, so they are just as equipped to answer questions about it as the TV studio audience was about general knowledge questions.

You don't need to ask all your colleagues- at least, not if you work in a medium or large organisation. You can identify a few people with a diverse range of interests in the topic- diversity is the key- and ask them to nominate others. Diversity is the opposite of the the process by which the item got onto the road map, so we're immediately getting somewhere.

Once you have your audience, you need to ask the right questions to get the right answers. Not the answers you came looking for; the right answers. You can control for potential biases and other shortcoming by asking questions the right way, in the right order.

This can all be done quickly and efficiently, if you are clear about the process. It doesn't depend on key individuals being available: the essence is that the collective knowledge of the many gives you more than any individual could.

Have you got experience of decisions that could have gone better with wider input? Or experience that sheds another light on this issue? I welcome your comments.