Using the Wisdom of Your Crowd

In his famous 2004 book "The Wisdom of Crowds", Surowiecki told us that in the game show "Who wants to be a Millionaire?", the Ask the Audience lifeline got questions right 91% of the time, far better than Phone a Friend. Surowiecki cites lots of research giving examples of how the combined independant opinions of a group of lay people have much better predictive power than one expert judgement.

We trust the same mechanism in jury trials- 12 laymen make better calls than the expert policeman.

But in business, unless the definitive facts stare us in the face, we mostly seem to default to accepting the intuition of someone in a leadership role.

The leader will have gained respectable expertise on their path to becoming a leader, but Tetlock's ground-breaking 2006 study "Expert Political Judgment" clearly demonstrated that acknowledged experts are just a bit better than random at predicting outcomes in their sphere of expertise, and substantially less reliable than just saying "no change".

The leader probably also takes advice, but Yaniv in 2004 showed that people assign rather low value to others' advice, especially where they feel they already know something about the subject. See et al in 2011 studied individuals' relative power and summarised that "higher power participants were less accurate in their final judgments".

So it seems clear that, if we can't get at all the facts in a reasonable time, we will get better decisions if we combine the judgements of many people.

How do you tap the wisdom of your crowd? It is very easy in principle, just 4 steps:

1. Decompose the problem
Breaking any problem down and looking at each part separately has been shown to increase forecast accuracy, so it's a natural place to start. It also allows you to ask different crowds about different parts of the problem, respecting both their interests and their time available to participate.

2. Identify your Crowd
You have a ready crowd around you- people who already speak the same business language, but differ in their perspectives. So no need to go out on the street and educate completely lay people. But make sure you get sufficient diversity in your crowd. If you can, involve customers and suppliers as well as colleagues.

3. Pose the Questions
You need to ask your crowd questions that they can answer (from their judgement, and whatever facts they have at hand). You also need to ask questions such that many answers can be combined. Mostly that means questions that can be answered with a number or a few numbers. With small crowds, a great approach is to ask each person for three numbers representing optimistic, most likely and pessimistic out-turns. These three together let you measure your crowd's confidence in their ability to predict, alongside the prediction itself.

4. Aggregate the answers
This is the trickiest part to do well, and I explain it in more detail in another post.

There is a fish-hook in this. Other research tells us that well-adjusted people are habitually over-confident, both in their ability to predict and in the outcome they predict. I will write another post on how you minimise the effect of over-confidence.

There is a printable PDF infographic depicting AcuteIP.com's approach to using the Wisdom of your Crowd

With enough preparation, you can do this yourself. AcuteIP.com offers a set of services to make it easy, from workshops and technical services to managing the whole project.
Contact Graham.Harris at AcuteIP.com to explore possibilities.

Is this post useful to you, or could it be useful to someone you know? Please do us both a favour- spread the word by sharing it through the colourful social media buttons at left or below.

(c) 2018 Graham.Harris at AcuteIP.com