A call for wiser research on collective wisdom

Research has just shown that “The Wisdom of Crowds” phenomena can be biased – a bias that can be avoided by focusing on the responses of “confident” responders and ignoring everyone else. While this is interesting, it neglects a number of important points, such as (a) how the whole process is limited to questions that have a single right answer (which, it turns out, the researchers failed to do); (b) problems with “confidence” as a source of accurate information; and (c) the fact that making guesses about facts or predictions about the future is but a tiny part of the full reality and potential of collective wisdom – and focusing on that tiny part is distracting us from our urgent need to develop more comprehensive and powerful forms of collective wisdom and to apply them to our current global predicament.

James Surowiecki’s book The Wisdom of Crowds popularized the notion that remarkably accurate predictions and guesstimates regarding a question with an unknown or little-known answer can be produced by aggregating the answers of a large number of diverse, independent people – “the crowd”.

Two Spanish researchers recently proposed a modification of that thesis. They observed that bias can be introduced into the crowd – for example, in the form of supposed expert or majoritarian responses to the question – which influences subsequent crowd responses and degrades the accuracy of the final result. They further noticed that, when that happens, some responders maintain their previous views while others are easily swayed by the biased information. When they excluded the more easily swayed participants and only aggregated the views of the “confident” ones – the seemingly more independent thinkers – they produced what they claimed was better “collective intelligence” than when they aggregated everyone’s answers together.

Several articles describing this research wonder how we might apply this new discovery – especially online, as in Reddit, where people rate online articles and stories for “importance” and influence each other as they vote, through being exposed to the evolving results of their collective voting activity. This kind of influence is exactly what the Spanish researchers say biases the final result and therefore we should seek to avoid. But how do you identify the independent thinkers – the “confident” voters – so you can aggregate their votes to the exclusion of everyone else’s?

That’s a fair question, but may even be irrelevant.* I think a number of more important issues are raised in the comment section below the research report. Here are several I’d like to highlight:

1. We can only assess the accuracy of the crowd’s guesstimated “wisdom” if the answer to the question posed is a mathematical or factual truth that has only one answer. If the question’s answer is a matter of opinion or has a range of possible answers – like Reddit.com’s “Which article is most important?” – there is no solid standard against which to judge the supposed “accuracy” of the aggregated ratings. The question presented to participants in the Spanish research was “What is the length of the border between Switzerland and Italy?” – a seemingly objective question for which the researchers claimed the correct answer was 734 kilometers. But one insightful commenter noted “the actual length of the boundary between Switzerland and Italy depends on how you measure it. Measurement with a chain a kilometer long would give a different answer than a meter long stick. I’m also going to guess that the length given is based on a pretend situation where the world is flat and neglects the elevation of the terrain.” (This so-called “coastline paradox” was first noted by Lewis Fry Richardson whose work contemplating the length of nations’ borders became a foundation of fractal mathematics when Richardson’s research was referenced by mathematician Benoît Mandelbrot in his 1967 paper “How Long Is the Coast of Britain?”) So “the wisdom of crowds” approach actually applies to a very narrow and specific group of questions – notably excluding inquiries that have any significant complexity or meaning associated with them.

2. It might be wise to question the value of “confidence” as a reference point for collective intelligence or the accuracy of guesstimations. Research indicates that confidence is an emotion that varies independently from the accuracy or truth of the views about which people feel confident.  Perhaps even more importantly, wisdom and even intelligence involve a level of uncertainty, even humility, in order to keep ourselves open to contrary or additional information which, if investigated further, would modify our views and increase our understanding. A wise person may, in fact, be less confident because they see a more complex picture than their overly confident fellows, whose confidence arises from their oversimplification of what is actually involved. This reality provides a provocative contrast to the Spanish research and invites some additional creative research to clarify the seeming contradiction between the two.

3. More fundamental still is the issue of whether an aggregated guesstimation of any sort actually constitutes either “collective intelligence” or “collective wisdom”. I would propose that there is much more to intelligence than guesstimates – and much more to wisdom than intelligence. Intelligence includes an interdependent assortment of capacities that enhance our ability to function in the world, including

• pattern recognition and conceptualization;
• knowledge-creation, understanding, and all forms of learning from our experience and from others’ knowledge;
• memory;
• reasoning, logic, and problem-solving;
• judgment, practical sense, planning, intuition, creativity, and other dimensions of applying existing knowledge to new situations

– to name just the most obvious dimensions of intelligence.

Ultimately intelligence is a capacity – and idealy an ongoing exercise – that mediates our success in dealing with the changing conditions of our lives – our evolutionary “fit with our environment”. But intelligence often gets applied narrowly, missing important facts, dimensions, perspectives, or possibilities that often come back to haunt us later. The remedy for that limitation is wisdom – the exercise of intelligence in ways that embrace a bigger picture, to see and respond to more of the whole of life and of situations, to generate broader fit and benefit for more of life over longer periods of time.

The study of collective intelligence and wisdom is, then, an inquiry into how groups, networks, organizations, communities, nations, systems, species, and other collective entities manifest and accomplish all these capacities for optimum survival and success. And today this subject must especially be about how we humans – in our collective lives and systems – manifest all these abilities to more effectively meet the unprecedented challenges and opportunities of our current era.

So the upshot of all this is that the Spanish researchers’ discoveries have a significant but very limited utility – correcting a glitch in the overall “wisdom of crowds” practice of collective guesstimation. However, the whole “wisdom of crowds” phenomenon is only a tiny part of collective intelligence and wisdom when those capacities are viewed in their fullest – and most crucial – manifestations.

COLLECTIVE INTELLIGENCE AND CONVERSATION

I take issue with another major assumption of the “wisdom of crowds” thesis advanced by James Surowiecki, author of The WIsdom of Crowds – specifically, his bias against conversation, dialogue, and deliberation. Harri Oinas-Kukkonen summarizes that assumption as follows: “Too much communication can make the group as a whole less intelligent.”

This principle exhibits a profound ignorance of the varieties of communication and conversation – an ignorance that prevents researchers in the field from even glimpsing – to say nothing of clarifying – more comprehensive and authentic forms of collective intelligence and wisdom. Most forms of collective intelligence and wisdom are deeply dependent on the interaction of diverse entities, usually in the form of conversation.

When Surowiecki and his followers speak out against communication among the guesstimators in a “wisdom of crowds” exercise, what they are actually speaking out against (without realizing it) are interactions that reduce the level of diversity in the system. What produces the crowd’s accurate collective answers is aggregation of its non-manipulated diversity. This is one way to “use diversity creatively” – a central feature of collective intelligence. But this “wisdom of crowds” aggregation approach is limited to getting collective answers to questions of fact – including predictions (future facts) and currently unknown facts (like the location of a sunken submarine).

But what about real collective problem-solving, especially around a public issue? What about collective learning from collective experience, or developing broad public understanding from academic and scientific research, one of our civilization’s most powerful (potential) forms of collective learning? What about deepening the nuances and expanding the applications of collective understandings we already have as an organization or society? What about the accumulation of collective memory or the making of collective judgments according to shared values? These are the kinds of collective intelligence we need in our world, not just guessing how many jelly beans are in a bottle or what the price of a bushel of corn will be next year.

And then there’s wisdom – the expansion of intelligence into the depth, breadth, and heights of life and reality… the creation of workable solutions that bring broad benefits to most of the beings involved in a situation…. the full acknowledgement of and creative engagement with life’s uncertainties and complexities… all the exercises of our cognitive capacities that usefully stretch us into the Big Picture beyond the personal, the immediate, the obvious.

We don’t get these kinds of collective intelligence and wisdom without adequate diversity AND interactions – which usually means conversations! – that use diversity creatively. So the kind of research we REALLY need is what KINDS of diversity and what KINDS of conversation can generate the most useful and powerful collective intelligence and wisdom? Because there ARE conversational approaches that counter the conformist dynamics Surowiecki and his followers rightly critique and that maintain the independence and diversity they properly value. There are conversational processes that go even further, cashing in on the gifts of people’s diversity – and even their conflict – while channeling group energy towards greater collective understanding and co-creativity.

Could even more powerful forms of conversation and understandings of collective intelligence and wisdom be developed? Surely! But only if the wisdom of crowds crowd – and everyone else who thinks that “prediction markets” or computer programs or meditation are the epitome of collective wisdom – can climb out of their tiny-box worldview and join in the larger ecosystem of inquiries and possibilities concerning the full scope and potential of collective intelligence and wisdom.

What we DON’T want is to reduce the comprehensive and authentic capacity of collective intelligence and wisdom to a bunch of spectators guessing about isolated facts or outcomes. While it is undoubtedly a piece of the bigger picture of collective intelligence and wisdom, it is SUCH a reduction of our potent collective capacities and offers us so little help in co-creating the kind of world we need – including co-creating our way out of some truly horrendous messes we are currently getting ourselves into as a global civilization! This is not a time to be thinking in narrow bands that leave us at the mercy of our own limited capacities.

In my next post – “Factors that support collective intelligence and wisdom” – I’ll explore factors that support not only collective intelligence and collective wisdom, but collective stupidity and that partial intelligence represented by The Wisdom of Crowds.

Coheartedly,
Tom

* There are solutions to the biasing dynamic that ignore the confidence factor and simply prevent respondents from seeing the aggregate votes before they themselves vote. A variant – used by sites like codigital.com – solicit preference votes on random pairs of items – e.g., “Which one of these is more important?” – without revealing how other respondents voted on those items until later.

= = = = = = =
Note: For a prior discussion of these issues, see “Comparing ‘the wisdom of crowds’ to real collective wisdom”
= = = = = = =

How to Get Wisdom from the Crowd, Divide by Confidence
by Chelsea Kerwin
August 4, 2014

The article titled Forget the Wisdom of Crowds, Neurobiologists Reveal the Wisdom of the Confident on MIT Technology Review engages with new research on collective opinions in a group of people. The concept of the wisdom of the crowd posits that a group of people might be better able to form a correct opinion than an individual expert. Does this sound wrong to you? The article states, “This phenomenon is commonplace today on websites such as Reddit in which users vote on the importance of particular stories and the most popular are given greater prominence. However, anyone familiar with Reddit will know that the collective opinion isn’t always wise…It turns out that if a crowd offers a wide range of independent estimates, then it is more likely to be wise.”

Once biased is introduced to a crowd, however, there opinion is much more likely to be “stupid.” Scientists believe that by separating confident thinkers from more sheep-like thinkers allow for a greater chance of the groups arrival at wisdom. Anyone who has played a trivia game on a team already knows that the most confident person is often trusted over other hesitant players. But what happens when this notion is applied to search and retrieval? Scientists are still working on applying this information to real world scenarios.

************
Final report on the Co-Intelligence Institute’s June-July fundraiser:
74 people donated $15,255. Big “THANK YOUs” to all who responded..
**********
________________________________