Wisdom of the Crowds

I've heard a lot about the power of the web and how it can be used to harness the "Wisdom of the Crowds." However, I think that concept is a little thin on follow-through. I believe it could be better stated that the power of the web is to find the "wise ones among the crowds."

I lack evidence to the refute that the sum of average minds is greater than any single exceedingly wise individual, however, I also counter there isn't a proof-positive example either.

A crowd could be considered wise when it is obvious to everyone in the crowd that they have come to the correct solution to a problem (and in fact they are right, which isn't always the case). In order for a crowd to be wiser than each individual, each person must contribute some small amount of information leading to the correct answer. Each incremental piece of information must also be assumed by the other members to be correct without significant scrutiny. In some aspect this requires a problem that is addressed by the crowd to be something similar to a labor task. Building a pyramid is a monumentous task where each person's incremental labor is obvious and trivially accepted by the next laborer assisting with putting up the next row of blocks.

Intellectual problems are rarely a case of brute force labor where like minds can march in lockstep to add a brick at a time to the cause. In general significant academic or industrial problems are usually solved by a few specialized or highly intelligent individuals that may collaborate in small groups. The power of the web to create seemingly wise crowds may in fact be that it bridges the gaps between people who are either highly intelligent or highly specialized. While there are also typically a large number of people who are interested in solving difficult problems, they may lack the time, specialty or intellect to solve them.

However, in contract to the intellectually challenging problems, I bet there are examples of problems where the average minds focussed on relatively straightfoward problems can produce results. I have heard of one intriguing problem where supposedly the CIA or some U.S. security agency was going to make public records it has found on weapons of mass destruction in Iraq. The problem is a large one in that there is a gigantic amount of documentation that is in (to the U.S. at least) a foreign language. The vast majority of this documentation is likely to have nothing to do with weapons of mass destruction nor would be helpful in actually finding them even if they are on topic. A large crowd of dedicated people could spend their energy each contributing to mining the documentation for information. A positive response could be validated pretty easily by other members of the group under several assumptions. The assumptions include:

• everyone searching is faithful to the same cause
• the evidence they find to support is contained within a small subset of documents (for example no more than 3 or 4 needed to piece together some proof)

A group working on a problem like this probably doesn't need too much organization. That is, it needs to be able to reject members who based on some amount of feedback by other members are deemed not faithful to the cause. For example, a person reporting every page as evidential. The group would not need to know what other members are working on because duplication of effort should be considered trivial and also the evidence may have been missed by a single reviewer or possibly ignored by an unfaithful member.

It is the power of the web and some group dynamics and leadership that would harness the power of the crowd, but its widsom is likely safe with the relatively few individuals that possess it to begin with.

Anyway, I got to thinking about this some because of work and I happened to also think about a quote from Albert Einstein that I see as part of a signature on the game posting board:

"He who joyfully marches in rank and file has already earned my contempt. He has been given a large brain by mistake, since for him the spinal cord would suffice."

I wonder what he would think of the Wisdom of the Crowds since hardly any of the crowd understands the theories he created other than meagerly quoted "E equals M C squared!".