One problem with the assumption that crowds work is the independence assumption. Particularly with regard to markets, people don’t buy a stock as a guess as to what the stock is going to do, per se. That is, I don’t start by saying that IBM should be $50/share, and if it’s not then I’m going to buy/sell it. Instead, I’d guess that how it works is that I make a guess as to what other people are going to think that shares of IBM will be in the future. That is, it doesn’t matter what IBM should be at, if $35/share is what it is at. The assumption that markets trend to efficiency over the long run may or may not be true, but knowledge of that underlying ‘true’ or ‘natural’ or ‘fundamental’ price of IBM is unknowable.
If markets are a combination of what the thing is worth, and what others think the thing is worth, then independence will be impossible to come by. One way to test this – take the same future guess/prediction, and ask people to guess at the outset what the final number will be (so, what a movie would gross in its opening weekend, or how many innocentive challenges will be solved by June 1). Then take those same people and let them at an information market on the same prediction. Which should do better?
If the answer is the latter, why? New information becomes available, perhaps. Or perhaps there is information in other peoples’ guesses. That is, information precisely in the dependence of each member on others’ perceptions of that market.