Human collective intelligence as distributed Bayesian inference
And now for something completely different. I came across an interesting paper (by Krafft et al) on the arXiv and thought I would share it here. You can download the full text from the link above, but here is the abstract:
Collective intelligence is believed to underly the remarkable success of human society. The formation of accurate shared beliefs is one of the key components of human collective intelligence. How are accurate shared beliefs formed in groups of fallible individuals? Answering this question requires a multiscale analysis. We must understand both the individual decision mechanisms people use, and the properties and dynamics of those mechanisms in the aggregate. As of yet, mathematical tools for such an approach have been lacking. To address this gap, we introduce a new analytical framework: We propose that groups arrive at accurate shared beliefs via distributed Bayesian inference. Distributed inference occurs through information processing at the individual level, and yields rational belief formation at the group level. We instantiate this framework in a new model of human social decision-making, which we validate using a dataset we collected of over 50,000 users of an online social trading platform where investors mimic each others’ trades using real money in foreign exchange and other asset markets. We find that in this setting people use a decision mechanism in which popularity is treated as a prior distribution for which decisions are best to make. This mechanism is boundedly rational at the individual level, but we prove that in the aggregate implements a type of approximate “Thompson sampling”—a well-known and highly effective single-agent Bayesian machine learning algorithm for sequential decision-making. The perspective of distributed Bayesian inference therefore reveals how collective rationality emerges from the boundedly rational decision mechanisms people use.
It’s an interesting question how Bayesian inference relates to the multitude of ways in which individual humans update their understanding of and beliefs about various aspects of the world in the light of new information. This is not always a rational process! This paper extends the discussion to how collective beliefs are shaped, and how this process relates to what happens at the level of the individual.Follow @telescoper Follow @telescoper