It’s nearly the end of 2012 – the year I will remember as (among many other things) the year in which I discovered one of the neatest and most brilliant theorems I’d ever heard. I don’t know how it had not crossed my path before, but I’ve definitely been enriched since becoming aware of it, so I thought it’d be a good Blog subject on which to end the year. The theorem is called Aumann’s Agreement Theorem.
Here’s how it goes. You may have noticed (I expect you have) that people in the world disagree about lots of things– religion, politics, ideologies, moral systems, land rights, tastes, facts, and many other things. And in disagreeing with each other they make points of argument and listen to the counterpoints – back and forth, with each trying to rebut the other’s contentions. It may seem the most natural thing in the world that opinions vary so widely, but I have long thought that most disagreements are either psychological and emotional, or due to biases, incomplete information, and poor reasoning – so I was delighted to find someone had formally confirmed my suspicions with one of the most elegant mathematical theorems I’ve ever read*.
The idea behind the theorem is that most people disagree on ‘facts’ that are really statements about relations between data points. Game theorist Robert Aumann stated in his theorem that if two independent computers have the same programs, the same set of data (the input) and the same computational steps, then both computers should display the same output. Not only that, two computers with different information should still reach the same output provided the computational steps are the same. Imagine a complex murder scene with 1000 bits of information related to the crime. The agreement theorem states that if computer A had 500 facts and computer B had the other 500 facts they should both exchange data and come to the same conclusions about who the murderer is. The probability of their reaching the same conclusion increases further with every single instance of shared facts. Unfortunately, humans aren't logic machines - they have flawed reasoning skills, misinformation, sensory faults, biases and incomplete knowledge of any situation in its entirety. If Jill stops John and asks for directions to city hall, and at the end of the journey Jill ends up at the market 2 miles away, we can conclude that either John's information was wrong or Jill didn't follow the instructions precisely.
But despite the many human flaws, the upshot is, two logical, unbiased minds debating a subject (like the computers) ought to soon reach an agreement, either on the right answer, or on how far a human mind can logically accept or deny a set of premises, or else on rare occasions admit a roadblock holts their tracks. The roadblock may be due to human limitation of knowledge, not having access to all the facts, or it could be that the wrong question is being asked. But in terms of all the facts, difference in subjective probabilities ought to be solely down to differences in information. This is what Aumann means with the term 'the assumption of equal priors'. If all disagreements are errors in analysing posteriors then when two people reach different conclusions, we know that one or both may be at fault. If there is no rational basis on which two people with the same information should disagree, then what two debaters need to do is, first, make sure each has all the information, and second, keep discussing the individual issues face to face with openness and honesty until a resolution is reached.
This may sound like idealism – but it is idealism only in the sense that we know everyone is not going to start to agree on everything. That doesn't mean the observation is devoid of genuine power and utility. One's own awareness of why people diverge in opinions, and the method by which they can understand how to converge, can only be a good piece of wisdom to have. Remember the agreement theorem isn't saying that everyone will agree - it is stating that people can (and perhaps should) in principle converge on an agreement as long ask they honestly wish to find it. To have that summarised in an elegant mathematical theorem is great if you want a welcome recourse in the face of all those mindless disagreements happening out there.
I think it is evident that as humans have continued to develop and co-exist the magnitude of disagreement has diminished over time**. That is to say, the longer humans have been able to develop their knowledge and understanding together, the more closely we have converged in understanding, particularly since the rise of empirical and rationalist paradigms. The take home lesson is that the reason humans disagree so much is not through lack of availability of data or methods of assimilating that data, which should serve as a signpost for those who genuinely want to enhance their knowledge and understanding, and find themselves in the company of like-minded people who feel the same. Robert Aumann and Scott Aaronson have given us a mathematical proof that shows this is how knowledge would work if people passionately and intelligently and diligently sought the truth without getting so swayed by emotive or biasing factors.
* Mathematician Scott Aaronson has formalised the agreement theorem by providing a mathematical proof which shows that agreement should be converged upon in fairly quick time. Aaronson's proof is the best and most comprehensive summary of the theory I've seen.
** This is offset by the fact that there are now more people, and hence, more things about which to disagree, but proportionally we are increasing our convergence rates.
The problem with this theorem is not that humans are illogical, but the assumption that logic alone can solve problems out in the real world. Don't get me wrong, I'm a huge fan of logic, but one of the biggest lessons of logic puzzles comes from situations when they fail to work.
ReplyDeleteTake your example with the computers. You say that they each start with 500 facts. That's an astounding assumption: how did they get these facts? I'm assuming these facts are all data points relevant to the case, but your labeling suggests they are incontrovertible. These are not opinions, heuristics, or even strong claims. They are facts. How was the data collected, sorted, and measured so that these data points are considered factual?
If there really existed an AI computer that could analyze forensics, then I'd expect hooking two of them together would not produce the positive effect you suggest, exactly BECAUSE they are logical. How would computer A know that all of computer B's claims about blood spatter were valid? How would computer B know to trust computer A's measurements of the suspects? They would have to double check, assuming the crime scene was still available. And since all measurements are estimates, they would have cause to criticize if one rounded to the tenth of an centimeter, and the other rounded to the hundredth.
Even if we eliminate human error in the collection and measuring of those data points, they cannot form them into schemas without weighting them, using heuristics as well as algorithms. And when that happens, even the most logical minds disagree all the time.
DeepShadow, this isn't a theorem that says logic will do all the work - it says that humans can use all kinds of mental tools to agree, and that they should if they apply them without bias and help each other in the process.
ReplyDelete