An American software developer has given up haggling with the CIA over intellectual property issues and will soon put his new intelligence analysis tool online as open source software, according to Danger Room, the national security-themed blog at Wired magazine.
This could be a useful freebie not just for other intelligence agencies, but for any person or company overwhelmed by information and needing to make sense of it. That means academics, students, journalists, market analysts, lawyers, medical researchers etc.
As the contractor's website explains, the way intelligence analysts process information reveals a cognitive bias that is common to anyone who has trouble coping with a lot of data:
...when their desks overflowed with evidence pointing in many directions, their brains could no longer assess each new piece of information objectively. To compensate, analysts would simply choose a preferred hypothesis, and then assess each new piece of evidence only in the context of that single hypothesis. This created two glaring problems:
1) When analysts found a piece of evidence that supported their preferred hypothesis, they attached added weight to it. On the other hand, analysts were inclined to dismiss contradictory evidence as anomalous.
2) By focusing only on one hypothesis, they neglected to consider that a given piece of evidence could likely support many other hypotheses as well.
I don't think I've ever read a more succinct summary of how the intelligence community got the Iraq WMD story so wrong. I dare say you could find similar cognitive biases operating among financial analysts who missed the US meltdown that led to the GFC.
The software that Matthew Burton developed for the CIA is designed to help analysts overcome this bias. It's not clear when exactly it will become public, but you can sign up for email notification here.
Photo by Flickr user illuminaut, used under a Creative Common license.
