Wednesday, October 18, 2006

Lies, damn lies & statistics

Statistics is more science than art, but projections generated by statisticians are not as certain as those produced by a physicist studying the effect of gravity on a falling object. The physicist can say with 100% certainty that a brick dropped from a height of 20m will hit the ground in just over 2 seconds (I hope I'm remembering my physics correctly). He makes this statement without any fear of someone proving him wrong.

The same is not quite true with statistics and statisticians. Projections are not as sure and depend on the methodology of the data collection, which can be influenced by factors that can require adjustments based on judgment. In other words, a key question for any statistical study is, "Is this methodology sound"? To some degree that is along the same lines as, "Is this painting beautiful"?

Steven Moore questions the methodology used by the Johns Hopkins Bloomberg School of Public Health in their study of the Iraqi death toll since the March 2003 invasion. Moore's focused on the number of clusters and the failure to ask demographic questions in the surveys.

Moore is a political consultant who has done work for the International Republican Institute and the Coalition Forces in Iraq, so he clearly has an agenda. However, there's no good reason to assume that the Bloomberg School of Public Health doesn't also have an agenda, always releasing their findings in the last month of an election cycle. {The last study's findings were released in October 2004.)

Unlike the numbers provided by the physicist, these statistics are built around a very soft scientific core, which means anyone can accept or reject the findings without fear of being proven wrong. I think I'll take the 650,000 death toll figure with a large pinch of salt.