Obviously the biggest news of the day is the recent Lancet article which concludes that the number of excess deaths in Iraq since the war began is around 655k with 95% confidence interval (393k,943k)*. Cause of death is also attributed with over half due to gun shot, 90% violent and a third attributed to coalition forces.
The methods used to estimate this headline number is based on a sampling procedure developed by Centres for Disease Control and has been used without controversy to estimate disease prevalence (obviously) but also back in 2001 to estimate excess deaths in the Congo conflict and indeed in Iraq two years ago and arrived at a figure of around 100k.
The obvious question to ask is why can you not just count deaths reported in morgues and hospitals. The answer is you can count them there are between 43,491 and 48,283 up to September 26, but most deaths are not reported. In an environment resembling civil war, if uncle Hassan gets shot dead you don’t bother taking him to the hospital or morgue. You just bury him as fast as you can, especially if you are Muslim. Experts in field say that body count methods under-estimate the true count by a factor of at least 5, and typically a factor of 10.
Another approach is to estimate the daily death toll over a representative period and extrapolate to a larger time span. The US Department of Defence collected such data which gives an estimate of 117 civilian casualties per day between May 2005, and June 2006 which over the almost 1300 day length of the Iraq conflict scales up to about 150k. This still relies on a successful census of all deaths during a particular time, which is practically impossible in Iraq.
Statisticians are historically very innovative in designing methods of counting things that are hard to count. An early success was in estimating the number of German tanks using what were the precursors of capture-recapture methods. (They were helped by the fact that the tanks came off the production line with consecutive serial numbers!)
The Lancet study is based on a quasi random cluster sample of households randomised at four levels i.e. multi-stage sampling. You divide Iraq into what we might call LGA’s and choose some at random. Within the LGA’s you random choose a geographical area, then you randomly choose a street then you randomly choose a number. You then sample that household and all surrounding households until 40 households are sampled centred at that location. There were 50 such locations selected in this study, of which 47 were actually sampled. The 3 locations that were missed were not missed because of the danger to the interviewers, which may have introduced some bias into the results. In the 2004 study there were only 33 locations.
The key count was the number of people who had died, who had lived in these households for three continuous months prior to death. There were 82 surveyed deaths prior to the war and 547 after. This seems like a small number to non-statisticians but we know that the standard deviation of a count is the square root of the mean which here is about 25 or 4%. There are obviously other sources of variability in arriving at the final uncertainty estimates. And accounting for unintended bias is where most of the public discussion will probably focus.
Excess deaths are measured by comparing the death toll during the period January 2002 to beginning of the war in March 2003 and comparing this with deaths thereafter. A simple minded calculation would compare say the 82 deaths over 14 month prior to the war which is 5.8 per month to the 547 over the 42 months since the war began which is 13.0 per month. Scale up the difference of 13.0-5.8=7.2 per month by the appropriate population factor and you have your estimate.
You can obviously obtain death estimates for different geographic regions using this method, see the graphic below, and the estimation problem should be broken down by region (and demographic factors) in coming to a total figure.
One thing that might affect these results would be immigration out of Iraq or more pertinently from areas of high mortality to areas of low mortality which would decrease the base population in high mortality areas and suggest a lower total death toll in those areas. The Lancet study also estimated this effect. Households were also asked to identify movement into and out of their households during the study period.
This has been a quick summary of the paper during a busy day while it is still topical. You can download the paper yourself HERE. Not sure I will be able to respond to any comments until tomorrow.
* Corrected on October 16
I notice that one of Australias foremost statisticians has doubts about the accuracy of this study. Howard didn’t go into the details of his methodological critique.
Two or three questions occur to me:
1. Does he really believe it?
2. Is he structurally incapable of believing it?
3. What would he do if he did believe it?
I’m not sure where to start with these. From a cynical point-of-view it’s just another excercise in using media to discredit “inconvenient” facts but what if that’s not the full story. I’d love to know what really goes on in his head.
It seems that the number of deaths are excess deaths, meaning that the number arrived at is over and above the number of deaths that the internal terrorism that Saddam Hussein resulted in. It’s a hell of a number. Noticeable that 216,666 deaths are attributed to coalition forces. These troops must be going around shooting up and killing many hundreds of civilians on a daily basis. The censorship of these military killings has certainly been remarkable successful, as precious little anecdotal evidence has emerged. Of course some portion of them must be attributable to the initial invasion, but the lion’s share of the 216,666 must have occurred in day-to-day killings.
You are quite correct WIIS that the estimate is of excess killings – so any counter argument about Saddam’s pre-war murder rate being traded off against post war deaths fails.
There are many possible biases in the study and some surprising figures – the 200,000 deaths caused by coalition forces being one of them. You might like to have a look at my blog HERE where a commenter has listed some of the inadequacies in the study. That is not to say that the headline figure is wrong.
Interesting comment by John Aitchison. Statistics is a very specialised science about which I have no expertise. I suggest the same is true of 99% of people who will either use this number as an established, set in concrete fact, and those who will instinctively come out and say it has no credibility. I would suggest that a healthy scepticism is probably a valid approach until some independent verification (or debunking) occurs.
What is clear is that a great many deaths are occurring in Iraq and at an escalating rate. What I’d like to see is some analysis of reasons for those deaths. How many are caused by Sunni and Shiite insurgents, how many are caused by retaliation against Sunni and Shiite insurgents. How many by outlaw members of the invasion forces, how many in legitimate military and police actions, and so on.
It seems to me that most of the insurgent killings are carried out by Sunni insurgents against Shiites, but the extent of retaliation by Shiite gangs (militias?) is increasing. There is no doubt a number of unlawful murders by members of the invading forces, but I suspect it’s very minor in the overall death toll.
There also appear to be a large number of deaths caused by disruption of normal medical services because of insurgent action, and medical problems leading to death caused by the excess stress of every day life in such an environment.
I think some scepticism about the actual figure is appropriate but not its order of magnitude i.e. a reasonable estimate would be 500,000 rather than 50,000. If you read through the list of possible bias sources mentioned by various bloggers there is no way you end up with a factor of 10.
I would also like to see some more data on the causes of deaths as I find it very hard to believe that 200,000 (over 1000 per week) Iraqi’s are being killed by coalition forces. Perhaps it is true but it was certainly not the perception of most of us.
As to Bush and Howard describing the figure 650k as “not credible”, it is a rhetrical trick to ascribe the property of being not credible to the figure rather than the person hearing it. People make the decision to believe or not. It is not a trait of the figure! Reminds me of my son who, when he was young and broke something, would say “it broke”. Sounds like the kind of tricky device that Steven Poole could write a 1000 entertaining words on.
500,000 is well within the stated confidence range anyway. But I don’t see how you can give the overall figure credibility and find the number claimed killed by coalition forces not credible. Aren’t both part of the same set of conclusions? Your comment would be more consistent if you claimed that while 200,000 is hard to believe, perhaps 160,000 would be believable. About the same ratio.
“Perhaps it is true but it was certainly not the perception of most of us.”
This returns the situation to one of subjective perceptions which was what I thought the study was designed to counter, to introduce an element of objectivity.
We can all talk about our perceptions, but where do these come from? Media reports in most cases I would have thought. Not the most reliable source by any measure.
I think we are mainly in agreement WIIS but I am not expressing myself very well. I am very surprised by the 200,000 figure because of – as you say – my subjective perceptions. But I am not saying I disbelieve the figure on that basis.
But there is a difference between the reliability of the estimated total figure (650k) and the reliability of estimated deaths attributed to coalition forces (200k). The total figure depends mainly on people correctly identifying how many people in their household have died and placing the death the correct side of March 2003. The second figure depends on them correctly attributing the cause of death. This could be quite a difficult task and I can also imagine that there could be some anti-coalition bias in the attribution.
Echoing your sentiments about the quality of media scrutiny of these issues, I think it is pretty disgraceful that there has not been more discussion of the 200,000 figure. One of the best science journals in the world has published a paper saying that coalition forces are killing more than 1000 Iraqi’s per week. This seems like an issue worth investigating to me. They could start by directly asking Rummie whether or not his forces are killing this many people. I expect that the army keeps records of every engagement and estimated casualties. Whether they would release the figures is another matter.
Chris
Many thanks for this post. Your commenter John Aitchison at your own blog certainly makes a lot more sense in his measured critique of the methodology of the latest death count survey than the predictable scoffing of the usual RWDB suspects.
I don’t find the 650,000 (approx) excess death estimate to be inherently incredible, given that 970,000 or thereabouts died during the US Civil War, which was of approximately the same duration as the Iraq war/occupation to date. That death toll also represented an approximately equal percentage of the total population of the country at 3%. It also occurred at a time when weapons technology was much less sophisticated than today.
However, one factor supporting a somewhat sceptical attitude is the evident tendency of some Middle Eastern Arab cultures towards fabulism/exaggeration. It was very evident during the recent Israeli incursion and bombing of southern Lebanon, and I suspect it might also be a factor here. I think John Aitchison pointed out that dwellings were selected in clusters of 40 or so, which would have given plenty of time for the word to spread in a neighbourhood that a death count survey was under way. If we assume (not unreasonably) that there is a pretty high level of hostility towards the Americans/COW, especially in Sunni areas (but in quite a few Shia ones as well), residents might well have every motivation for exaggerating the scale of slaughter to cast the Americans in a bad light.
That said, even if the actual excess death count is “only” 200,000 to date, it’s still a horrendous toll and just as starkly indicates what a disastrous and ill-advised venture the COW invasion and occupation was, when one considers that Saddam had neither a significant existing WMD program nor links to Al Qaeda (although his regime DID provide tangible backing for Hamas’s activities in Palestine).
BTW I’m not suggesting that any “fabulism” factor could by itself reduce the actual excess death toll to “only” 200,000. After all, I gather that something like 80% of the reported deaths were verified by actual death certificates. Nevertheless, together with factors such as those discussed by Aitchison, it could conceivably mean that the actual excess death toll might be a bit less than the 400,000 low point of the new Lancet survey. However, as I say, even making as much allowance as possible for such factors, it’s still a horrendous situation that you would have thought would have given at least some more thoughtful and compassionate RWDBs (like Currency Lad) cause for a moment’s hesitation and reflection.
I still think some analysis of the reasons for these deaths would be helpful, albeit that any information gathering in Iraq is extremely problematical. I tend to agree with Chris’ differentiation between the credibility of the figures of total deaths and those caused by COW troops. My (again subjective) impression is that the vast majority of deaths are caused by Iraqis killing Iraqis.
Who should bear the responsibility for this? The conventional wisdom is that the US and Allies are responsible because, de0spite Saddam’s brutality, there are far more killings now.
But Iraqis are not wild animals – they are human beings. The war did not cause the inadvertent release of zoo-ed beasts of the jungles to slaughter indiscriminately. I do get a strong sense in reading commentaries about the war that we must be apologetic towards Iraqis for visiting this slaughter upon them.
But they themselves are human. They are not beasts. They must assume human responsibility for their actions. They ought to be called to account. They’re not being.
One of the authors of the report will be interview on Radio Natinal this Sunday, see HERE. Doesn’t look like he will receive the most exacting interrogation from the ABC but still worth a listen.