Stats 101 for Policymakers

It's a problem that is easy to recognize, but hard to get around: policy isn't made by public health epidemiologists or statisticians. In between the researchers (who have their own biases) and the policy makers is a whole industry of  interest groups and advocates. Of course, I've been one of those advocates at times, as has pretty much everyone who has worked in public health or politics. Why am I thinking about this? I just read "Interpreting health statistics for policymaking: the story behind the headlines" in the Lancet (available for free here) by Neff Walker et al. The paper outlines this problem:

Estimates would be more credible if they come from technical groups that are independent of the organisations that implement programmes and advocate for funds.

Maternal mortality is much more inequitably distributed than neonatal mortality, but does that mean we should focus on it more? Of course, some of this comes down to fundamental philosophical differences concerning whether we should concentrate our health investments where they will make the most difference in terms of absolute numbers of lives saved, or where they will make the most difference in terms of reducing health care inequalities.

Statements like “Maternal mortality is 100-fold higher in many low-income countries than in high-income countries” sends a clear message with respect to inequities, but no information about the absolute magnitude of the problem. Statements more useful to decisionmakers are those that use a standard metric to provide sets of meaningful comparisons. For example, the ratio of inequity between low-income and high income countries for deaths from severe neonatal infections is far lower, at 11-fold. In absolute numbers, however, two to three times as many lives are lost to neonatal infections each year (1·4 million) in developing countries than to maternal mortality (500 000).

It's easy to criticize HIV/AIDS advocates because they're such, well, good advocates. Example 1:

For example, a common practice is to present an estimate at the global or regional level and then to elaborate on it by giving a specific and often unrepresentative example. HIV/AIDS advocates talking about the eff ect of AIDS on under-5 mortality often use as examples countries in southern Africa where AIDS accounts for 30–50% of deaths in under-5s. But for sub-Saharan Africa as a whole, AIDS is thought to account for less than 10% of under-5 deaths.

Example 2:

Advocates of funding for [HIV/AIDS] often quote the cumulative number of global deaths from HIV/AIDS since it was first identified. But, if historical estimates were used for other diseases, the number of HIV/AIDS deaths would be small in comparison. For example, if the same statistical procedures were applied for pneumonia as for HIV/AIDS, the cumulative deaths since 1975 would be about 60 million—almost three times the estimated cumulative deaths from AIDS in the same time period.

They end the paper with a list of recommendations for how policymakers should consider health stats coming from advocates or any other source.