Off by a factor of 100

GiveWell is an "independent, nonprofit charity evaluator" that finds "outstanding giving opportunities and publish[es] the full details of [their] analysis to help donors decide where to give." Their Giving 101 page is a good place to start regarding their methodology and conclusions. I want to highlight a recent blog post of theirs titled "Errors in DCP2 Cost Effectiveness Estimate for Deworming". DCP2 stands for "Disease Control Priorities in Developing Countries," a report funded by the Gates Foundation and produced for many partners including the World Bank.

The DCP2 blog post and its comments are wonky but worth reading in full because of their implications. It's a pretty strong argument for why calculations need to be as transparent as possible if we're going to make decisions based on them:

Over the past few months, GiveWell has undertaken an in-depth investigation of the cost-effectiveness of deworming, a treatment for parasitic worms that are very common in some parts of the developing world. While our investigation is ongoing, we now believe that one of the key cost-effectiveness estimates for deworming is flawed, and contains several errors that overstate the cost-effectiveness of deworming by a factor of about 100. This finding has implications not just for deworming, but for cost-effectiveness analysis in general: we are now rethinking how we use published cost-effectiveness estimates for which the full calculations and methods are not public...

Eventually, we were able to obtain the spreadsheet that was used to generate the $3.41/DALY [Disability-adjusted life year] estimate. That spreadsheet contains five separate errors that, when corrected, shift the estimated cost effectiveness of deworming from $3.41 to $326.43.

From later in the post:

Whether or not the long-term effects are taken into account, the corrected DCP2 estimate of STH treatment falls outside of the $100/DALY range that the World Bank initially labeled as highly cost-effective (see page 36 of the DCP2.) With the corrections, a variety of interventions, including vaccinations and insecticide-treated bednets, become substantially more cost-effective than deworming.