Monitoring Bill Brieger | 16 Dec 2008 02:48 pm
Can we trust the numbers?
The Washington Post reports that, “Many of the world’s poorest countries have for decades routinely exaggerated the number of children being immunized against disease, apparently driven by political pressure and, more recently, financial incentives. That is the finding of a huge analysis (by Christopher Murray and colleagues) that has provoked heated discussion even before its publication in the Lancet.”
Kenya’s The Nation explained that, “Researchers analysed independent surveys and found gaps between actual rates of immunisation and estimates reported to the World Health Organisation and the UN Children’s Fund.” This gap can be seen in the chart above.
The Post article suggests that the “pay for performance” approach of the Global Alliance for Vaccines and Immunizations (GAVI) may have contributed to the exaggerated performance reports. Specifically, “The study also found that the GAVI ISS program, which pays countries US$20 for each additional child that countries report to have immunized, leads to over-reporting in two out of three countries.” The Washington Post continues by saying that, “GAVI performs ‘data quality audits’ that test the validity of official counts by following the data trail in four health districts per country.” But that is not sufficient to detect over-reporting, Murray and his colleagues concluded according to a press release.
How are we sure that the same problems of over-reporting do not afflict statistics from projects supported by GFATM, PMI or the World Bank Booster, to name some of the major players? “The Global Fund follows the principles of performance-based funding in making funding decisions. The aim is to ensure that investments are made only where grant funding is managed and spent effectively on programs that achieve impact in the fight against HIV/AIDS, tuberculosis and malaria.” and accomplishes this through Local Fund Agents.
The US President’s Malaria Initiative 2008 Annual Report recognizes “The need to strengthen monitoring and evaluation systems for malaria so that national malaria control programs and partners can monitor the progress of their activities, make adjustments, and report on their results.” The World Bank Booster program addresses results-based monitoring and evaluation.
This does not mean that malaria programs are immune from data quality problems. Vigilance is always needed to ensure that the best quality data are gathered and that these inform program decisions.