Measuring the value of health

In a recent essay for the Wall Street Journal our co-chair Bill Gates shared the best investment he’s ever made: the $10 billion the Gates Foundation has invested in Gavi, the Global Fund, and the Global Polio Eradication Initiative. He breaks down why he considers this a strong investment:

“Suppose that our foundation hadn’t invested in Gavi, the Global Fund and GPEI and had instead put that $10 billion into the S&P 500, promising to give the balance to developing countries 18 years later. As of last week, those countries would have received about $12 billion, adjusted for inflation, or $17 billion if we factor in reinvested dividends.

What if we had invested $10 billion in energy projects in the developing world? In that case, the return would have been $150 billion. What about infrastructure? $170 billion. By investing in global health institutions, however, we exceeded all of those returns: The $10 billion that we gave to help provide vaccines, drugs, bed nets and other supplies in developing countries created an estimated $200 billion in social and economic benefits.

As Bill notes in his essay, the foundation worked with experts at the Copenhagen Consensus Center to calculate that outsize return on investment. The Optimist sat down with Damian Walker, deputy director of data and analytics at the Gates Foundation, to talk about how the team approached this problem, the challenges of evaluating investments in global health and why these analyses should be a key input in decision-making.

A health economist at the Gates Foundation

Before I came to the foundation in the summer of 2010 I was an associate professor at the Johns Hopkins School of Public Health. I taught health economics, with a focus on economic evaluation of health programs in low- and middle-income countries.

In January 2010 at Davos, the Gates Foundation announced they’d pledge $10 billion in a call for a “decade of vaccines.” Colleagues at the foundation, worked with modelers and estimated that nearly nine million lives could be saved. Tracking progress against this number, and updating the estimates, become one of my core responsibilities at the foundation. And over time, I’ve become the go-to person for estimating the health and economic impact of a lot of the global health work we do.

This commitment to ensuring that we optimize the foundation’s return on investment is an extension of my career as a health economist. Cost-effectiveness estimates and their component parts (costs, burden, coverage, efficacy and quality) are critical inputs to the foundation’s upstream and downstream decision-making and programmatic goals. In terms of strategy development, these data help inform areas we prioritize in terms of disease, interventions and research. In terms of discovery and development, these data help determine the relative cost-effectiveness of products in our pipeline, and therefore which products we should accelerate, course-correct or kill. And in terms of delivery, these data have help inform which newly licensed and pre-qualified products should be adopted by a country and / or made available through global health funders such as Gavi, the Global Fund, or UNITAID. These data have also helped inform the choice of delivery strategy—e.g., whom to target, and where to target.

My six-year-old thinks I’m a doctor of numbers (as opposed to a “real” doctor like my father and elder sister). But I think it’s about trying to inform tough trade offs with the best available data. You can apply that at multiple levels. I have as a health economist, mostly applied that to and within global health. But the foundation works in other development sectors such as education, agriculture, and financial services, and it’s also about helping make those tough trade offs across and within these sectors as well.

Evaluating the impact of global health financing

This is a deceptively simple exercise. Count the costs. Count the benefits. Present in terms of a ratio or net benefits. If the former is greater than 1, than the benefits are worth the effort. Alternatively, if the net benefits are positive (benefits minus costs), again, the thing being evaluated is “worth doing”. Seems simple enough right? But here are just a few of the challenges to this kind of work.

First, whose costs and benefits should be included? Including, or conversely excluding, some costs and benefits can really influence your findings. So, it’s always important to unpack results and assess whether you think the most important costs and benefits have been included.

Of course, in practice, data availability and quality will often determine what’s possible. But that doesn’t mean we can’t list other costs and benefits we believe to be real but that we didn’t or couldn’t include. Let me give you an example. We know that accessing health care isn’t costless, and those costs can act as a barrier for many people. We can estimate how much time it takes—does it take twenty minutes or an hour to get to the local health center (although we actually don’t have much data on this). But trying to place a value on that time becomes much harder; deciding, for instance, that twenty minutes is worth two dollars to that mum who lives in rural Tanzania. For this reason, we decided to exclude these costs from the return on investment estimate reported in the WSJ.

Second, to estimate what has been (or will be) achieved, we need to compare the impact to what would have happened (or will happen)—how do we do that? Should we assume that nothing will happen? Should we assume that past trends will continue? The choice will heavily influence the estimate of impact.

For this work, the question was what would have happened in the absence of these Gs? We just don’t know how countries would have responded without the existence of Gavi, the Global Fund, and GPEI. Would they have stepped up and made more investments in health? I think it’s probably fair to say that they might’ve, but it would’ve been much slower than it’s been with those Gs.

And even if we think we have plausible estimates of what would’ve happened, we also must understand what happened and due to what or whom, i.e. can we attribute impact? To sum it up the issues are: What would have happened in the absence in the Gs and what’s happened due to the Gs? In our return on investment analysis we took the position that the burden of disease in poor countries would not improve in a world without the Gs, i.e. no other efforts would have taken place in lieu of the Gs. Were we to undertake a prospective analysis, this would seem, in my opinion less plausible. And we tried to attribute impact by using the percentage of total program spending that each G accounts for.

Third, and finally: do we care whether a life is saved today or in the future? Economists call this concept “time preference”, and they address it in evaluations by discounting future events, that is to say, future lives are worth less. Ethicists think this is immoral—isn’t a life saved in 20 years just as important as a life saved today?

Not only is there debate about whether to discount. But even among those who agree that we should discount, there’s debate about the rate we should use. Here at the foundation, we don’t have a consistent viewpoint on this topic.

For example, the value proposition of elimination or eradication efforts is that future generations will be disease free, so we clearly value those future lives saved. But at the same time, we are “impatient optimists”, and discounting is essentially a measure of impatience. We want to see results today and not in the future. On this point, we decided to go with a 5 percent discount rate, consistent with new guidance. (Note that the choice of discount rate didn’t really change the results of this analysis—using 3 percent the ratio was 22:1 and using 8 percent the ratio was 18:1.)

So you see, while there’s a lot of “science” involved in estimating the impact of global health financing, there’s an awful lot of “art” (and of course “politics”) too. I think it’s important that we need to be able to have a sense of whether our estimates are conservative or possibly too optimistic.

Calculating a $200 billion return on investment in global health

To calculate the number in the Wall Street Journal [editorial note: a $10 billion investment in global health funds resulted in a $200 billion return] we asked the Copenhagen Consensus Center to estimate the return on investment of three global health financing arms: Gavi, The Global Fund, and the Global Polio Eradication Initiative.

They did this by estimating the combined costs of the respective funds and comparing that to the combined benefits. The costs include money disbursed since the inception of each fund. The benefits include the value of reductions in morbidity and mortality, and savings to the health care systems in low- and middle-income countries.

The team at the Copenhagen Consensus Center used published studies where available to underpin their work and updated the data where possible. Finally, they made some adjustments to bring the estimates into line with best practices today. Specifically, two years ago I had made a grant to Harvard University to develop best practice for how you conduct benefit-cost analyses. One of the work streams in that grant was to produce a paper to give us what we think are the best estimates of the statistical value of a life. Another work stream investigated how best to account for the timing of costs and benefits (see above mention of the 5 percent discount rate).

I’d say that study gave us the beginnings of an emerging consensus—but we need more and better data. We will have to continue to iterate on this.

The 20 to 1 return on investments in Gavi, The Global Fund and GPEI wasn’t a surprise to me. What I think was new in the essay was the comparison Bill used, and I imagine it will lead to lots of discussion and debate:

“Suppose that our foundation hadn’t invested in Gavi, the Global Fund and GPEI and had instead put that $10 billion into the S&P 500, promising to give the balance to developing countries 18 years later. As of last week, those countries would have received about $12 billion, adjusted for inflation, or $17 billion if we factor in reinvested dividends.”

The value of these analyses

For me it’s about trying to institutionalize the processes for producing and consuming these estimates. We, at the foundation, tend to produce one-off pieces of information to inform an investment, but then we often go quiet until, for example the next replenishment—it’s often a 5-year cycle.

We need to start embedding this kind of work in the Gs so it can inform trade-offs on a more systematic basis. And countries need to build this muscle too—we need to work with them to ensure that their scarce resources are used as efficiently and effectively as possible.

I’m pleased to report that my team at the foundation is supporting several groups to do precisely that. We know that it will take time to build and strengthen global and national priority-setting institutions.

But as more countries graduate from eligibility for support from the Gs, it will become more and more important that they make the best possible decisions given the data and evidence available to them. The cost of wrong decisions can be counted in lives not saved.