ANALYSIS of the difficulties in comparing vaccines

Why COVID vaccines are so difficult to compare Despite the widespread roll-out of several vaccines, it could be months before they can be ranked. Heidi Ledford nature

Yusuff Adebayo Adebisi knows that a vaccine that offers 70% protection against COVID-19 could be a valuable tool against the coronavirus pandemic in Nigeria — especially if that vaccine is cheap and doesn’t have to be stored at extremely cold temperatures. But what if another vaccine — one that is more expensive to buy and to store — was 95% effective?

“Should we send the less-effective vaccine to Africa? Or should we look for a way to strengthen the cold storage?” asks Adebisi, director for research at African Young Leaders for Global Health, a non-profit organization based in Abuja.

These are the kinds of question facing researchers and government leaders worldwide, as they take stock of the emerging selection of coronavirus vaccines and try to decide which will be most useful in putting an end to a pandemic that has already taken nearly 2.5 million lives. It is a decision shaped by limited supplies and hampered by limited data, says Cristina Possas, a public-health researcher at the Oswaldo Cruz Foundation in Rio de Janeiro, Brazil. “It is not possible to compare these vaccines at this point,” she says.

In Bangladesh, health economist Shafiun Shimul at the University of Dhaka worries about the risks if governments delay vaccinations for months to build cold-chain infrastructure. “If you want to control infection, you have to rely on something that is contextually doable for you — it’s not only about effectiveness,” he says. “If they wait for perfection, I think it will be a long wait.”

Given the demand for speed amid limited supplies, any effort to rank the vaccines must take into account not only their reported effectiveness, but also supplies, costs, the logistics of deploying them, the durability of the protection they offer and their ability to fend off emerging viral variants. Even so, many people might find it hard to look away from clinical-trial results that suggest an efficacy gap. So far, more than 200 million doses of coronavirus vaccines have been delivered, and data have been rolling in from clinical trials in several countries. The top-line results from those studies suggest a range of protection: from 95% efficacy for a vaccine made by Pfizer of New York City and BioNTech of Mainz, Germany, to about 70% suggested by initial results on a vaccine made by AstraZeneca of Cambridge and the University of Oxford, both in the United Kingdom.

It might be tempting, but it simply isn’t possible to directly compare the effectiveness of vaccines on the basis of those results alone, cautions David Kennedy, who studies the ecology and evolution of infectious diseases at Pennsylvania State University in University Park. Each measure of efficacy comes with a degree of uncertainty, and trials might have differing definitions of important criteria, such as what constitutes a ‘severe’ bout of COVID-19 compared to a ‘moderate’ one.  ....