Statisticians React to the News

Missing the evidence

09 March 2021
Vaccine

A story in the Guardian on February 28 is headlined Pfizer vaccine may be less effective in people with obesity, says study. As the story says, the ‘study’ in question is a preprint. That’s not necessarily a problem; preprints have been a valuable source of up-to-date information during the Covid pandemic, at least when used with care.

And there isn’t anything particularly wrong with the study itself: a hospital in Italy measured antibody responses after vaccination in a group of healthcare workers and reported correlations with other characteristics of the people. Obesity is known to be correlated with various differences in the immune system, so it wouldn’t be astonishing to see differences in immune response to vaccine, and the researchers saw lower levels of some antibodies.

On the other hand, we do have a randomised controlled trial of the Pfizer/BioNTech vaccine, and the Guardian article doesn’t mention anything about what it found on the subject. Clinical trials can sometimes be unhelpful for this sort of question; they often end up with an unusually healthy or unusually sick selection from the population we actually care about. The Pfizer trial, though, did reasonably well in trying to recruit a representative selection of people (or, at least, of people from the USA). In particular, it recruited a lot of people with obesity (35% of participants), because that’s what happens in a representative selection of people the USA. Since we know the vaccine was 95% effective in preventing COVID-19 (averaged over everyone in the trial), it must have been reasonably effective in any large subset of people — and that’s real effectiveness in preventing disease, not just differences in antibody levels.

It’s worse than that, though. The Pfizer trial actually reported obesity as a planned subgroup analysis. That is, the trial results (publicly available in the FDA briefing documents) give the effectiveness of the vaccine for obese and non-obese participants separately, and also further break things down by age. These are reported in Table 8 of the FDA Briefing document (pdf here). The estimated efficacy was 95.5% in the 13297 people with obesity, and 94.1% in the other 24228 participants; the Italian preprint reports on 26 obese participants out of 248.

There isn’t the slightest suggestion in the trial that the vaccine is less effective with obesity. Maybe things could change over time, and immunity could decay faster, but the vaccine efficacy in the trial was evaluated over a longer period of time than the antibody response in the preprint.

The research in the preprint was motivated by the question of how well the vaccine protects against all infection, possibly asymptomatic, with the SARS-2-CoV virus. That’s a sensible question, and not one covered by the initial trial results, though it’s also not one that a small antibody study will answer all that well. Four days before the Guardian story, the New England Journal of Medicine published a sophisticated analysis of vaccination data from more than a million people from Israel. That paper estimated 92% protection against ‘documented infection’, and (a bit more tentatively) 90% protection against asymptomatic infection. Part of the motivation for the analysis was to get more information about subgroups than was available in even a large trial, and Table 3 gives an estimate for people with obesity as a risk factor: 95% protection against documented infection, based on over 100,000 vaccinated and 100,000 unvaccinated people. Again, there is no suggestion that the vaccine is less effective for people who are obese.

None of this evidence based on actual cases of infection and disease in large numbers of people with this specific vaccine is even mentioned in the Guardian story. There is a link to one additional study, however. The story says

In a separate study of Brazilian healthcare workers, Altmann and his colleagues showed that reinfection with Sars-CoV-2 was also more common among people with a high BMI, and that they tended to have lower antibody responses to the original infection.

but the linked paper says

Comorbidities such as arterial hypertension, obesity, diabetes, and asthma were not associated with recurrence, but were associated with moderate/severe clinical presentation of recurrent COVID-19.

and does not report a comparison of antibody responses by BMI or obesity.

The headline raises a legitimately interesting and important public health question. It’s a pity that the story does not engage with the most reliable sources of evidence on that question.

By Thomas Lumley
New Zealand