I’ve got a few questions: 1) Has there ever been a time when scientific publications were as widely discussed in the news as they were last year? 2) How did COVID change the way people think about science? and 3) How did science change the way people think about politics?
Regarding the first question: I guess if Neil Armstrong’s first steps on the moon count as “science”, then the news coverage about that science was probably near 100%. But how about scientific papers in a narrow sense?
Here are some data. For nine years, Altmetric has been the best-known company tracking online mentions of scientific studies – counting citations in news outlets, in blogs, on Twitter, Facebook, and Youtube. Last week, on 20 January, Altmetric inaugurated its list of the top 100 papers of the year 2020.
As expected, the five papers that received the highest online attention in 2020, and likely of all time, were all about COVID. People who are familiar with Altmetric scores will be impressed by the numbers. For example, a study suggesting that the virus was a product of natural selection, rather than being engineered, was cited by 649 news outlets and got the top Altmetric score of about 35’000 (see here about how the Altmetric score is calculated). In comparison, the highest Altmetric scores found until 2019 were around 13’000. Setting aside general problems with such “all-in-one” indicator values, it looks like 2020 was an exceptional year with regard to the volume of online discussion of scientific publications.
How did this wide coverage of COVID studies change the way people think about science? I would hope that the discussions in the media may help non-scientists understand that statistics, and indeed the whole of science, is reasoning under uncertainty, and that this is not the same as decision making under uncertainty. For example, decision making needs to involve considering costs and benefits, and this may partly explain why politicians around the world, for the better or worse, did not always strictly obey to the advice of virologists and epidemiologists. I hope a scientific solution to the current health crisis will lead people to appreciate science more, but it’s too early to judge whether this will be the case.
But did COVID change the way the public, and the scientists, interpret and discuss scientific results? It looks like the same old problems prevail, only inflated as if under a magnifying glass. The paper with the second-highest Altmetric score is the Danish face mask study that was discussed by Olle Häggström on this blog. The study reported a point estimate that wearing masks in public reduces the risk of infection by 18%, yet data were highly compatible with a more pronounced risk decrease or even some risk increase — the result was “statistically non-significant”.
Of course, even if the result would have happened to be “statistically significant”, the face mask study would only be one single piece of evidence and any generalized conclusion about wearing face masks would need to account for the narrow focus of this particular study. If you want to get a full flavor of how a single and controversial result may be discussed in public, have a look at the Altmetric collection of news citing this study, where you may find statements ranging from “mask wearing has no impact on saving lives” through “Don’t expect Covid-19 research to deliver certainty” to “Masks work. Get over it.” A “narrative update” in the same scientific journal in which the original Danish study was published makes a carefully worded and reasonable point: “evidence suggests that the potential benefits of wearing masks likely outweigh the potential harms.”
If we go further down the list of the Altmetric top 100, we find, ranked #3, “Dying in a leadership vacuum,” the call by the editors of the New England Journal of Medicine to not re-elect Trump, and, ranked #5, the retracted study on the effect of Hydroxychloroquine. Thus, four of the top five studies of the Altmetric list are as much about politics as they are about COVID, leading us to my third question: How did science change the way people think about politics?
Apparently, not a lot. For example, as a naive European observer, I really wonder how it was possible that around 47% of the popular vote in the US went to a presidential candidate for whom only 40% approved of his handling of the coronavirus crisis. Despite the arguments one could read in editorials of journals such as Science and Nature, I guess there must have been scientists who voted for Trump. Maybe those colleagues voted for the party, not for the person (but well, then they still voted for the person). Or else, have there been any arguments about science or science-policy that could have outweighed the problems and argued for Donald Trump? I just wonder.
For those who are still continuing reading despite this short excursion to politics: If you scroll down the Altmetric list, you’ll notice a sharp drop in the score if we go from rank #5 (score of 24’400) to #6 (score of 10’700). For the first time, Altmetric did not simply list the highest overall scores, but broke this down according to field of research, listing only five papers per subject area. Such planned changes to the Altmetric top 100 list were already discussed before COVID, but they make particular sense for 2020. As the company says, it is “important to remember that other, important, research was also taking place.”
If you are interested in the full rank order of about 100 million scientific publications according to their Altmetric score, dating back to the year 1665, see here (or here). As of publishing of this blog post, we need to go down 39 steps on that list to find the first publication that is not about COVID. Instead, this 39th article is yet another commentary arguing against statistical significance. I find it remarkable how monotonous the media’s interest in science is these days!
