TRANSCRIPT: John Ioannidis, Closing Remarks at Stanford Pandemic Policy Conference, 4 October 2024
Includes slides and links to content
This is an enhanced transcript of closing remarks from Dr. John Ioannidis at the Stanford University Pandemic Policy: Planning the Future, Assessing the Past symposium held on 4 October 2024.1 Provided for anyone’s use and my future reference.
UPDATE (21 Nov 2024): I wrote about some things I think John Ioannidis is wrong about the includes content from his closing remarks below:
Laura Carstensen: It is now my pleasure to introduce our Stanford colleague John Ioannidis for closing remarks. Professor Ioannidis is a Professor of Medicine, Epidemiology, and Population Health, and he is an internationally recognized expert in scientific evidence itself. And he's a legend.
John Ioannidis: Hello. It's been a long day and I have learned a lot today. I want to thank all the panelists for the wonderful contributions and insights that they contributed today from very different perspectives. I want to thank all of you for being here for a whole day and please do keep the conversation open, even after this session ends.
A very special thanks to our [Stanford] president. I had the pleasure to hear his inaugural speech a few days ago, and I think that he's a Visionary and our 500-year-old year old humanist motto that um - Die Luft der Freiheit weht, you know, “the wind of Freedom blows,” will become stronger, on our campus. And I think his remarks today also prove that.
My real hero, Doug Owens. Amazingly brave what he put together today and I want to pay homage both to Doug and his wife Sarah Cody, who led the response the public health response in Santa Clara County. They’re both people of amazing integrity, and I think integrity is something that we desperately need at the highest possible level today. And of course many thanks to the organizers and to our staff.
So, my disclosures. Number one, I know next to nothing. Number two, I have declined - not sought and declined - to get any funding for my COVID-19 work. I did get an honorary award that came with $100,000, but I asked for the money to be given to two philanthropy organizations for poor children, because personally I feel that we let children down. We let poor people down. We let poor children down. We let our future down. We let our best part of what is human down.
The other disclosure is that I have published about a hundred peer reviewed papers on COVID-19, and it's very likely that since I have also published a paper with the title ‘Why Most Published Research Findings are False,’ it's very likely that much of my work must be wrong. And, you know, certainly all of that can be improved. So please do try to improve it.
I will share some evidence, not as a legend -- as someone who's struggling to understand evidence. I heard the term disaster very early in the morning -- that COVID-19 was a disaster. I agree it was a disaster for some countries, but there were hardly any excess deaths in others.2
And, for the vast majority of countries around the world, we have no clue exactly what happened because we don't even have good death registration systems. So as to even count how many people die, let alone what they die from. But, basically, in the 34 countries that have the best quality data for death registration, we saw that half of them, 17, had roughly no excess deaths compared to the pre-pandemic, immediate pre-pandemic years during the pandemic, and the other half had really a disaster.
The worst disaster was United States and Bulgaria. If you limit to less than 65 years old, we're far worse than anywhere else in the world. The best? Sweden. And I'm very happy to have Anders Tegnell today with us. It's the first time that we meet in person, and I think he is Legend, for sure. And also New Zealand. So, you know, two countries that had very different approaches to how they would handle the crisis.
The common denominator from the way that I see it, and it may be wrong, is that COVID-19 was a disaster in countries that had high inequalities and were in crisis even before the pandemic hit.
So, countries that didn't have resources, countries that had poverty, large shares of poor people marginalized people, countries that had a large inequality in income. You know, we [United States], on average, we're a rich country, but we have a lot of very poor, marginalized people. And these were the people who were hit the most – far, far disproportionately compared to me or, you know, to people who are well off.
So, these countries were in crisis in the past [and] during the pandemic. Unfortunately, I think they will continue to be in crisis after the pandemic, and this is what I worry [about] the most.
Future crises actually may be far worse. And I want to pay attention specifically to one group of the population: people who are in healthcare.
They're the unsung heroes. They went through a double crisis and - you know, an exponentiated crisis - and I have amazing respect for them, for everything that they did.
I'm known as a researcher, but actually I'm a physician, you know. I'm a clinician trained in Internal Medicine and infectious diseases. I decided not to practice medicine in the United States because I really want to spend hours with patients and, you know, you realize that's not feasible. But it is feasible still in Europe. I'm still board certified in Greece, in European Union.
I did have engagement pro bono for consultation for infectious diseases with hundreds of COVID-19 patients, ranging from those who had lethal outcomes and spent very difficult times in the ICU, to those who were completely asymptomatic and healthy, and they got a positive test and it was seen as a death sentence. You know, they were thinking of euthanasia or, you know, very very weird type of thoughts.
And I also got to know a lot of Physicians who are also engaged in taking care of COVID patients and again you saw the tremendous stress that they were under. This is a part of the population that is routinely under tremendous stress. We see tremendously high levels of burnout and that was really what tipped things completely over for them, regardless of whether their services were overwhelmed or empty, as it was in many situations.
This is just some calculations showing the risk though of death or excess death for Physicians, versus the rest of the population during the COVID-19 crisis.3
In countries like Canada or Finland, physicians had fewer deaths compared to the pre-pandemic years and far, far fewer compared to the rest of the population. In the United States, a little bit more, compared to pre-pandemic years, but again far, far less compared to the average person out there.
How about if we are hit by something like Ebola, when it hit in 2014 Africa? You know, you hear that there was Marburg virus - a case in, you know, someone traveling from an area to Europe. How about that? What would happen then, if we have something that is a thousand times worse? We need to be prepared for that. Not die in advance but be prepared for that.
COVID-19 massively mobilized scientists, scholars, policy experts, and more. Influencers, social media, journalists, politicians, policy makers, Big Tech. I mean, we've heard lots of these stakeholders as they interfered in the process of science.
During 2020 and 2021, 98 and then 92 out of the top 100 cited papers most-cited papers across science were on COVID-19, so it's not that we didn't try.4 We did try. We had this is a calculation that I run for a preprint that we released recently, updating a previous publication.5
Almost 2 million scientists publishing about 720,000 papers. It got more than 10 million citations in in Scopus. If you divide science into 174 fields, then every single of these fields had its experts moved to COVID-19. The last field to fall was automobile engineering in, you know, early fall of 2020. And it was automobile engineers actually doing epidemiology and virology. A bit scary.
But the average paper was horrible.
We have a lot of empirical assessments of the evidence. Looking at COVID papers versus other papers, we know that the average paper in the scientific literature is horrible, but COVID papers were more horrible than horrible, and I say this with full respect for all the amazing work that happened during the pandemic.6
We had major successes. We managed to develop pretty effective vaccines for severe outcomes in a very short time. I don't think that most of us thought that it could happen so quickly. We had randomized trials, large-scale randomized trials, adaptive trials that got us results within three months -- and showing that some interventions like dexamethasone are effective, most interventions are not effective, probably at least those that we thought might be.7
So, we had a lot of successes, but we came across the weaknesses of science that we think we can work on and get better.
Thinking for the future, we want more useful science. I'm all in for basic research, and we need to strengthen it and fund it better. But when it comes to clinical and public health, we need research that can change outcomes.
So, what are the features of useful clinical research? We need these eight features that I have thought about, and perhaps more. And this is a paper that pre-dated the pandemic by many years.8
First of all, we need to deal with problems, not create problems. A lot of the research that we do, instead of solving problems, it creates problems and that don't exist.
Second, context placement. We need to be honest about what we know and what we do not know. So, to decide on what we need to do next, not cut corners in that regard.
Information gain: ask ourselves, “Is that going to increase information that we have, compared to what we had before?”
Pragmatism: Is that real life? Is that a mathematical model? Is this - I do a lot of mathematical modeling - most of that is irrelevant - or is it real life data that we're talking about?
Patient-centeredness or community-centeredness. Have we asked patients, have we asked the community what they care about? Not what I care about or what other researchers care about or what experts care about, but what are the values that are important for the average person out there? Not for the politician, not for the policy maker, not for the legend scientist, not for irrelevant people -- for the real people.
Value for money. I heard some numbers about $26 trillion. I lost count. We have a lot of money that has been wasted, but at least let's invest in something that might work.
Visibility. Let's not be overtly ambitious. We can do things that can be done.
And finally, transparency: can we trust what we see in the scientific literature? Are the methods, the data, the analysis -- are they verifiable? Can we believe them?
It is okay to disagree. Inferential reproducibility may be doomed, to be modest, by its very nature.
Along with Howard Bauchner, we have published this short piece where we say that, you know, not just in COVID, but in anything else, you can have a lot of evidence and you can have the best experts in the world [descending?] on their perspective, their specialty, their sponsors, their biases. They may reach different conclusions - this is perfectly fine, but we should try to make this visible for everyone to see -- why they shape these opinions, why they shape these beliefs, why do they make these recommendations, and what is the evidence? So, openness in terms of sharing data.
I heard all this discussion, people disagreeing on the lab leak hypothesis. I want to believe that it was a natural evolution, but I cannot exclude, “It's a lab leak.” We should have all the lab data available. We should have all these notebooks available for every scientist to be able to see same thing. For other studies, we should be able to see the raw data, the codes for the models that, you know, dictated our lives, practically, for years. What are they?
And also conflicts: disclosures of funding, disclosures of who funded what, disclosures of advocacy, disclosures of activism, disclosures of political positions sometimes, you know, that might be important - and other conflicts of interest. The scientific literature is still pretty dark in that regard. We have run tax mining exercises of the entire life sciences literature, and very little of that is happening.
There is a disclosures placeholder, but we realize that they're incomplete. But in terms of open access to data and code and protocols, we don't have much.
We should accept that science is not vote counting.9
My bias - and that was a bias that I had published even before the pandemic - was that, I'm really sorry, but I will not sign declarations, memoranda, open letters, you know, whatever manifestos. I think it would be dangerous for me to do that. I'm in a position of power. All the people who work with me might be intimidated to think that, “Well, since John signed, we need to sign also.”
I also want to be able to change my perspective if I had something wrong. I don't want to be stuck that I signed a Manifesto and now that's completely wrong. I want to be able to change. And I really feel threatened by the advent of Science by Vote-Counting, by numbers, by “I have more on my side.” I feel unfortunate that, on most positions currently, I have more on my side because, goodness, we might well be very wrong, and I'm really looking to that person who will prove us wrong.
For those who are interested, please join us next year in the International Congress on Peer Review.10
We want to try to understand how to improve science through the regular way that this happens, which is peer review. Some people say we need no peer review. Well, let's see if that would work. We need different types of peer review. Who is the reviewer of the reviewer, whom can we trust, and so forth.
At the moment, there's very little space in the official literature to look back. Papers are published; they sit there. It's very difficult to challenge them.11
I think we need an open spirit of challenging, of destroying sometimes, some of the things that we believed, if they were incorrect.
We need to do that constructively - not obsessively. I think that the pandemic unleashed a lot of obsessive commenting at all levels, with blogs, with - even in scientific cycles.12
And finally, we need to think positively about the future. I don't want to think that our future will be a death spiral of wrong decisions.13
We have described how that would work, unfortunately for humanity, if we allow authoritarianism. And unfortunately there is authoritarianism around us. If we allow inequalities - and unfortunately there are inequalities around us. If we allow people who are marginalized to be more marginalized - and unfortunately this is happening as we speak. It may not be so obvious in this room, but it's happening out there in our community. If we allow the poor to become more poor, if we allow the oppressed to become more oppressed, if we allow the silenced to become more silenced, if we allow humanity to disappear.
Thank you very much. Thank you.
Related (21 Nov 2024)
Some Things John Ioannidis Was - and Still Is - Wrong About
John Ioannidis is wrong about a lot of things when it comes to COVID - and he knows it.
“Enhanced” meaning that I have inserted the presentation slides and links to biographies of people who are named, studies cited/shown, or other content Dr. Ioannidis references.
“previous publication” https://royalsocietypublishing.org/doi/full/10.1098/rsos.210389
The dexamethasone mention is possibly a reference to this study published in July 2020: https://www.nejm.org/doi/full/10.1056/NEJMoa2021436
Papers shown on slide: https://onlinelibrary.wiley.com/doi/10.1111/eci.13162 | https://www.bmj.com/content/371/bmj.m4048
Paper shown on slide: https://www.bmj.com/content/382/bmj.p1992
Paper shown on slide: https://pubmed.ncbi.nlm.nih.gov/36039285/
Extraordinarily, he manages to falsify the pandemic hypothesis without realising it.
He seems to be a very confused, conflicted man. First tenet of scientific study of something must be that the something exists. Studying an entity based on dogmatic linear thinking and assumptions of existence leads to horrible studies and misleading, dangerous results.