By Robert L. Moore, MD, MPH, MBA, Chief Medical Officer
“Half of science is asking the right questions”
-Francis Bacon, father of empirical science
The recent troubling activities by the U.S. Food and Drug Administration (FDA) and the Advisory Committee on Immunization Practices (ACIP) are part of a larger pattern of drawing conclusions based on unacknowledged cognitive biases.
Charles Piller, an investigative journalist of the prestigious journal, Science, recently authored “Doctored: Fraud, Arrogance, and Tragedy in the Quest to Cure Alzheimer’s,” which meticulously catalogues a huge volume of falsified images and data related to the nature of Alzheimer’s disease. Piller’s work has led to the retraction of numerous scientific papers going back more than three decades and a cessation of National Institutes of Health (NIH) grants to the neuroscientists associated with these studies, several of which are well-documented in Piller’s New York Times guest essay, “The Devastating Legacy of Lies in Alzheimer’s Science.”
Piller’s book narrates the story of an individual whistleblower who reported evidence of fraud to the NIH, FDA, individual universities, and academic journals starting in 2021. The overlapping interests of those not wishing to expose the irregularities and potential fraud led to little response. It was not until the details were revealed to the wider press and in Congress that several actions were taken in 2024.
Prior to publicizing the results, Piller offered every researcher suspected of doctoring results the opportunity to review the evidence and comment or be interviewed. Those who accepted the offer were most often the co-authors of the papers publishing results of experiments performed by others. For each researcher who was found to have fabricated the results, there were several co-authors, many world-famous, whose biases led them to ignore negative experimental results incongruent with “groundbreaking” results, potentially yielding them wealth and fame — confirmation bias in action.
Piller identified one professor who routinely berated graduate students whose lab experiments did not produce desired results, which could disprove the theory. Conversely, the professor showered praise and privileges on graduate students whose results favored the preferred theory. Although the professor did not explicitly tell students to falsify the results, the culture of the laboratory created an environment lacking integrity to the scientific method in favor of obtaining particular results.
Significant financial conflicts of interest were also a major driver, enticing scientists to look the other way when fabricated results or poor-quality studies made a particular drug look promising. In such cases, the scientists or pharmaceutical company leaders would overzealously promote the promise of a drug if it served their personal financial interests. Scientists who favor a particular result plainly reveal their bias towards that result, and extra caution is warranted before accepting any results at face value.
In the area of Alzheimer’s disease research and drug development, precious resources and research dollars were siphoned away from legitimate Alzheimer’s research projects and granted to those whose papers were based on fabricated results. In retrospect, these were dead-ends, and the totality of harm is immeasurable.
Piller’s book focuses on and reveals misbehavior in scientific research. Insights into the behavioral economic origins of this misapplication of the scientific method can also be applied to the area of health policy.
In an ideal world, health policy decisions would be made based on an understanding of science and statistics. Policies would also apply the principles of quality improvement: iterating repeated small[1]scale pilots before mandating state-wide implementation of a new approach, rather than using a compliance policy approach for lofty aspirational goals better suited to incentives.
Well-crafted policy relies on several principles in common with good scientific research: humility and willingness to test new ideas objectively and acknowledge failure as we learn, so we can make the next attempt better. Having a pre-conceived bias or propensity for a particular policy approach, coupled with an unwillingness to logically consider scientific and statistical arguments, will lead to policy that is ineffective at achieving its goals.
While we do need to have visionary leaders with charisma and energy, this should ideally be coupled with an understanding of the importance of keeping the scientific method and quality improvement methodology at the forefront in the policy development process.