Featured Report - Shifting Sands: Flimsy Food Findings

“Trust the science.” For the past two years, we’ve heard this mantra repeated constantly by government officials, school administrators, and next-door neighbors alike. The maxim itself is simple—but as soon as you start asking questions, the clarity evaporates. What is “the science”? Who decides what “science” is worth trusting, and to what extent? At what point can we say the trust has been misplaced and that “the science” should be reevaluated?
 

COVID brought these questions to the forefront, as the CDC and other agencies established and reestablished guidelines and procedures throughout the pandemic that were said to be based in science (never mind the contradictions from month to month). But the federal government’s emphasis on trusting “the science” is nothing new—COVID only provided a new, more public application of the principle. Nearly every aspect of our daily lives, from the food we eat to the air we breathe, is controlled and directed according to the federal government’s understanding of “the science.”
 

Before the accusations start flying, I’m not “anti-science” (whatever that means), nor do I have any desire to return to an era without modern medicine or technology. We at the National Association of Scholars simply believe that it’s important to spend the time, effort, and resources to ensure that the regulations that shape American society are grounded not only in “the science” but in the best available science. And, yes, there is a difference.
 

Many scientific and social-scientific disciplines today are plagued by an irreproducibility crisis that calls into question the integrity of entire bodies of research—including many studies that are viewed as foundational in their discipline. In 2018, NAS published an in-depth report on this crisis titled The Irreproducibility Crisis of Modern Science: Causes, Consequences, and the Road to Reform. The report found that “improper research techniques, lack of accountability, disciplinary and political groupthink, and a scientific culture biased toward producing positive results together have produced a critical state of affairs” in which “many supposedly scientific results cannot be reproduced reliably in subsequent investigations.” In other words, the science can’t be trusted.
 

The consequences of this crisis extend far beyond the walls of the ivory tower. In a series of four follow-up reports titled Shifting Sands: Unsound Science and Unsafe Regulation, NAS researchers are examining the extent to which federal regulations have been justified by irreproducible research. The first report, Keeping Count of Government Science: P-Value Plotting, P-Hacking, and PM2.5 Regulation, focused on the field of environmental epidemiology and the unreliable research that underlies many of the Environmental Protection Agency’s rules and regulations. The second report, Flimsy Food Findings: Food Frequency Questionnaires, False Positives, and Fallacious Procedures in Nutritional Epidemiology, was released just a few hours ago and examines the troubling flaws in the research underlying many of the Food and Drug Administration’s policies.
 

The reports are both very technical, diving deep into the flawed statistical methods and research practices that make key findings within the fields of both nutritional and environmental epidemiology unreliable. But the motivating principle behind the reports is quite simple. The report authors sum it up nicely in the executive summary to Flimsy Food Findings:

The government should use the very best science—whatever the regulatory consequences. Scientists should use the very best research procedures—whatever result they find. Those principles are the twin keynotes of this report. The very best science and research procedures involve building evidence on the solid rock of transparent, reproducible, and actual reproduced scientific inquiry, not on shifting sands.

We’re not pushing a particular agenda here. Believe it or not, I don’t have particularly strong opinions about nutritional or environmental epidemiology. It’s certainly possible that many of the unreliable studies, if conducted again with sound research methods, would produce similar results. But as we have been reminded again and again over the past two years, “the science” matters—and the studies that shape our lives should be conducted in a way that makes them worthy of our trust.
 

Until next week.
 

P. S. In an article published earlier this month by the James G. Martin Center for Academic Renewal, NAS Texas Affiliate Head Richard Lowery writes about the effort by conservative faculty at the University of Texas at Austin to create an independent institute to house faculty whose perspectives leave them shut out elsewhere on campus. Although the proposal for a new Liberty Institute was backed by the state legislature and influential donors, it was ultimately co-opted by UT President Jay Hartzell and progressive administrators. The donors and legislators who supported the original vision simply surrendered to the hostile takeover.
 

The affair offers a poignant lesson for those attempting to reform a hostile campus culture from the inside. In the end, Dr. Lowery explains, “The effort was undone not through the machinations of the campus Left but through weakness on the part of supposed conservatives…it was conservative politicians and donors, not Marxist faculty, who brought [the Liberty Institute] down out of their unwillingness to confront a supposedly prestigious Texas institution.” NAS commends Dr. Lowery and the Texas Association of Scholars for their continued determination to fight for intellectual freedom, however fierce the opposition.

About the NAS