First Opinion is STAT’s platform for interesting, illuminating, and provocative articles about the life sciences writ large, written by biotech insiders, health care workers, researchers, and others.

To encourage robust, good-faith discussion about issues raised in First Opinion essays, STAT publishes selected Letters to the Editor received in response to them. You can submit a Letter to the Editor here, or find the submission form at the end of any First Opinion essay.

We welcome the call for new research into the role of nonpharmaceutical interventions (NPIs) during the Covid-19 response using U.S. county-level data, but we regret that Marc Lipsitch and Sara Cody’s recent First Opinion mischaracterized our 2023 Lancet study as a strawman in order to make the case for theirs. In doing so, they made errors that a fact check and a closer reading of our study would have identified.

After we alerted Lipsitch and Cody to the errors a week ago, they updated their original piece to address some of the mistakes, but others still persist. More importantly, the authors miss the opportunity to state plainly the real challenges of demonstrating the value of NPIs in those early years of the U.S. Covid response.

Lipsitch and Cody stated:

1. “A full analysis would consider in detail how people behaved, not only how they were told to behave.” Our study did. We assessed 10 policy mandates (individually and as a package) and multiple associated behaviors (mobility, mask-wearing, vaccination), finding both associated with fewer infections. Lipsitch and Cody now concede our study contains “some analyses of this question.”

1. They wrote that the study should have recognized that “adherence to control measures was strongly predicted by politics and trust.” Our study did. We state in that paper: “Our results suggest that partisanship reduced the use of protective mandates and behaviors.” Our study also finds social trust was associated with vaccine coverage, mask-wearing, reduced mobility, fewer deaths and infections, and our study indicates “trust played an outsized role in the COVID-19 pandemic.” Lipsitch and Cody now acknowledge our study addresses politics but make no mention of trust.

2. Lipsitch and Cody assert our paper did not account for early-hit states or variation in the timing of policy mandates imposed by states. Neither is accurate. We conducted the analysis for various starting dates in January, February, and March 2020 for sensitivity analysis, and we considered variation in state mandates. Our results were stable across those time periods.

3. Most concerning, Lipsitch and Cody mischaracterized the stated conclusion of our Lancet study, which was that NPIs were effective. We found states that imposed more protective mandates, such as mask use and mobility restrictions, maintained them for longer, and also those that had better compliance with those policies had lower infection rates.

The accurate statement about our 2023 Lancet study that Lipsitch and Cody make is that we did not find a statistical relationship between NPIs and deaths. The reasons are not, however, because our study did not assess behavior, politics, and trust, or account for early-hit states and the timing of mandates, all of which we did.

The reasons that our study did not find a statistical association between NPIs and deaths are those already highlighted in the discussion and limitations sections of that 2023 Lancet study. NPIs mattered, but they are not the only factor that did, especially with regard to deaths. While the empirical signal connecting NPIs, such as mask-wearing and infection, is direct, many factors (e.g., comorbidities like diabetes and obesity, access to quality health care) muddled the relationship going from infection to death, especially during that first year of the pandemic.

Nearly every U.S. state intervened with mandates within a several-week period, making it hard to assess what would have happened without any intervention. In states like Florida that imposed fewer, later, or shorter mandates, the public sometimes still adopted protective behaviors even without mandates. U.S. county-level data were not available in early 2023 for many measures, which meant, as we state in the study, “variation within states with districts and localities imposing different mandates” might “cloud the impact of the state effect that we are estimating.”

Not identifying a statistical relationship in real, imperfect, and noisy data is not the same thing as there being no effect, which we explain at length in our original paper and in subsequent press. There is a clear scientific record that NPIs mattered and impacted Covid-19 transmission.

We share the concerns over Covid revisionism and the blanket denigration of NPIs for future emergency response. Partisans and opportunists have magnified and mischaracterized the inconsistencies and inequities in the U.S. Covid-19 response to sow polarization and doubt about public health programs generally. Misrepresentation of the literature, specifically our work in this instance, only contributes to this failed dialogue. The appropriate counter to Covid revisionism is careful, rigorous, and honest assessment and reassessment of the strengths and weaknesses of the U.S. response that builds on the best available evidence of where and how we can do better. Only that will ensure better outcomes in the future.

Thomas J. Bollyky is the Bloomberg chair in global health and director of the Global Health Program at the Council on Foreign Relations. Joseph Dieleman, Ph.D., is a professor in the Department of Health Metric Sciences at the University of Washington and is faculty lead of the Resource Tracking team at the Institute for Health Metrics and Evaluation.