VISION Vaccine Effectiveness Network: Why It Succeeded

With the emergency phase of the COVID-19 pandemic behind us, interest has grown in understanding why the VISION Vaccine Effectiveness (VE) Network, launched in 2020 by the Centers for Disease Control and Prevention (CDC), succeeded in guiding vaccine policy decisions and providing vital information to the American public when other similarly focused initiatives fell short. Knowing what made this collaborative “virtual network” of 9 integrated U.S. medical systems, coordinated by Westat, able to rapidly deliver timely, robust data on vaccine effectiveness (VE) for different population segments, in different real-world conditions, and deepen knowledge about COVID-19’s outcomes and complications is critical to helping us fight other pathogens should they emerge.

Over the course of the project, the VISION network published over 25 papers, both in peer-reviewed journals and in CDC’s Morbidity and Mortality Weekly Report (MMWR). These publications served to update the scientific community about the effectiveness of the COVID-19 vaccines and to provide policymakers with data to inform decisionmaking. In one especially productive period, the network published 14 manuscripts in less than 1 year. The following figure shows all the different publication efforts that were occurring simultaneously during this period.

Seeking insight into the success of the VISION network, Westat turned to those on the frontlines of this work—the Principal Investigators (PIs) at the 9 medical systems engaged in the VISION Network—as well as the director of the VISION project, Westat’s Sarah Ball, ScD, MS, MPH, Vice President and Lead Scientific/Epidemiology Advisor.

For over 2 years, the PIs oversaw their organizations’ rapid extraction of data from electronic health records (EHRs) and vaccine registries related to SARS-CoV-2 testing, COVID-19 vaccinations, clinical characteristics and discharge diagnoses from COVID-like-illness-associated medical events, and patient demographic information. This was possible because of decades of investment by collaborating institutions into their data systems so they were ready to provide high-quality data during the pandemic.  

Once gathered, the data were securely transferred to Westat, which conducted quality checks and combined the data into analytic datasets with additional derived variables (such as underlying medical conditions). The data were then delivered biweekly to CDC to inform key policies to protect Americans and establish resource allocations. The 9 sites that submitted data to Westat were:

• Baylor Scott and White Health (Texas)
• Columbia University Irving Medical Center (New York)
• Children’s Minnesota
• HealthPartners (Minnesota and Wisconsin)
• Intermountain Healthcare (Utah)
• Kaiser Permanente Northwest (Oregon and Washington)
• Regenstrief Institute (Indiana)
• University of Colorado
• Vanderbilt University Medical Center (Tennessee)

What Contributed to the Network’s Success?

PIs responding to this question pointed to the swift, continuous turnout of high-quality, real-world data, the extensive and diverse sample of patients, CDC’s leadership and Westat’s ability to quickly scale up operations, coordinate the sites’ output, and provide ongoing technical and analytic support to sites. Specifically, they and others involved in the project described the following factors.

Timely, High-Caliber Real-World Data

“VISION’s comprehensive collection of standardized data from geographically diverse health care systems is one of the major factors…,” said VISION PI Malini B. DeSilva, MD, MPH, Research Investigator and Travel and Tropical Medicine Physician at HealthPartners. By “using these standardized data, the network was able to utilize a test-negative study design to provide COVID-19 VE estimates in a timely manner.” She added that eliminating the need for chart reviews to capture the information from patients who tested positive or negative for SARS-CoV-2 enabled “faster analyses,” which helped guide policy decisions.

Pointing to Westat’s support, VISION PI Manjusha Gaglani, MBBS, FAAP, FIDSA, FPIDS, Director of the Center for Research in Vaccines and Infections (CRVI) at Baylor Scott and White Health, Texas, remarked that it was “Westat’s great coordination with timely updates to the data dictionary and regular communication to provide timely feedback to sites [that led to] improvement in data quality.”

Toan C. Ong, PhD, VISION PI and Associate Professor at the University of Colorado Anschutz Medical Campus, commented that “despite many challenges, VISION sites never stopped trying to improve the quality of the data, either through internal data exploratory activities or external collaboration with other sites.”

VISION Project Director Ball said by developing a common protocol for harmonized data collection and procedures across sites and implementing an innovative data management system, Westat was able to deliver real-world data and real-world evidence.

“Supplying the government with timely data was essential to its understanding of the early epidemiology of the disease, VE among high-risk groups, and when VE was waning—all of which was necessary to formulating policy decisions,” Ball said.

Diverse Sample Populations

Were it not for the large sample size of diversified populations, key questions about COVID-19’s VE for different groups would not have been possible, said DeSilva. Echoing this comment, VISION PI Anupam Kharbanda, MD, MSc, Clinical Vice President and Chief of Critical Care Services at Children’s Minnesota, added that this diversity—“both in terms of age, geography, and racial make-up”—stemming from the collaboration of geographically diverse health systems—contributed to VISION’s success.

Collaboration, Resilience, and Dedication

“VISION [was] the most demanding and fast-paced project that I’ve ever been involved in,” said Ong. “Yet our teams in Colorado and all other teams of the VISION sites were resilient, dedicated, and rose to the challenge…to respond to the needs of [delivering] data and scientific expertise.

“We had a talented team that stayed focused and worked tirelessly to rapidly analyze SARS-CoV-2 vaccine effectiveness data and disseminate results…,” said Gaglani.

“Credit goes to CDC for encouraging us to collaborate with subject matter experts and leaders in epidemiology, infectious diseases, and data science at our collaborating sites,” said Ball.

CDC’s Leadership

CDC’s leadership was unquestionably a factor in the Network’s success, said Ong. Expounding on this, Ong explained that CDC’s “decisive actions and support steered the project team in the right direction in response to the emergency of the pandemic. CDC’s leadership was stellar,” he exclaimed. “Together with the Westat team…CDC colleagues always tried their absolute best to clearly define the objectives, data requirements, and data definitions.”

Westat’s Exemplary Support

“The importance of Westat’s role as a data coordinating center…cannot be overstated,” said Ong. [The Westat] team …provided “excellent and unwavering support technically and scientifically. Things ran smoothly and timely because of the professionalism, courtesy, and competency of [the] team.”

“The Westat team was willing to ask important research questions, explore different analytic methods, and find innovative solutions to move the epidemiology and VE science forward,” Gaglani noted.

“Our collaborative work contributed to CDC’s understanding of the disease and the effectiveness of the vaccines,” said Ball. “Importantly, our research contributed to decreased morbidity and mortality from COVID-19.”