Conducting National Surveys in the Age of COVID-19

Leyla Mohadjer
Leyla Mohadjer discusses how innovative uses of statistical tools and techniques are key to maintaining data quality and integrity during the pandemic.

Conducting data collection for national surveys during a crisis is not unusual. Hurricanes, earthquakes, wildfires, civil unrest, and other emergency situations have posed challenges to data collection in geographically localized areas at different times. But how do researchers tackle the challenges of conducting a nationally representative survey during a health pandemic? “The situation with this COVID-19 pandemic is especially unpredictable,” notes Leyla Mohadjer, Ph.D., Westat Vice President and Associate Director for Statistics and Evaluation Sciences. “So, we continually plan for different scenarios and quick turnaround strategies that rely on the fundamentals of statistical research and survey methodology.”

Proactive planning is key to conducting survey research especially at this time. Westat is in ongoing communication with our clients on strategies to move forward on in-person data collection. “We’re continually taking the temperature of projects as we move forward and continually planning for different contingencies depending on the expected timing of returning to the field,” explains Dr. Mohadjer.

Mode Changes in Surveys

Many surveys that were conducted in person have transitioned to telephone or web surveys. “We’re studying any mode changes to see if there is a mode change effect. If so, we have developed plans for adjusting the final samples to minimize the impact of mode change. Also, for some surveys, combining data from the pre- and post-COVID period may not be justifiable,” cautions Dr. Mohadjer. “This is true for surveys where the pre-COVID and post-COVID respondents are coming from different parts of the country. In some cases, the statistics of interest, like employment status, changed drastically from pre- to post-COVID. We have developed special sampling procedures that create a bridge between the pre- and post-COVID samples to best deal with these kinds of situations.”

Response Rates Are Key

Getting high response rates is critical no matter how the survey is conducted, but the U.S. Census Bureau and the Bureau of Labor Statistics have already raised concerns about lower response rates for their large surveys. “We need to prepare for a similar impact on response rates in surveys we conduct,” notes Dr. Mohadjer. “In some cases, it is a matter of working with survey methods specialists to devise ways to improve response rates. We are also mitigating this risk by doing extensive analyses to assess bias due to nonresponse, and to adjust the resulting sample to reduce bias in outcome statistics by applying various sample weighting adjustments, like benchmarking.”

Lower response rates not only impact the representativeness of the sample but also result in a smaller sample size for the survey. “One way we to mitigate this,” Dr. Mohadjer explains, “is to split the selected sample into a main sample and many smaller random samples, with the main sample being released to the field first. Throughout the course of data collection, Westat monitors data collection activity in real time and is prepared to release new samples if need be. We can implement this seamless process because of our integrated network of survey statisticians, survey operations staff, field data collection oversight, and analysis staff. If the original sample released to data collectors is not providing enough responses,” she continues, “we’ll release another subsample of different households, maintaining random representativeness of the survey at the same time as aiming to achieve the desired sample size.”

Dealing with Unpredictable Shifts

Part of the planning for surveying during a pandemic is to also manage potential pauses in data collection either in local hot spots or across the country. “We have to be cognizant and flexible as different parts of the country are affected at different times, and how we can reduce bias in samples that we’re collecting,” notes Dr. Mohadjer. Westat has established procedures to minimize disruption and risk by developing data collection plans with quick turnarounds, restarting data collection quickly after a pause, or changing the mode of data collection if needed.

Focus on Quality Data Collection

“At Westat, we put priority on the quality of the data collected during the pandemic, and also prepare for correcting for any potential bias through sample adjustment and weighting,” continues Dr. Mohadjer. “We share the methods that are working effectively across different data collection projects, and this benefits all Westat projects and clients.”

Statisticians also harness other tools, like data linkage, where data sources are linked to augment or replace missing data as a result of the pandemic, and data science techniques such as machine learning to quickly gather large amount of data and conduct various analyses either using paradata or external data sources for various needs.

The rapidly evolving nature of the COVID-19 pandemic requires researchers to quickly adapt and be nimble in meeting the challenges of sampling and data collection. While the challenges may appear daunting, innovative uses of statistical tools and techniques, including adaptive survey methods, are instrumental in achieving the quality and integrity of the data.

 

The situation with this COVID-19 pandemic is especially unpredictable, so we continually plan for different scenarios and quick turnaround strategies that rely on the fundamentals of statistical research and survey methodology.

- Leyla Mohadjer, Ph.D., Westat Vice President and Associate Director, Statistics & Evaluation Sciences

Want to work with us?
You’ll be in great company.

About Us Careers

Westat Employees.
Westat Employee.
Centers for Disease Control and Prevention
Centers for Medicare & Medicaid Services
Substance Abuse and Mental Health Services Administration
The National Institutes of Health
AAA Foundation for Traffic Safety
The Johns Hopkins University
University of Maryland Baltimore Campus
University of Denver
U.S. Department of Veterans Affairs
U.S. Department of Transportation
U.S. Department of Justice
U.S. Department of Health and Human Services
U.S. Department of Education
U.S. Department of Agriculture
Toyota
The Verizon Foundation
Texas Education Agency
Baltimore Metropolitan Council
Teach for America
Social Security Administration
SiriusXM
Robert Wood Johnson Foundation
Organization for Economic Cooperation and Development
NYC Mayor’s Office for Economic Opportunity
National Science Foundation
Michigan Department of Health and Human Services
Maryland Cancer Registry
Internal Revenue Service
Georgia Department of Transportation
DC Public Schools
ClearWay Minnesota
Chicago Metropolitan Agency for Planning
University of Michigan
Explore Our Clients

Please wait...

Forbes 2020 The Best Employers for Diversity Powered by Statista Copyright 2019 Forbes Media LLC. Used with permissionForbes 2019: The Best Employers for Women Powered by Statista Copyright 2019 Forbes Media LLC. Used with permission

Westat is an Equal Opportunity Employer and does not discriminate on the basis of race, creed, color, religion, sex, national origin, age, veteran status, disability, marital status, sexual orientation, citizenship status, genetic information, gender identity or expression, or any other protected status under applicable law. Notices to Employees & Applicants.