Ting Yan on the Future of Multimode Survey Designs

Multimode surveys provide more options to collect data, enabling researchers to reach more people and different populations. Multimode surveys can improve coverage, increase response rates, reduce cost, and improve measurements. Ting Yan, Ph.D., a Westat Associate Director with 20+ years of experience conducting methodological research on all aspects of survey operations and data collection, shares her insights on the uses, benefits, and future of multimode survey designs, and why Westat is a leader in this area.

Q: What exactly are multimode surveys?

A: Multimode surveys involve using more than 1 mode of data collection to contact and interview potential respondents, including web, phone, mail, email, text, video platform, and in-person surveys, or a combination. With Westat’s design expertise, we can create an optimal way to combine multiple modes to contact respondents and encourage participation.

Q: Why is multimode research so prevalent these days?

A: Survey costs are increasing and response rates are declining. Previous ways of sampling and interviewing people are facing challenges. So we’ve adapted to this reality with other survey modes.

Q: How does Westat handle the complex sample processing and experimental conditions needed for this research?

A: We start with our team of expert statisticians, data scientists, methodologists, developers, and operational staff to tailor and implement strategies that are optimal for each survey. Next, we implement our proprietary multimode management system (M3) that we developed to allow for seamless integration of modes while controlling protocols, including sample flow and experimental allocation. M3 also holds all the survey data and paradata allowing for reporting across modes and conditions.

Q: What studies have used multimode research?

A: We have used multimode surveys for a number of surveys, including the Redesign for the National Intimate Partner and Sexual Violence Survey (NISVS) for the Centers for Disease Control and Prevention (CDC), the Survey of Doctorate Recipients (SDR) for the National Center for Science and Engineering Statistics (NCSES) within the National Science Foundation (NSF), the Medical Expenditure Panel Survey (MEPS) for the Agency for Healthcare Research and Quality (AHRQ), and the American National Election Studies (ANES), which was conducted for Stanford University and the University of Michigan with funding from NSF.

We tested several modes (web, paper, computer-assisted telephone interviewing [CATI]) in various sequences to evaluate the redesign of NISVS, and our M3 allows for seamless integration of modes while controlling protocols. For the SDR, we reach out to sample members by mail, email, and phone. They can respond via an online web questionnaire, a telephone interview, or a paper questionnaire. Also, they can start their survey in one electronic mode, and then, if interrupted, pick up at the last unanswered question in another electronic mode. We make sure the transition is seamless for the respondent.

Because of COVID-19, we quickly transitioned from computer-assisted personal interviewing (CAPI) to phone interviews for MEPS. For a 2021 supplement on social determinants of health, a self-administered questionnaire, we developed web and paper versions and are using email and SMS to invite respondents to complete the web version. In addition, for the 2020 ANES, we replaced the face-to-face survey with a multimode strategy that uses mail for invitation; web for screening; and web, video interviewing, and phone for pre- and post-election surveys.

Q: What are some of the biggest challenges we face in conducting multimode research?

A: The greatest challenge is determining what modes to use and how to sequence the modes. For instance, we must decide whether a sequential mixed-mode design or a concurrent multimode design is the best way to reach and interview a target population. This involves taking into consideration the survey’s requirements and budget limitations. But these challenges are minor because of our design experience.

Q: What are some ways Westat is innovating in multimode research?

A: We pioneered multimode research, and we continue to improve ways to contact, recruit, and interview respondents. We actively look to emerging technologies—for example, smartphones and video conferencing software—to collect data and seek alternative design protocols that will benefit our clients and ensure accurate, quality data. Westat is innovative in using responsive and adaptive designs in multimode surveys and in assessing and estimating comparability of data across modes. We continually consider forward-thinking strategies, developing and reimagining what can work better for respondents.