Advancing Web-Based Surveys with Multimode Outreach

The web-based survey is an important way to gain critical information fast and inexpensively. Westat has significant experience in encouraging people to participate in online surveys. This includes surveys of the general population, as well as specialized groups, such as college students, physicians, and business executives. For example, Westat recently piloted a design that pushed 70% of general population respondents who normally respond on paper to the web.

“Online surveys have several advantages,” says David Cantor, Ph.D., a Westat Vice President, Senior Statistical Fellow, and principal investigator. “They allow for inexpensive, rapid collection of data, and they make it easier to recruit large sample sizes. They also enable us to access geographically and demographically diverse audiences.”

Other Benefits

Both the respondent and the researcher benefit from web-based surveys:

• Surveys can be self-administered, which is especially beneficial when asking questions on sensitive topics.
• The “interviewer effect” is eliminated:
  • People aren’t inclined to respond in a way that they think the interviewer wants them to. Instead, respondents feel comfortable in answering questions honestly and can be ensured of their anonymity.
  • The interviewer doesn’t mediate how the question is delivered (e.g., misreading; idiosyncratic intonations that affect interpretation).
• The computer drives the navigation, rather than relying on the respondent to read and interpret which question should be answered next.
• Answers can be checked by the computer to see if they are plausible (e.g., whether a number is in a possible range; if one answer conflicts with a previous one).
• These surveys offer the opportunity to embed probes that invite respondents to rethink answers or take more time to answer, providing another quality check.

Range of Topics

Dr. Cantor notes that Westat is conducting a variety of online surveys on a range of topics, including health, criminal justice, and veterans issues. For example, data are being collected for the National Cancer Institute to understand how adults use different communication channels to obtain vital health information for themselves and their loved ones.

In a longitudinal survey that is conducted entirely online, researchers are uncovering data on the risk of domestic or intimate partner violence among young adults ages 18 to 24. To encourage participation, Westat created a youth-oriented website, using graphics and themes associated with online gaming.

For the Wounded Warrior Project (WWP), Westat conducted regular web-based surveys of WWP alumni to identify trends among its alumni, compare their outcomes with those of other military populations, and measure the impact and mix of WWP services and programs.

Westat Provides Savvy Solutions to Limitations

Using the web for surveys, however, has 2 important limitations.

Scope. It is difficult to directly contact the general population by web. There is no universal list of email addresses that can be used to contact respondents.

Westat has been experimenting with ways to “push” respondents to the web by using address-based sampling (a general population list) to send snail mail to households, requesting them to answer by web. As part of this effort, we have been offering different incentives, such as gift cards or cash, to motivate respondents to answer by web.

Population gaps. There are still gaps in who accesses the web. Older people, as well as those who do not have easy access to the internet, are often left out of online survey respondent pools. This affects the representativeness of surveys that are restricted to the web.

Westat overcomes this issue by supplementing web surveys with other modes, such as paper surveys. The answer lies with mixing different survey modes to take advantage of the web while still ensuring that results are representative of the full population. In several of Westat’s studies, some respondents use the web, while others respond on paper or some other mode. In these studies, we are exploring how best to accommodate the modes while still maintaining comparable data.

Efficiencies with web-based surveying will continue to develop. Westat continues to build on the opportunities web-based surveys offer while also exploring innovative ways to overcome any potential pitfalls that may restrict respondents from participating.