How does the public learn about cancer and cancer prevention?

Challenge

Knowing more about how Americans get their information about cancer, their knowledge, attitudes, and use of cancer-related information, and their cancer prevention behavior helps the National Cancer Institute (NCI) choose the most effective communication strategies, messages, and channels to understand, reach, and motivate various populations. NCI initiated the Health Information National Trends Survey (HINTS) and awarded the HINTS contract to Westat in 2003 to research these important questions. HINTS is focused on monitoring changes in the rapidly evolving fields of health communications and health IT.

Solution

Collaboration is key in this work, and our instrument design specialists work in tandem with NCI to ensure that the questions asked of survey respondents can be both clearly understood and yield reliable, meaningful responses. All new survey questions are tested before being added to a HINTS instrument. We conduct cognitive testing with English-speaking respondents, as well as populations targeted by specific rounds of HINTS (i.e., cancer survivors, caregivers, tobacco users, and rural residents).

Our methodologists create mailing and web materials to encourage survey response; periodically, Westat conducts focus groups to elicit feedback about what messages would inspire a respondent to complete the survey. The focus groups have included English- and Spanish-speaking respondents.

For each round of data collection, Westat’s statisticians develop a single-stage, stratified national sample of non-institutionalized adults living in the U.S. Since 2011, Westat has shifted from telephone interviews to collecting data by mail using a standard mailing protocol. Paper surveys were made available in English and Spanish; but in 2019 and 2022, we offered respondents the option of a web survey.

In 2022, Westat fielded HINTS 6 with a sample size twice as large as in recent years with a target of 7,000 completed surveys. Special topics such as discrimination experiences, use of telehealth, and belief in false health messages were included in this round. In addition, web respondents received experimental prompts designed to lead to better respondent behavior. The HINTS 6 sampling plan included enhanced contacts for Spanish-speaking respondents.

Maximizing Response Rates

To increase response rates, reduce bias, and obtain the highest quality data, Westat runs methodology experiments. These have focused on incentive levels, number and method of telephone calls, respondent selection methods, order and contents of mailings, cover appearance, grid item formatting, and activities to encourage response from Spanish-speaking respondents. In addition, Westat used a split-sample method (half telephone, half mail) when the survey was transitioned from telephone to mail, allowing analysts to monitor trends across the time series, regardless of interview mode.

In 2019, Westat conducted a similar dual-mode experiment (mail and web) to test whether offering respondents the web option affected response rates or data quality. This dual-mode pilot included an additional incentive for responding via the web and interventions for prompting web respondents with incomplete or low-quality responses. The 2019 pilot was successful, demonstrating that we could get a good response using a dual mode. Also, the web experiment brought in people who were young, low-educated, and healthier so the overall result was data that were more representative of the population.

To increase minority participation, we oversample in high-minority areas. For the most recent HINTS data collection, Westat’s statisticians further stratified the sample by rural areas to boost the number of rural participants. We adopted special procedures to increase Spanish-speaking respondents, including trying to identify which households previously sampled were likely to be Spanish-speaking and sending both Spanish- and English-language materials to these households. We offer a 1-800 number so if respondents did not receive Spanish-language materials, they can call to request them. To facilitate analysis for specific regions of the country, Westat has added codes to the dataset after collection that identify, for example, respondents from Appalachia or the Mississippi Delta in addition to other rural areas.

Delivering a Cleaned, Weighted Dataset

For each round of HINTS, Westat statisticians have created full-sample weights and replicate weights for all survey respondents. The full-sample weights are used to calculate population and subpopulation estimates. Replicate weights are used to compute standard errors for these estimates. The procedures include calculating household-level weights, adjusting for household nonresponse, calculating person-level initial weights, and calibrating the person-level weights to population counts.

Additionally, the Westat team delivers a cleaned and weighted dataset with a codebook/data dictionary that defines all variables in the questionnaire for each round of data collection. These items are accompanied by an annotated instrument and a methodology report describing that particular round.

Including Cancer Survivors

In one round, we worked with NCI’s Surveillance, Epidemiology, and End Results (SEER) program to identify cancer survivors through central cancer registries. This provided a larger sample size of cancer survivors than a typical HINTS survey administration and allowed NCI to better identify the communication needs of this population. SEER variables were added to the HINTS-SEER dataset to provide additional information about the respondent’s experience.

HINTS panel

In early 2022, Westat began recruiting for the HINTS Panel to build a probability-based web survey group of respondents agreeing to be surveyed regularly. The panel is designed to provide NCI with supplemental heath communication data between the regular HINTS data collections and enable the agency to survey the same respondents over time. HINTS Panel recruitment will continue with HINTS 7 in early 2024.

Results

The data generated by HINTS have deepened NCI’s insight about Americans’ knowledge, prevention behaviors, and attitudes about cancer and the degree to which they understand cancer prevention messages. This insight enables NCI and practitioners to fine-tune and target their messaging to encourage greater awareness of cancer prevention methods.

HINTS data can be found in over 700 peer-reviewed articles, and the data have been the focus of a number of presentations at both health conferences such as the Society for Behavioral Health as well as at methodology conferences such as the American Association for Public Opinion Research.