All Projects

How can text data be quickly and accurately processed?

Using natural language processing and deep learning to classify comments in MEPS

The Medical Expenditure Panel Survey (MEPS), funded by the Agency for Healthcare Research and Quality, is a set of large-scale surveys of families and individuals, and their medical providers across the United States.

More than 20,000 open-ended comments are entered by the interviewers each year into the MEPS computer-assisted personal interviewing (CAPI) system to clarify the respondents' answers.

Then, a group of human coders reviews each sentence to assign a topic label to the sentences out of 10 predefined classes, and use the associated procedures to further process the data. Also, MEPS is a panel study and there is a short time window, usually a week, to process the comments so that the data can get back to the field staff for use for dependent interviewing in the next wave. Processing this data is labor intensive and time consuming.

Westat harnessed the power of artificial intelligence (AI) capabilities to make the process more timely and efficient.

Westat uses natural language processing, machine learning, and deep learning techniques to train a classification model to automatically label the comments into 10 predefined classes.

We then deploy the model as a RESTful API in production so that it can run in the backend of the system used by the human coders. The model suggests the top 3 classes for each sentence ranked by classification probability, which allows human coders to make a selection out of 3 rather than 10 when reviewing the comments.

The data tool has been in production for the past 2 data collection periods in 2020. The tool achieved more than 95% classification accuracy for the top suggestion in processing 10,000+ comments for each round, with an efficiency gain of about 5% and reducing backlog to virtually zero.

Want to work with us?
You’ll be in great company.

About Us Careers

Westat Employees.
Westat Employee.
AAA Foundation for Traffic Safety
Baltimore Metropolitan Council
Centers for Disease Control and Prevention
Centers for Medicare & Medicaid Services
Chicago Metropolitan Agency for Planning
DC Public Schools
Georgia Department of Transportation
Internal Revenue Service
Leadership Montgomery
Maryland Cancer Registry
Michigan Department of Health and Human Services
National Science Foundation
NYC Mayor’s Office for Economic Opportunity
Organization for Economic Cooperation and Development
Robert Wood Johnson Foundation
Social Security Administration
Substance Abuse and Mental Health Services Administration
Teach for America
Texas Education Agency
The Johns Hopkins University
The National Institutes of Health
The Verizon Foundation
U.S. Department of Agriculture
U.S. Department of Education
U.S. Department of Health and Human Services
U.S. Department of Justice
U.S. Department of Transportation
U.S. Department of Veterans Affairs
University of Maryland Baltimore Campus
University of Michigan
University System of Maryland
Explore Our Clients

Please wait...

Forbes 2023 America's Best Mid-Size Employers Powered by Statista Forbes 2022 The Best Employers for Women Powered by Statista

Westat is an Equal Opportunity Employer and does not discriminate on the basis of race, creed, color, religion, sex, national origin, age, veteran status, disability, marital status, sexual orientation, citizenship status, genetic information, gender identity or expression, or any other protected status under applicable law. Notices to Employees & Applicants.