Program Integrity: Efficiently Collecting Data on Meals Programs

Increasing food security and reducing hunger among low-income children and adults is the mission of the Food and Nutrition Service (FNS). To support this mission, FNS administers 15 federal nutrition assistance programs. The Payment Integrity Information Act of 2019 requires FNS to identify and reduce improper payments in its programs. Westat provides FNS with reliable, statistically valid estimates of improper payments in its programs that serve meals to children in child care settings and in schools. Roline Milfort Rosen, Ph.D., P.M.P., a Westat Senior Study Director, discusses best practices in collecting and examining data and using innovative research strategies to minimize burden on respondents, maximize response rates, and ensure that the sample reflects the population in order to minimize bias.

Q: What are the differences in the various FNS improper payment studies, for example, the Third Access, Participation, Eligibility, and Certification Study (APEC-III) and the Erroneous Payments in Child Care Centers Study (EPICCS)?

A: The differences are related to the context, target population, and how errors can be introduced. In APEC, school food authorities manage the school meals programs for students and come with extensive experience and infrastructure. EPICCS is conducted in child care settings, which can be more fluid and typically serve meals family-style. These varying context result in different potential sources of errors and ways to mitigate them.

Q: Respondents may be apprehensive to report micro-level data that are considered sensitive. How is this overcome?

A: The first step is to build a rapport with respondents by demonstrating respect for their time, their data, their confidentiality requirements, and their protocols. We have an understanding of the complexities and sensitivities, and we have systems in place accordingly. For example, we have protocols in place for accessing schools and ensuring the secure transmission of data, and we collaborate with the client through the entire process. Our customizable protocols meet client needs and have high standards for data security.

Q: The increasing number of requests for survey data collections on respondents and the fact that their organizations may house their data in different formats can be challenging. How are these addressed?

A: We are very sensitive to “respondent fatigue” and recognize that respondents, such as school districts and administrative departments, are increasingly selective on what studies or surveys they choose to participate. Staffing and infrastructure limitations add to this challenge, but Westat leverages emerging technologies to reduce the amount of time required to complete data collection while also balancing this against the learning curve for understanding and accessing the technology. What’s more, we develop accessible tutorials or on-demand training on how to use the technology and are also prepared to develop customizable resources to alleviate barriers.

Another key to our data collection success is building in flexibility wherever possible. For example, we incorporate multimode methods and use the data in the format that is readily available to the respondent rather than compelling them to make major changes to their systems to be compliant. Our goal is to “meet them where they are.” Our team is skilled at various types of data extraction through programming and other methods. Saving the respondents even 1 or 2 steps in the process goes a long way.

Q: How do statistical techniques help reduce respondent burden, tease out more granular aspects of the data, and provide greater insight?

A: Employing sophisticated statistical methods such as Bayesian modeling helps save time, reduce cost, and address capacity issues when study results rely on large amounts of primary data. It also helped us develop an error forecasting model that uses current and historical APEC study data in combination with administrative data. Having such a model allows for less frequent data collection as FNS can use the forecasting model to estimate the error rates in years when it does not collect new data.