Document Tag: evaluation

Improving the Productivity of Education Experiments (2012)

Given scarce resources for evaluation, we recommend
that education researchers more frequently conduct
comprehensive randomized trials that generate evidence
on how, why, and under what conditions interventions
succeed or fail in producing effects. Recent experience
evaluating a randomized need-based financial
aid intervention highlights some of our arguments and
guides our outline of the circumstances under which
the examination of mechanisms and heterogeneous impacts
is particularly important. Comprehensive experiments
can enhance research productivity by increasing
the number of theories both tested and generated and
can advance policy and practice by exploring the conditions
under which interventions will bemost successful
in scale up. Paradoxically,while the emphasis on average
treatment effects is typically associated with efficiencyminded
economists, we argue that the approach is often
inefficient from the standpoints of science and policy.

Meal Vouchers Matter for Academic Attainment: A Community College Field Experiment (2023)

Given growing awareness of the high prevalence of food insecurity among college students, higher education leaders are
implementing various food interventions on their campuses. However, there is little research on the efficacy and impact of
these initiatives. Using data from a field randomized control trial, we find that a relatively modest financial investment in
campus meal cards coupled with proactive outreach by an existing campus office improved community college students’
academic attainment outcomes. Students who were invited to participate in the meal voucher program attempted and
completed more credits during their first year of college and were more likely to graduate in 2 years than otherwise similar
peers, indicating that a campus meal program can promote college success.

Supporting the Whole Community College Student: The Impact of Nudging for Basic Needs Security (2021)

Even after decades of improvement efforts, completion rates at community colleges remain low, particularly among students who need developmental education. Basic needs insecurity contributes to these low completion rates. As a result, community colleges throughout the country have launched benefits “hubs” to help students secure their basic needs. However, there is limited evidence on whether connecting students with these hubs improves academic success.

This report details an evaluation of the Advocacy and Resource Center (ARC), a benefits hub at Amarillo College (AC) in Texas. In partnership with AC, we conducted an evaluation to advance two institutional goals:

Increase utilization of the ARC, particularly among students most at risk of leaving college, with a low-cost technology-enabled approach.
Estimate the academic impacts of connecting students to the ARC.
Over the course of a year, we emailed randomly selected students from low-income households and those enrolled in developmental education coursework, informing them about and inviting them to the ARC. We found that the emails (or “nudges”) paid off for students who received them:

Rates of visiting the ARC more than doubled from 22% to 56%.
Developmental education students nudged to visit the ARC were 20% more likely to pass developmental education courses, a crucial milestone.
However, we did not find clear evidence that nudged students completed more credits, received higher grades, or passed other courses at higher rates.

As community colleges across the nation work to improve student success and help students recover from the pandemic, this evaluation offers two lessons:

Insufficient information about existing basic needs supports keeps students from getting the help they need. The information barrier may be effectively overcome with personalized nudging, a low-cost solution.
Connecting students to basic needs supports helps students make academic progress, particularly those in need of developmental education.