Document Tag: evaluation

Homelessness and Housing Insecurity Among Community College Students: A Longitudinal Evaluation of a Housing Choice Voucher Program (2024)

Housing insecurity and homelessness among American community college students are widespread
problems that reduce the odds of college attainment and undermine students’ health and well-being.
In 2014 Tacoma Community College and the Tacoma Housing Authority launched the College Housing
Assistance Program (CHAP) to address this challenge by offering housing choice vouchers to local community college students experiencing or at serious risk of experiencing homelessness. If students could
successfully navigate the application process and local housing market, the vouchers offered a short-term
subsidy to reduce their rent and hopefully promote degree completion. Over the next several years, CHAP
received national and regional awards and became a model for affordable college housing programs. This
evaluation examines its effects on students before the housing authority ended the program in 2022.

Machine Learning Who to Nudge: Causal vs Predictive Targeting in a Field Experiment on Student Financial Aid Renewal (2023)

In many settings, interventions may be more effective for some individuals than others, so that targeting interventions may be beneficial. We analyze the value of targeting in the context of a large-scale field experiment with over 53,000 college students, where the goal was to use “nudges” to encourage students to renew their financial-aid applications before a non-binding deadline. We begin with baseline approaches to targeting. First, we target based on a causal forest that estimates heterogeneous treatment effects and then assigns students to treatment according to those estimated to have the highest treatment effects. Next, we evaluate two alternative targeting policies, one targeting students with low predicted probability of renewing financial aid in the absence of the treatment, the other targeting those with high probability. The predicted baseline outcome is not the ideal criterion for targeting, nor is it a priori clear whether to prioritize low, high, or intermediate predicted probability. Nonetheless, targeting on low baseline outcomes is common in practice, for example because the relationship between individual characteristics and treatment effects is often difficult or impossible to estimate with historical data. We propose hybrid approaches that incorporate the strengths of both predictive approaches (accurate estimation) and causal approaches (correct criterion); we show that targeting intermediate baseline outcomes is most effective, while targeting based on low baseline outcomes is detrimental. In one year of the experiment, nudging all students improved early filing by an average of 6.4 percentage points over a baseline average of 37% filing, and we estimate that targeting half of the students using our preferred policy attains around 75% of this benefit.

Does Basic Needs Funding Improve Persistence Among College Students? Findings on How HEERF Dollars Impacted Student Persistence at SNHU (2023)

In October 2022, the Center for Higher Education Policy and Practice (CHEPP) published the first of a two-part series on the use and impact of Higher Education Emergency Relief Fund (HEERF) grants at Southern New Hampshire University (SNHU) and the national challenge of accessing basic needs for today’s learners. The first paper, Basic Needs Funding for College Students: What SNHU Learned During the Pandemic, detailed learnings from SNHU’s distribution of $107 million in HEERF dollars to 51,257 qualified learners, a sample which represented a mere 7.6% application rate among potentially eligible students. Housing, food, and transportation were identified as the greatest basic needs challenges for learners based on an analysis of the data. This paper examines whether HEERF had a significant positive impact on learner persistence among a sample of learners from the initial population of HEERF recipients at SNHU (n=47,381). It includes data and analysis on the impact of emergency grants and basic needs programs on persistence for higher education students to inform national policy discussions related to expanding learner access to such supports. Key Takeaways +Three out of five learners among a national sample (n=195,000) experienced basic needs insecurities in Fall 2020 (The Hope Center, 2021). The national impact of unmet basic needs on college persistence and success is not yet quantifiable. However, there is evidence that unmet basic needs negatively impacts learner outcomes, making the basic needs support gap an urgent challenge facing higher education and our nation. +Data indicates that emergency grants contributed to learners’ academic persistence at SNHU. Students enrolled at SNHU who received HEERF emergency grants (n=47,381) were more likely to stay enrolled in the next term, when compared with control groups. Specifically, HEERF II recipients were 15.5% more likely to stay enrolled and HEERF III recipients were 8.6% more likely to stay enrolled. +As a result of these findings, SNHU approved funding to pilot an emergency grant program for learners in need. This pilot was conducted during the 2023 Spring and Summer terms. Findings from the pilot will be used to inform future projects related to this topic at SNHU.

The Design and Testing of a Text Message for Use as an Informational Nudge in a Novel Food Insecurity Intervention (2023)

Food insecurity is an ongoing problem in the U.S. with implications for health problems and social disadvantages. Past food insecurity intervention studies have targeted barriers to accessing healthy food. However, information barriers have not been adequately addressed by food insecurity interventions. This research included the first phase of a two-phase feasibility study that involved testing a text message as an informational nudge through interviews among a small, preliminary sample of adults in the southwest U.S. (n = 10). Interview questions focused on the message content, persuasiveness, relevancy, clarity, and details that participants liked and disliked. A qualitative content analysis of interview responses then highlighted any necessary modifications to the text message. Interview responses highlighted that the message conveyed details about how to access food assistance resources. The message was perceived as informative, clear, and friendly. Participants highlighted the need for better spacing in the message layout, more details about the Supplemental Nutrition Assistance Program (SNAP), and a better explanation for why SNAP is relevant. Text message readability, relevancy, and informativeness were common concerns raised during interviews. Participant-informed modifications to the text message were applied in preparation for using the message in a food insecurity intervention. This research is a novel contribution to the extant literature on nudges, as no known intervention study has used informational nudges to exclusively promote food security. Future food insecurity research efforts should similarly test novel interventions to ensure the intervention is acceptable and relevant for the target population.

Addressing Food Insecurity on Campus: Connecting Students with Basic Needs Supports to Improve Academic Outcomes (2023)

This report highlights the efforts of four Arkansas community colleges to address students’ basic
needs by transforming their campus food pantries from supplemental food distribution centers to
basic needs hubs connecting students to a broad array of additional, more sustained basic needs
supports.
Drawing on college administrative data, this study assesses the benefits of this basic needs hub
model on students’ academic success. Results from regression analyses point to notable academic
benefits. Specifically:
• Students accessing the basic needs hub are 6 to 8 percentage points more likely
than students not accessing the hub to be enrolled one semester and one year later,
and to earn a credential.
• Low-income students, adult students, and students of color are more likely to access
basic needs hubs, driven by colleges’ targeted outreach efforts to key student
groups.
• The notable academic benefits of the basic needs hub are present for Pell recipients,
for adults, and for students of color – with especially high proportional increases in
credential attainment for students of color who access the hubs.

When Care Isn’t Enough: Administrative Burden in Federal Higher Education Pandemic Emergency Aid Implementation (2022)

Departing from traditional financial aid policies, during the pandemic the federal government
introduced emergency aid to higher education for the first time. This study examines the implementation
of that program, including students’ need for and access to the resources and the processes they navigated
to obtain help. We identify multiple forms of administrative burden present, and using both survey data
and focus groups, explain how they affected students and institutions. The psychological costs of
administrative burden were particularly substantial and should be addressed in future programming.

Emergency Aid Distribution in West Texas Community Colleges (2023)

related to food and housing instability, inadequate health care and rising attendance costs. In fact, the U.S.
Department of Education recently released the first-ever national estimates of food insecurity and homelessness
and found that college students are more likely to face these issues than the general population. These financerelated
challenges have a significant and negative impact on a student’s academic performance and are
associated with stopping-out and dropping-out of college – even when the financial challenge amounts to a
relatively small dollar amount. Emergency Aid (EA) programs are one approach that colleges use to assist students
with these unforeseen challenges.
The threats students have faced in meeting their basic needs while in college have increased significantly in the
last 15 years, with less than half of all public community colleges today meeting criteria for being affordable. The
COVID-19 pandemic compounded these affordability issues as the entire sector forced students out of housing,
shut down dining halls, and shifted their instructional model overnight.
In response, Believe in Students and the then-startup mobile app called Edquity – now Beam – partnered with three
colleges in West Texas to provide emergency aid funding over two school years, from fall of 2020 through spring
of 2022, to help keep students in school as well as in their homes with their families intact. In total, $835,750
was disseminated throughout this period to help 1,937 students address food, housing, transportation, and other
expenses.
One of the partner institutions involved, Odessa College, was able to provide persistence and graduation data for
the students who received emergency aid through this program. This data shows that over 90% of all aid recipients
persisted in college the following semester or earned their degree. This outcome is particularly noteworthy
considering the national persistence rate for public two-year institutions (students continuing their education at
the same or a different institution) was 61.5% in fall 2020, while the retention rate (students returning to the same
institution) was 52.4%.
This report describes the unique partnership between community colleges in West Texas, Believe in Students,
and Beam, sharing information about how the dollars were used by students and how the partners adapted as a
result of the project. The findings and recommendations shared here are scalable to other campuses in Texas, and
the student outcomes will serve the region’s workforce and help meet the state’s 60X30 attainment goals and
workforce needs.

Connecting Students to Basic Needs Hubs During the COVID-19 Pandemic: An Evaluation of a Cross-Sectoral Partnership (2023)

Reducing basic needs insecurity among community college students is an equity imperative for improving college attainment, particularly given the challenges the pandemic introduced. One popular approach is co-locating campus support services to help students access support (beyond financial aid), including public benefits and emergency aid. Some institutions operate their own basic needs hubs, while others engage outside providers. This study evaluates a campus-based cross-sectoral approach at two community colleges in King County, Washington. Together, the United Way of King County and area colleges and universities operate “Benefits Hubs” for students, offering support from peer navigators and helping them access financial resources and information. However, many students experiencing basic needs insecurity do not use hubs—a problem shared by many other student support services. Is it possible to increase students’ use of hubs through low-cost outreach? Does that outreach also improve students’ academic outcomes? Evaluators examined these questions during the COVID-19 pandemic, a time when students’ needs for support were especially high, and staff were particularly constrained. The colleges collectively identified a group of approximately 3,000 low-income students who might be eligible for public benefits and thus find the Benefits Hubs’ support useful. Those students were divided at random into two groups. Beginning in fall 2020, staff sent the first group emails encouraging them to use hubs. The second group did not receive that outreach but still had access to hubs. A comparison of the two groups following that outreach revealed whether sending those emails—a strategy widely known as “nudging”—those students improved their use of hubs and/or their odds of academic success in terms of grades and retention. The results are mixed and largely inconclusive. On the one hand, outreach modestly increased students’ use of Benefits Hubs. It also reached students in several target demographic groups—older students and those from marginalized communities who are at heightened risk of basic needs insecurity. This suggests that informational barriers contribute to basic needs insecurity and may be partially overcome with inexpensive outreach strategies. However, the benefits of outreach dissipated over time, potentially because the targeted students shared information with their peers who did not receive the emails, and then those students also used hubs. Even with the additional outreach efforts, most targeted students did not use hubs and academic improvements were not evidenced. As community colleges continue to recover from the pandemic and support students to graduation, providing basic needs supports to help students afford college may help. There are several reasons why this evaluation might understate the benefits, including analytic limitations and how the pandemic affected the program. Recent legislation and philanthropy are funding many basic needs hubs, making it especially important to engage in ongoing assessment to develop strategies for strengthening their use and ensuring maximum equitable impact.

Do expenditures other than instructional expenditures affect graduation and persistence rates in American higher education? (2010)

During the last two decades, median instructional spending per full-time equivalent (FTE) student at American 4-year colleges and universities has grown at a slower rate than median spending per FTE student in a number of other expenditure categories, including academic support, student services and research. Our paper uses institutional level panel data and a variety of econometric approaches, including unconditional quantile regression methods, to analyze whether these non-instructional expenditure categories influence graduation and first-year persistence rates of undergraduate students.

Our most important finding is that student service expenditures influence graduation and persistence rates and their marginal effects are higher for students at institutions with lower entrance test scores and higher Pell Grant expenditures per student. Put another way, their effects are largest at institutions that have lower current graduation and first-year persistence rates. Simulations suggest that reallocating some funding from instruction to student services may enhance persistence and graduation rates at those institutions whose rates are currently below the medians in the sample.

Connecting College Students to Alternative Sources of Support: The Single Stop Community College Initiative and Postsecondary Outcomes (2020)

Single Stop U.S.A.’s Community College Initiative was designed to improve the well-being of low-income communities by connecting individuals to public benefits and other institutional and community resources to address nonacademic barriers to college completion. Through offices located on community college campuses, Single Stop provides students with a range of free services, including screenings and applications for public benefit programs; tax services, financial counseling, and legal services; and case management with referrals to a wide variety of resources and support programs across the institution and community. This report presents an evaluation of the Single Stop program and its impact on students’ postsecondary outcomes. The authors examined the Single Stop program at four community college systems: Bunker Hill Community College, City University of New York, Delgado Community College, and Miami Dade College. The analysis indicates that use of Single Stop was associated with improved postsecondary outcomes. The findings suggest that access to alternative financial resources from government benefit programs alongside a network of institutional and community support programs can offer valuable support to college students.