6 Behavioral case study #1: Encouraging community college students to renew financial aid early

Completing and renewing the Free Application for Federal Student Aid (FAFSA) can support college enrollment persistence, yet according to national statistics, students who do not file the FAFSA leave about $2.6 billion in federal grant money on the table each year. One persistent challenge is a drop-off in FAFSA renewal rates among continuing students after their freshman year. Approximately 40% of students who received financial aid in their first year at one of New York City’s community colleges (City University of New York) do not complete a FAFSA for their second year of attendance, placing them at risk to lose the financial resources to facilitate degree completion (Nissan et al. (2020)).

Behavioral science has informed multiple cost-effective strategies for increasing FAFSA submissions. A simple text-based intervention that encouraged FAFSA submission increased sophomore year retention by 14% (Castleman & Page (2014)). Additionally, ideas42 research at Arizona State University showed that behaviorally informed student reminder emails, which include devices to trigger loss aversion, plan making, and commitment, increased priority deadline FAFSA renewal by over 10 percentage points, from 29% to 39% (Fishbane & Fletcher (2016)). When combined with reminders emailed to parents, FAFSA submissions increased from 29% to 50%.

Research and behavioral diagnoses

Drawing on this previous research, the Student Persistence Initiative (SPI) — a five year collaboration between ideas42 and the City University of New York (CUNY) — designed and tested multiple interventions aimed at increasing FAFSA renewal rates by the June 30th priority deadline. Students who file before this priority deadline will have secured financial aid for the upcoming semester, rather than waiting until the year is over and going out-of-pocket for expenses during their enrollment period.

SPI’s FAFSA renewal interventions addressed the following behavioral hypotheses:

• Students face hassles during the filing process (coordinating documents with parents, remembering FAFSA PIN numbers, passwords, etc.)
• Students have difficulty developing a plan to complete the FAFSA given complexities and unknowns associated with the process (needing parental tax information, lack of clarity on the time required to file the form, etc.)
• Students misperceive consequences of not filing the FAFSA in a timely manner
• Students’ fear of making errors on the FAFSA form leads to inaction
• Students are unaware of the full host of resources available at CUNY to support filing the FAFSA, for example computer labs with staff support

Intervention design

ideas42 and CUNY ran four FAFSA renewal campaigns between 2016 and 2019. This report focuses on the 2017 intervention, which used a combination of behavioral emails and texts to encourage continuing students at three CUNY community colleges (Borough of Manhattan Community College, Bronx Community College, and Hostos Community College) to renew FAFSA. The 22,192 students who had not yet renewed their FAFSA for the following academic year by February of their Spring semester were randomly assigned to a treatment or control group. The treatment group received business-as-usual emails as well as supplementary behavioral emails and text messages addressing the hypotheses above (see figures 6.1 and 6.2 for example emails and texts). The control group only received business-as-usual emails from the college.

Figure 6.1: Text messages sent to BMCC students.

Figure 6.2: One of nine emails sent to BMCC students.

Results: Across all schools, the renewal rate was over 6.4 percentage-points greater in the treatment than in the control group (p<0.001), which represents a 17% increase in early renewal. These results varied significantly by school, ranging from an effect size of 4.8 percentage points to 7.7 percentage points.

Figure 6.3: Average treatment effects by school in the 2017 FAFSA experiment with 95% confidence intervals.

In this analysis, we pre-specified several subgroups in order to estimate how treatment effects vary across subgroups. Based on our a priori knowledge of group-based performance, these groups included gender, age, credits in progress, dependent status, and grade-point average (GPA). For each of the pre-specified covariates, we ran a linear regression of filing by the priority deadline on treatment, the pre-specified covariate, and the interaction between treatment and covariate. The regressions find that students who attempt more credits have a higher treatment effect, and suggest that the effect is larger for older students.

Figure 6.4: Regression coefficients of treatment–covariate interaction terms from separate linear regressions of filing by the priority deadline on treatment, one of the pre-specified covariates, and the respective treatment–covariate interaction, with 95% confidence intervals. The covariates are student age at the start of the intervention (AGE), the cumulative credits attempted at the beginning of the invention term (CRU_CUMU_ATMPT_BF), the credits attempted in the semester of the intervention (CRD_SEM_ATMPT), whether the student previously filed independently of the student’s parents (FAFSA_DEPEND_STATUS_Independent), whether the student is female (GENDER_Women), and the student’s cumulative GPA at the beginning of the intervention semester (GPA_CUMU_BF).