Our comprehensive program evaluation plan includes comparing the performance of First Scholars to non-participants. We look at each scholar’s GPA, retention rate during the first four semesters of college, time spent in college before graduation, and student engagement. Comparing our scholars’ performance in these key areas against that of other first-generation college students generates measurable results about our program.
The preliminary results for the 2010–2011 academic year are encouraging, based on the relatively small sample size of the first two cohorts. We contracted with Career and Educational Consulting Services, LLC, to conduct an evaluation of the First Scholars program’s effectiveness and outcomes. CECS reported that First Scholars participants “appear to be competitive with similar first-generation students, demonstrate high levels of classroom engagement, have a clear sense of the resources available to them, and report higher levels of persistence intentions” than their first-generation peers.
First Scholars were matched with applicant non-scholars on the Academic Success Index of the Student Strengths Inventory™ to compare whether the First Scholars Program had a measurable impact on first-year grades. First Scholars’ grades at the end of the first year were not appreciably different from the grades of a closely matched group of non-scholars at each university.
However, the relation between SSI scale scores and academic outcomes were stronger for scholars compared to applicant non-scholars. At the University of Utah, scores on the Academic Engagement and Help-Seeking scales were the best predictors of academic performance. Interestingly, at the University of Kentucky. the primary predictors of students’ GPA were Campus Engagement and Social Comfort. Although the existing sample sizes are too small to draw inferences from this first year of analysis, the data is suggestive of differences between scholars and applicant non-scholars that may be due to the influence of the First Scholars program.
First Scholars’ participants reported high satisfaction with their program experience, particularly their program director and mentors. The next set of data points will be collected during the fall 2011 semester.
As the program matures, our data-driven approach supplies affiliates with the critical measures of success needed to expand their base of support for ongoing operations. In addition, data demonstrating the impact and success of our approach can attract donors and investors interested in expanding the network of affiliated universities.
