Higher Learning Commission

The Use of Diagnostic Metrics to Focus on Solutions

Janet Houser and Max Sotak

Introduction

The project described in this paper focused on the use of diagnostic analyses to improve the student experience through evidence-based advising and attention to factors affecting student persistence. Nontraditional statistical analyses were used to generate evidence about when and why students were lost to nonacademic attrition. The diagnostics showed how student persistence could be improved through curricular sequencing, developmental advising and course structure. These changes were implemented, and indications are that interventions have yielded results.

History

The College for Professional Studies (CPS) at Regis University in Denver, Colorado, has served adult learners for more than 30 years, offering accelerated undergraduate and master’s degree programs. As a pioneer adult program in the United States offering programs in business, computing, education, liberal arts, and social sciences, the college has been successful and respected in the world of adult higher education. In 2002, at the height of its enrollments, the college served more than 15,000 students in the classroom and online. After that year, the college experienced a gradual decline in credit hour generation. Multiple causes included the emergence of online-only competitors, which occurred after Congress eliminated the 50-50 rule; the rising cost of higher education and the decreased commitment of employers to providing employee tuition benefits; and the downturn in the economy after 2008, which dampened consumer confidence about the availability of post-college jobs. In the face of these external challenges, the college also experienced internal challenge. An early entrant in the online market, the college experienced rapid growth in the 1990s, and its systems were focused on scalability and standardization. While this strategy did not change, the market had—current students wanted customization, distinctiveness, and a personal touch. These forces conspired to create a downward trend in credit hour generation that threatened the health of the college’s programs.

The College for Professional Studies has always welcomed students who transfer at least 30 credit hours. A flexible transfer policy and programs designed for degree completion are significant differentiators for the college. Many students coming from traditional programs are unfamiliar with accelerated programs offering courses lasting five or eight weeks and often underestimate the demands. Therefore, approximately one in five students are lost during their first course, which is consistent with research showing that nontraditional students are at greatest risk during their first year (National Center for Education Statistics 2014). Thus, persistence challenges for the College for Professional Studies have always been as much a concern as the difficulty of recruiting new students in an increasingly competitive market. Many initiatives were put in place to improve the decline in enrollments, but little effect was seen. It became apparent that the initiatives were based on anecdotal information or “urban myths.” The problem was determining the root causes of the current situation using metrics so that effective interventions could be developed. More powerful tools were needed. A comprehensive statistical analysis of the current state was conducted to determine evidence-based interventions.

Description and Outline

Using evidence as a basis for devising strategies enables the academic administrator to target interventions based on what is known rather than on anecdotal opinion. Yet we often create action plans without adequate information about the root causes of problems we are trying to address. In this case, we used student survey data, survival analysis, logistic regression, and inferential analysis to diagnose the root causes of declining credit hour generation in an adult-serving college. Subsequent plans focused on initiatives to improve the curriculum and to enhance the student experience through developmental advising (Astin 1975; Tinto 2006).

Survival analysis—which is typically a tool used in health care—was applied to determine the time periods in which risk of retention was raised (Radcliff, Huesman and Kellogg 2006; Murtaugh, Burns and Schuster 1999). These results were used to change the sequencing of the curricula, assign faculty members to courses, and enhance developmental advising to avoid student loss. A student survey yielded data about the reasons that students dropped a course. Additional data gathered from retention counselors illuminated some of the reasons students drop out altogether. Inferential analysis was used to identify the pattern of transfer credit and whether it was related to student success. Logistic regression was used to determine which student characteristics best predicted student success, and chi square identified the critical time periods to enhance graduation rates. Finally, specific diagnostic evaluation was used to determine whether intensive courses were effective and whether students desired this type of course delivery. These analytics formed the pieces of a puzzle, which—when complete—revealed an evidence base for subsequent curricular and student experience initiatives.

Initiatives

Initiatives were incorporated into a plan devised collaboratively with academic leadership and faculty members. Following are the initiatives:

Personal attention: Focus both faculty members and staff members on the “personal touch” and customizing the learning experience for the adult student.

Curricular sequencing: Sequence courses so that first three courses (graduate) and five courses (undergraduate) are “gripping,” in the major, and taught by the best teachers.

Advising: Decentralize advising and shift to developmental advising, with undergraduate advising focused on curricular requirements and continuous enrollments and graduate advising on professional goals, barriers, workload and “inspiration.” Talk proactively with students about managing workload.

Reinstate intensives: Reinstate intensive courses for topics that meet preset criteria; and develop a three-year/30-credit-per-year track by providing a mix of five-week and eight-week courses.

Results

It has been one year since the initial interventions. New enrollment numbers have increased in three of the four programs in the initial study. Metrics were included in an institutional dashboard to assess stability and sustainability of the improvement measures.

References

Astin, A. W. 1975. Preventing students from dropping out. San Francisco: Jossey-Bass.

Murtaugh, P. A., L. D. Burns, and J. Schuster. 1999. Predicting the retention of university students. Research in Higher Education, 40 (3): 355–371.

National Center for Education Statistics. 2014. Table 326.30, Retention of first-time degree-seeking undergraduates at degree-granting postsecondary institutions, by attendance status, level and control of institution, and percentage of applications accepted: 2006 to 2012. Digest of Education Statistics. Washington, DC: U.S. Department of Education. https://nces.ed.gov/programs/digest/d13/tables/dt13_326.30.asp.

Radcliffe, P. M., R. Huesman, Jr., and J. P. Kellogg. 2006. Identifying students at risk: Utilizing survival analysis to study student athletes. Paper presented at the National Symposium on Student Retention, Bloomington, MN, October.

Tinto, V. 2006. Research and practice of student retention: What next? Journal of College Student Retention: Research, Theory and Practice, 8 (1): 1–19.

 

About the Authors

Janet Houser is Vice Provost and Academic Dean, College for Health Professions, and Max Sotak is Associate Dean, College for Contemporary Liberal Studies, at Regis University in Denver, Colorado.

Copyright © 2017 - Higher Learning Commission

NOTE: The papers included in this collection offer the viewpoints of their authors. HLC recommends them for study and for the advice they contain, but they do not represent official HLC directions, rules or policies.


Higher Learning Commission • 230 South LaSalle Street, Suite 7-500 • Chicago, IL 60604 • info@hlcommission.org • 800.621.7440

Home | About HLC | Contact Us | Privacy Policy

YouTube graybkgrdLinkedIn graybkgdTwitter graybkgd