Higher Learning Commission

Assessment Resource(fulness): Developing a Rich Culture of Assessment with Limited Resources

Jeffrey Kratz and Karin Wright

The scarcity of resources available to colleges and universities has changed the landscape of higher education in the United States. Each year, the trend in higher education seems to be budget cuts resulting in program discontinuations, increased reliance on adjunct faculty members, streamlining of course offerings, branch campus closures and the elimination of what were once considered fundamental student services. Every institutional type, public and private, has experienced a diminished capacity to deliver quality educational experiences for students. Even in the face of these ever-shrinking resources, external and internal stakeholders rightly demand quality educational offerings.

Assessment of student learning is not immune to the effects of diminished resource capacity. Educators must maintain robust assessment programs that produce meaningful data to inform curricular change despite the reduced capability to do so. The need for efficient assessment practices cannot replace the obligation to engage in effective assessment. In short, throughout the country, faculty members, staff members and administrators are challenged to “do more with less” in the area of assessment. Thus, the central challenge for many institutions in this current climate is this: How do we develop and/or maintain a rich assessment program when we are resource poor? While difficult for all institutional types, this dilemma is especially problematic for small, private colleges or universities who have fewer faculty members and staff members and receive little to no governmental monetary support. This paper is a case study of one institution’s experience with the need to develop a robust culture of assessment of student learning with limited resources.

Lincoln College is an independent, not-for-profit, liberal arts college with an enrollment of approximately 1,200 students across two campuses and two extension site locations in central Illinois. The college currently offers 11 baccalaureate degrees, associate degrees, dual-credit courses to two area high schools, and certificate programs on the branch campus (though budgetary constraints forced the college to announce the closure of the cosmetology and massage therapy certificate programs, effective fall 2016). The college offers traditional, face-to-face semester-long classes to traditional learners and accelerated, hybrid course formats (five-week and eight-week classes) for returning, adult learners.  Lincoln College currently employs 30 full-time faculty members across four academic divisions and hires an average of more than 70 adjunct faculty members in an academic year to teach in the main campus traditional programs and the branch campus accelerated course programs. The largest academic department (the math department and English department) comprises five full-time faculty members with several academic departments consisting of only one or two full-time faculty members. The small size of these academic departments presents a significant challenge for gathering meaningful and relevant assessment data.

In 2010, Lincoln College joined the Higher Learning Commission’s (HLC) Assessment Academy with the overarching mission of establishing a unified culture of assessment of student learning at the institution regardless of campus, educational delivery format or student service. The college had received a monitoring report on assessment resulting from its comprehensive, reaffirmation of accreditation visit in 2003. In 2007, the HLC accepted the monitoring report, though the college recognized the need to expand its assessment program. The assessment program developed in response to the reaffirmation of accreditation visit was cumbersome, inefficient, and decentralized. Indeed, Lincoln College recognized in its application to the Assessment Academy that “assessment often seems to be done in silos, and results are not widely shared.” In addition, continued administrative turnover at the college resulted in no consistent commitment to an assessment plan that faculty members could support or understand because the administrative turnover at the highest levels of the college resulted in changing levels of resource commitment (budget, time, personnel) to the assessment program. The effect was that faculty members came to view assessment as “additional work” and an “external imposition” to their role.

Initially, Lincoln College’s original Academy Project consisted of 16 specific objectives ranging from student services assessment to faculty accountability to general education assessment in an effort to develop a comprehensive, institution-wide program. The college’s administration committed personnel and budgetary resources to implementing the project by hiring a director of Institutional Research, Assessment, and Planning. However, even with the creation of the new position, the college did not possess the human or financial resources needed to implement every assessment strategy at every level, especially when faculty members viewed assessment as external to their work of educating students. After just two years and faced with budget deficits, the college was forced to eliminate the position of director of Institutional Research, Assessment, and Planning. For faculty members, the apparent lack of resources for assessment, as evidenced by the elimination of the position, undermined the overall project. The ambitious Academy Project was thus met with confusion, frustration and very little faculty participation.  

Recognizing that faculty members are the most important personnel resource in spreading a culture of assessment to all other areas of the institution, the vice president for Academic Affairs restructured the Academy team and contacted the college’s Academy mentor, who advised, “the smaller the institution, the simpler the plan.” Thus, the reconstituted Academy team quickly recognized that Lincoln College had “bitten off more than it could chew,” given its limited human and financial resources and a small institution’s capacity to implement such lofty goals in only four years. Although the fundamental goal of establishing and maintaining a culture of assessment remained unchanged, the Academy team shifted its focus to accomplishing that goal while remaining cognizant of the institution’s resource limitations.

The small size of the institution, and the feedback from the Assessment Academy mentors that the initial project was too ambitious, resulted in the restructured Academy team examining ways to accomplish meaningful assessment with limited faculty, administrative, and financial resources. The guiding principle regarding revisions to the project included developing a more simplistic assessment infrastructure that was aligned with the actual resource capacity of the college.

Resource Allocation for Assessment at a Small College

The most important resource recognized by the Academy team to reverse the negative culture regarding assessment was knowledge of the institution’s organizational saga. Clark (1975) defined “organizational saga” as the collective understanding of the uniqueness of an institution rooted in the organization’s historical development. As an element of organizational culture, organizational sagas establish the loyalty individuals feel toward an institution as result of recognizing this distinctiveness. Clark (1975) and subsequent authors (Bolman and Deal 2013) have indicated that understanding the organizational saga is one of the most valuable resources for making an organization effective. Lincoln College’s revamped Academy team recognized that the college had a strong organizational saga, which had been identified by faculty members even while the subculture regarding assessment of student learning was weak, largely due to the ambiguity of resource commitment associated with the consistent administrative turnover.

The new Academy team chose a mantra of “adapt, not adopt, best practices in assessment” to develop a plan that would produce meaningful, relevant and compelling data for the faculty without requiring excessive additional time, administrative personnel increases or expanded financial resources. The simplification of the Academy Project to a focus on just assessment of general education was the starting point for allowing the organizational saga to dictate other assessment resource allocation. Because every aspect of the institution is tied to the general education and institutional learning outcomes, simplifying the learning outcomes and creating an assessment process that did not overburden the small departments allowed the college to gain faculty acceptance of assessment, which can later be expanded to the nonacademic divisions of the college.

To balance the need for effectiveness and efficiency, the college instituted some of the following changes to the Academy Project to align resource capabilities with the need for gathering usable and effective data:

  • The reconstituted Academy team comprised faculty members and administrators with long tenures at the college who utilized their collective knowledge of the institution’s resource capabilities and their knowledge of the organizational saga to develop an assessment program that is both effective and efficient. Thus, organizational knowledge and expertise became the most important resource in determining the restructuring of the assessment project to optimize other available resources while simultaneously reversing negative faculty attitudes.  
  • The Academy team adapted the assessment triangle model established by Huba and Freed (2000) in Learner-centered Assessment on College Campuses: Shifting the Focus from Teaching to Learning. Aligning the specific program learning outcomes to the general education learning outcomes allows the college to engage in general education assessment either directly through embedded, course assessments of general education courses or through the assessment plans in place in specific academic programs. In essence, assessment of the programs also results in assessment of general education.
  • At a regularly scheduled faculty workshop in 2013, each division developed assessment rubrics from the AAC&U Value Rubrics, which were revised to align specifically to one of the five General Education Learning Outcomes. Thus, faculty members themselves created one direct measure of assessment without needing an additional assessment workshop day or without “training” in the use or meaning of the rubrics.
  • The GELO rubrics are embedded in course-level assignments, allowing a faculty member to gather compelling assessment data without overly taxing his or her workload. Faculty members have reported that the completion of the embedded assessment rubric, which they developed in a workshop, takes less than one minute per artifact. This process significantly increased the college’s capacity to gather quality data regarding the achievement of student learning outcomes, which faculty members are willing to use and interpret to make curricular change, especially as they developed and continually evaluate the effectiveness of the rubric. A project once considered by the faculty as cumbersome, time consuming, and foreboding is now viewed as an achievable and relevant goal.

Since 2012 when the Academy Project was revised and presented to the faculty, Lincoln College’s assessment of student learning has flourished despite continued resource pressures. Faculty members have now embraced assessment of student learning, actively engage in assessment workshops, and widely participate in assessment processes. The college’s faculty has made substantial and rapid improvements in assessment, including the revision of general education learning outcomes, the establishment of master syllabi with common course learning outcomes aligned to programmatic learning outcomes, and the use of embedded general education learning outcome rubrics developed by faculty members to gather assessment data. To continue to expand the budding culture of assessment, a Sustainability Plan has been enacted that assists in allocating the limited resources in a manner that exposes the institution’s faculty members to effective practices in assessment adaptable to the institution.

The Assessment Sustainability Plan has resulted in the re-assignment of an academic dean to the position of dean of Assessment and Academic Planning to continue to work with faculty members on expanding assessment at the institution. Lincoln College is now entering its third year of gathering embedded assessment data on general education learning outcomes (GELOs) with approximately 80-percent participation in the reporting of general education assessment data by full-time faculty members. Faculty members are leading workshops on assessment rather than relying on the assessment leadership of the college (Academy team). Ownership of the assessment process is increasingly shifting to where it belongs, with the faculty. In addition, this change in the culture regarding assessment was accomplished with no significant increase in budgets for assessment. Rather, an understanding of the organizational saga allowed the college to design a program that faculty members could support and, more importantly, take ownership of in order to improve student learning.

References

Bolman, L. G., and T. E. Deal. 2013. Reframing organizations: Artistry, choice, and leadership. San Francisco: Jossey-Bass.

Clark, B. R. 1975. The organizational saga in higher education. Administrative Science Quarterly 17: 178–184.

Huba, M. E., and J. E. Freed. 2000. Learner-centered assessment on college campuses: Shifting the focus from teaching to learning. Needham Heights, MA: Allyn & Bacon.

 

About the Authors

Jeffrey Kratz is Dean of Assessment and Academic Planning, and Karin Wright is Division Chair of Language, Humanities and Social Sciences at Lincoln College in Lincoln, Illinois.

Copyright © 2017 - Higher Learning Commission

NOTE: The papers included in this collection offer the viewpoints of their authors. HLC recommends them for study and for the advice they contain, but they do not represent official HLC directions, rules or policies.


Higher Learning Commission • 230 South LaSalle Street, Suite 7-500 • Chicago, IL 60604 • info@hlcommission.org • 800.621.7440

Home | About HLC | Contact Us | Privacy Policy

YouTube graybkgrdLinkedIn graybkgdTwitter graybkgd