Institutional Effectiveness in Higher Education
Institutional effectiveness is an information-based decision-making model wherein the data gathered through organizational learning activities is used for quality improvement. Specifically, it refers to the ongoing process through which an organization measures its performance against its stated mission and goals for the purposes of evaluation and improvement. The term was first used to describe activities related to accreditation in the 1980s and is now a crucial component of the accreditation process, as well as the fundamental factor in accountability and performance funding in higher education.
Keywords Accountability; Accreditation; Community College Survey of Student Engagement (CCSE); Institutional Effectiveness; Organizational Learning; Performance Funding; Quality Assurance; Quality Enhancement; Strategic Planning; Student Learning Outcomes
Institutional effectiveness refers to the ongoing process through which an organization measures its performance against its stated mission and goals for the purposes of evaluation and improvement. Since the Southern Association of Colleges and Schools (SACS) adopted the term in the 1980s, institutional effectiveness has moved to the forefront of the dialogue among government agencies, accrediting organizations, and higher education administrators. In this age of ever-increasing pressure for accountability, students, parents, government officials, accrediting agencies, industry leaders, taxpayers, and the mass media are demanding responsiveness from institutions of higher education (Welsh & Metcalf, 2003). These stakeholders are pressuring higher education decision-makers for improved documentation about performance and a system wherein public policy tools are coupled with reliable data from institutions to improve the alignment of the performance of higher education with the expectations of the public it serves (Welsh & Metcalf, 2003).
Institutional effectiveness most often includes the measurement of performance in areas such as student learning outcomes, academic program review, strategic planning, performance scorecards, and benchmarking and quality measurement. These areas are studied in myriad ways using numerous divergent instruments to collect pertinent data. Although there are variations in the terminology used, each of the six accrediting agencies in the United States requires that colleges and universities have a process through which institutional effectiveness can be evaluated, measured, and reported (Welsh & Metcalf, 2003).
Sullivan and Wilds (2001) describe institutional effectiveness as the process of studying performance and engaging in related activities within the context of a number of concepts and criteria. In their model, all constituents of the higher education community must participate in the process, with each group bearing specific responsibilities for program review and data collection. The accomplishment of the institutional mission; the reflection of its vision, philosophy, goals, and objectives; and an interpretation of the environment are at the crux of an evaluation of institutional effectiveness. In order to measure performance against these self-defined standards, a historical review of institutional accomplishments, weaknesses, and aspirations followed by the preparation, collection, and interpretation of data by institutional staff and faculty must be undertaken. Faculty must be charged with the development and evaluation of curriculum and with the evaluation of student performance in relation to that curriculum. Administrators are called upon to interpret and utilize relevant data and information in ways that promote increased effectiveness. Finally, the president of the institution bears the responsibility for defining and communicating institutional priorities and for working with the board to secure the resources required to meet those priorities (Sullivan & Wilds, 2001).
While the plan for the study of institutional effectiveness must be outlined and communicated by senior administrators, the process is rarely straightforward. Smith and Parker (2005) comment that while the relationship between organizational learning (the data-collection component of institutional effectiveness) and the focus upon learning and research that defines higher education may seem apparent, the process tends to be disruptive to campus patterns. "From bringing together campus constituents across institutional boundaries and accessing campus information data systems to obtain usable information, the process of using an organizational learning approach for evaluation has challenged many campuses" (Smith & Parker, 2005, p. 122).
The process of institutional review began in the nineteenth century with the inception of accreditation. Originally intended as a means through which some external control could be exerted over educational standards, by the 1930s and 1940s, there was a trend toward an added emphasis on improvement (Selden, 1960, as cited in Dodd, 2004). Since the introduction of the concept of institutional effectiveness in the 1980s, institutions have improved the work done on writing annual goals and objectives, evaluating their accomplishment of those goals, and describing their responsive improvements based on that data (Sullivan & Wilds, 2001, p. 2).
The Accrediting Agencies
There are six regional agencies in the United States that accredit higher education institutions in their respective areas.
• New England,
• Middle States,
• North Central, and
Middle States was the first agency to require institutions to engage in the self-study and peer-review process, while Southern States was the first to incorporate the concept of institutional effectiveness into its requirements. The other five accrediting agencies quickly followed suit. An institution must be accredited by one of these agencies in order to receive federal financial aid, making the accreditation process a central focus for an institution when it is time for its review (Selden, 1960, as cited in Dodd, 2004). Additionally, there are a number of discipline-specific accrediting agencies that review curriculum and programs and whose standards must be met within the framework of the larger, regional accrediting bodies. While their accreditation may not have a direct impact upon the federal funding on which institutions rely, they can impact students' eligibility for professional exams and licensure, among other things. Hence, at any given time, an institution may be engaging in the self-study and accreditation process for a variety of agencies.
Since these accrediting agencies serve as the primary intercessors between postsecondary institutions and policymakers, they standardize accountability through their requirements for accreditation. In order to meet accreditation criteria, institutions must collect, format, report, and then use for improvement data about their programs and services (Welsh & Metcalf, 2003). With accredited institutions, Dodd (2004) explains, "constituencies such as students, the public, and government representatives have at least some assurance of quality and value. [The institution] is accomplishing the goals it has set within the context of its mission" (p. 14). Furthermore, argue Head and Johnson (2011), accreditation “can protect an institution from unwarranted criticism and . . . provide the stimulus for the improvement of courses and programs.” Accreditation also “promotes internal unity and cohesiveness” (p. 37). Evidence of these accomplishments may impact enrollment and funding.
Internal Quality Assurance
Focus on Outcomes
Accrediting agencies have revised standards to reflect a focus on the achievement of outcomes rather than an adherence to standards. The primary suggestion for reform is related to the development and refinement of internal quality assurance measures that ensure institutional effectiveness (Dill, Massy, Williams & Cook, 1996, as cited in Dodd, 2004). Earlier approaches failed to compare outcomes with known approaches and did not examine how data were used in decision-making and strategic planning (Ewell, 1998, as cited in Dodd, 2004).
The self-assessment required for accreditation, coupled with the periodic peer review, results in a wealth of information that can be used for both accountability and for institutional and program improvement. Though the process does provide for some quality assurance, its focus is on the inputs and the processes and not the student (Miller & Malandra, 2006). Since each institution must meet the same requirements, there is some standardization among the data that are collected, but its dissemination is not widespread unless individual institutions choose to publish it. "Higher education institutions and systems are focused inward and have a tendency to be unclear in communicating goals and outcomes to the public" (Miller & Malandra, 2006, p. 4).
Competing for Students
While the accreditation process served as the impetus for institutional effectiveness and provides some insight into performance, it is no longer the sole reason that colleges and universities engage in the continuous process of self-study and improvement. Miller (2006), writing to inform the commission that published A Test of Leadership: Charting the Future of US Higher Education (2006), explains that one of the core strengths of American postsecondary education lies in the number and variety of choices offered to students, which creates competition among institutions for students. Along with the ongoing competition for limited funds, the competition for students has forced increased accountability upon institutions so that student consumers can make the most informed choices. Miller (2006) asserts that having data available is not enough and that "it is essential to create a transparent system, which allows comparisons of rankings of institutions" (p. 4). Absent rankings and ratings, there is no impetus for change. He continues, "Today the U.S. News & World Report ranking serves by default as an accountability system for colleges and universities. Consequences that can modify behavior are an essential element of a productive accountability structure" (Miller, 2006, p. 4).
(The entire section is 4614 words.)