In recent years, the methodology used to create college rankings has shifted significantly, moving away from evaluating institutional conditions and student status upon entry (inputs) to focusing on the status of students after graduation (outcomes). This trend reflects a growing emphasis on the tangible results of a college education rather than the resources and prestige of the institution itself.
Emphasis on Outcomes
U.S. News, a leading authority in college rankings, has responded to criticism by highlighting the outcomes for graduates in its latest rankings. The organization stated, “We want to ensure the educational resources we provide emphasize the outcomes for graduates,” signaling a broader adoption of outcomes-focused measures. This sentiment was echoed in a recent Wall Street Journal headline about its own formula: “The WSJ/College Pulse College Rankings: Measuring Outcomes, Not Inputs.”
Forbes aligns with this approach, concentrating on financial outcomes such as alumni salaries, debt, and return on investment. Money magazine follows suit while also considering inputs. Washington Monthly takes a broader perspective, factoring in financial outcomes, research contributions, and community and national service (e.g., Peace Corps participation and voter engagement).
The Decline of Inputs
While inputs such as SAT scores, faculty salaries, spending per student, and institutional prestige are losing prominence, they are not entirely absent. They still hold some weight in Money’s formula and remain significant in U.S. News’s rankings despite their effort to emphasize outcomes. However, the prevailing logic now favors evaluating the final product (graduate success) over the ingredients (institutional resources).
Missing Elements in Rankings
Despite the shift towards outcomes, two crucial aspects are often overlooked in college rankings:
- Value of Inputs: Not all inputs are inherently bad. While traditional measures like SAT scores and faculty salaries may be problematic, other inputs such as curricular structure and class size provide valuable insights into the quality of education.
- Learning Process: The most critical component of the college experience—the learning process itself—is largely ignored. The goal of college is to enhance content knowledge and critical-thinking skills over four years, yet this fundamental purpose is seldom reflected in rankings.
Challenges in Measuring Learning
The primary reason for this oversight is the difficulty in obtaining relevant data to measure the learning experience. Effective assessment would require evaluating both the curriculum and the quality of instruction:
- Curricular Structure: Inputs like curricular structure should be part of any reputable ranking system. General education, which is a common requirement across institutions, could be assessed based on the breadth and depth of courses offered. The American Council of Trustees and Alumni (ACTA) provides a model with its “What Will They Learn?” ratings, which grade schools on their general education requirements.
- Quality of Instruction: Traditional measures of teaching quality, such as faculty salaries and student-faculty ratios, are insufficient as they reflect institutional wealth rather than classroom dynamics. Instead, metrics like class size could be more relevant, focusing on the percentage of small classes in a student’s learning experience.
Potential Outcome Measures
Some proposed outcome measures for assessing learning include:
- Exit Testing: Instruments like the GRE and the CLA (College Learning Assessment) measure general cognitive skills but not specific content knowledge. Mandatory testing poses challenges, including student and institutional resistance and the need for entrance testing to calculate value-added improvement.
- Student Surveys: The Wall Street Journal uses student surveys to gauge the overall learning experience, but these are voluntary and may not represent the entire student body. End-of-course evaluations are also problematic due to biases and the tendency of students to rate lenient professors more favorably.
Feasibility Issues
A comprehensive assessment of the learning experience would ideally involve expert observers evaluating classroom activities using standardized tools. However, the logistical and financial constraints of deploying experts to multiple institutions make this approach impractical on a large scale.
Conclusion
The shift towards outcomes in college rankings marks progress by reducing the emphasis on wealth and status-related inputs. However, the industry’s major players still rely on these inputs, ensuring that familiar names dominate the top rankings. Valuable inputs like class size and curricular assessments are often excluded, and the learning process remains inadequately measured. While college rankings continue to sell magazines by offering an array of statistics, they fall short in guiding students to the institutions that provide the best education.
Ultimately, despite the changes in ranking methodologies, the quest to identify the best colleges remains fraught with challenges and limitations, leaving prospective students without a clear answer on where to find the best educational experience.
