An examination of instructional effectiveness in higher education using multiple outcome measures.

Abstract

This study develops a technique of evaluating teaching effectiveness in higher education using two outcomes measures. The first measure, an instructor's score on the student evaluation of the teacher (SET), is considered to be the traditional measure of teaching effectiveness. The second measure, a comprehensive exam, estimates the cognitive performance of each teacher's students. These measures are merged to create a composite index, and this index is used to measure teaching effectiveness.

It is maintained that accurate performance measurement facilitates labor allocation and motivation schemes. Traditional faculty performance evaluation may elicit unfavorable economic consequences due to perverse incentive effects on teaching behavior such as moral hazard and grade inflation. A broader measure of teaching effectiveness may lessen some of the perverse incentives of the traditional system.

The first step to create the composite index is the estimation of educational production functions for the SET and the outcomes exam. The estimation uses both panel data and Seemingly Unrelated Regressions (SUR) models. Second, a predicted score is estimated for all instructors using the sample average values of the right-hand side variables. Third, the predicted score is compared with the actual score to identify exceptional performers. Finally, the measures are weighted to develop a composite score for each instructor.

The data for this study were collected from a survey administered to students, a comprehensive exam, and administrative records. The data were collected at a large, comprehensive university in the Fall semester of 1996. The population of approximately 700 students came from 24 classes of introductory economics taught by twelve instructors.

Under the panel data technique, it was found that student evaluations of the teacher are influenced by a student's expected grade, choice of section, native language of the instructor, academic major, and student aptitude. Cognitive learning is influenced by student aptitude, grade point average, age, and male gender. Using the SUR estimation method, several variables were added to each learning equation. For the SET estimation, the large class variable had a negative coefficient, whereas the instructor's years of service has a positive coefficient. For the exam equation, high school economics was significant with a negative coefficient and the instructor's terminal degree entered the equation with a positive coefficient, but the male and age variables were insignificant.

The composite index re-orders the measurement of faculty performance. It was observed that some outstanding performers were overlooked by the traditional measurement technique. Some exceptional performers by the traditional technique were deemed to be average performers by the adjusted measure. The study also found evidence of grade inflation.