Who will guard the guards themselves?

(Yes, Quis custodiet ipsos custodes? H/T Juvenal)

The excellent question raised by the Hechinger Report today perhaps would better go "who will hold accountable the accountability officers?" in light of Michelle Rhee's firing of 241 Washington, D.C. teachers. Their terminations were under D.C.'s new evaluation system, which, for teachers of grades 4-8, is 50% based on their "value added" or how much their students' test scores go up over the year.

Professor Aaron Pallas has an excellent summary of how this trend--and it is a trend, make no mistake about it--misunderstands at its base what tests evaluate and how they can legitimately be used:

Among the key critiques: the tests were neither designed nor validated for use in evaluating teachers; a teacher’s location in the value-added distribution can bounce around from one year to the next, or from subject to subject; students may be assigned to teachers’ classrooms in ways that invalidate the assumptions of the value-added models; a teacher’s value-added score may be sensitive to which student and school factors are included or omitted in the model; and the value-added models produce estimates that are riddled with error or uncertainty, yet are often treated as though they’re precise measurements.

What is troubling to me is that, to date, districts using these complex value-added systems to evaluate teacher performance haven’t made the methodologies known to the general public. New York City’s Department of Education has produced Teacher Data Reports for several years running, but technical reports on the methodologies used haven’t been released to the public. Not so serious when these tools are being used for internal diagnostic purposes, perhaps, but there is an important set of policy goals that are compromised when these methodologies are not fully disclosed.

We continue to have this conversation as if the tests are, in and of themselves, a solid measurement, which they aren't, and then extend the error by using them to judge something they were never designed to judge. To compound this further by refusing to make the process and reports public? A disaster.

Further notes: 165 teachers were terminated due to performance; the remainder were for licensure problems and the like. For what the rest of the IMPACT evaluation is like and the problems therein, see here and here.