Friday, October 10, 2008

Testing the test

(Full disclosure: though I never took a class from this professor, he does teach at my alma mater.)

Guest Column: MCAS doesn't measure up
Standards-based reform fails to address problems
By AL RUDNITSKY


Thursday, October 9, 2008

The Massachusetts Comprehensive Assessment System (MCAS) is our commonwealth's version of the federal mandate known as No Child Left Behind (NCLB). And NCLB is the national version of standards-based reform - which is supposed to be a solution to a problem.

For the most part, the public has taken standards-based reform as the only solution to a problem. The problem being, of course, the deficient quality of our education system and the inordinate number of poorly educated people who emerge, or sometimes never emerge, from that system.

There is a great deal to be said about the complex problems of our education system. These problems certainly have been around for a long time and periodically they become more noticeable, such as when the Soviets launched the Sputnik satellite, or our nation is judged to be at risk because of poor literacy, or when good jobs are outsourced to up-and-coming countries. Whatever particular issues people may focus on, most would agree that our education system has problems.

To solve our educational problem, standards-based reform offers the following two steps: First, set high standards. Second, measure progress toward those standards and hold schools and teachers accountable for achieving them.

The original impetus for standards-based reform came from the business community. (Ross Perot was among the more vocal proponents.). After all, businesses set goals (let's sell a million widgets at a 12 percent profit) and then measure their progress toward those goals (how much money have we made?). Anyone can see how well this has worked for business - until lately?

Truth is, it's impossible to argue against high standards without sounding like a fool or a subversive. Imagine a politician who is seeking election coming out against high standards for schools. As for needing to measure progress toward standards, it seems a self-evident and commonsense notion.

The problem with MCAS - and standards-based reform - is in the details. Standards-based reform sits on the foundation of testing. Yet we never question whether our testing tools are up to the job we need them to do. Instead we put total trust in our ability to measure important educational outcomes. Our trust is misplaced. Testing's state of the art is on measuring narrowly focused skills and specific facts and information. Because we are adept at measuring these things and because measuring progress and holding teachers and schools accountable are core aspects of standards-based reform, we have allowed our standards to be set so that they embody what is easily measured. In doing so, we let the testing tail wag the whole educational dog. Meaningless phrases are often used to obscure the reality of our overly narrow focus. In its Oct. 2 editorial, the Gazette refers to "MCAS as a way of tracking and comparing student comprehension of basic skills." What is comprehension of basic skills? Is that the same as plain old basic skills? Does adding the term "comprehension" impart some special quality to basic skills? Is the term comprehension meant to imply some deeper, more significant kind of learning? A look at the curriculum frameworks and MCAS test items suggests otherwise.

A widely held conception is that students need the basics in order to get to meaningfulness, deep knowledge, and understanding. This simply is not true. The entire thrust of the science of learning informs us that skills, whether those of reading and writing or the ability to perform mathematical calculations, are best learned in contexts where they are used in meaningful ways by student trying to better understand their world. These skills are tools and isolating them from their authentic application will not prepare students for participation in the knowledge age in which we find ourselves.

The Gazette, on Sept. 17, ran an article in which Mitchell Chester, Massachusetts education commissioner, addressed falling scores in reading comprehension. Chester said, "I am concerned that the emphasis in early grade reading may have swung too far toward the mechanics of reading ¿ without enough attention being paid to understanding what you're reading." Exactly. Our tests of skill are imperfect; our tests of subject matter, like history and science, are a disaster. Here testing's state of the art focuses exclusively on specific information and yet we have unquestioningly adopted these tests as the bellwether of our education system. A student may have a growing understanding about aspects of science and how scientific inquiry proceeds; our tests, however, are "either-you-know-it-or-you-
don't." Students do not get a chance to reason with the information they do know. Facts and information are important when they are connected to ideas. Facts and information are best learned and remembered when they are part and parcel of meaning making in schools.

Make no mistake, the emphasis on accountability means teachers will teach to the test. They almost have to. Caught in a system which keeps raising the testing bar - NCLB requires that schools show continual improvement - teachers never get to the point where they can turn their attention to other, more important content. Schools that do not measure up receive funds that must be spent to improve scores and thus are spent on consultant services like those offered by Kaplan.

Kaplan, as a case in point, has gone from a small SAT prep company to an NCLB giant with revenues of $2 billion, accounting for more than half the income of its parent, The Washington Post Company. Much of Kaplan's program doesn't even address content; instead it is aimed at how to outsmart the tests. NCLB is not leaving test makers and test preparers behind; these companies are taking in big taxpayer dollars. By not insisting that our schools integrate academic skills and factual information with meaningfulness, right from the outset, we insure that meaningfulness won't appear at all.

Having said this, it is only fair to note that there are many teachers who manage to achieve much more. They do so despite the tests, not because of them. It is also fair to note that there are teachers who do not belong in classrooms. Giving these inept teachers a script for teaching to the tests is not a solution to the serious problems we face in education. These people should not be teaching. Some people fear that meaningful learning is an excuse for fun and games in classrooms. It is not. Achieving deep learning calls for serious and hard work by students and teachers. Deep learning has the added benefit that students just might find it interesting too.

Proponents of standards-based reform point to particular schools, which for years had been achieving nothing, and are now at least doing something. Perhaps this allows us to think we are doing the best we can with our afflicted, often urban, failing schools. But, if our education system has a problem, it is in our failing schools where the problem is most pressing. And it is the young people in urban schools that most need the kind of education that gives them a chance to be full participants in the knowledge age that will characterize their world.

If we addressed our education crisis with a fraction of the urgency applied to our financial crisis, we might make some significant strides. We need a public debate about education that examines some of our assumptions, most notably that the school experience, which may have worked for them, is not going to work for their children. Education needs leadership that can explain what high standards are and what it will take to achieve them. Education needs the incentives to attract really talented people, educate them appropriately, and support them in all our schools. Setting standards and holding schools accountable for achieving them is a great political applause line and an easy winner on the editorial page. But standards-based reform is reform on the cheap.

Al Rudnitsky is a professor of Education & Child Study at Smith College.


No comments: