Teacher performance should not be measured by a series of multiple-choice questions

In a February 4, 2010 Op-Ed in the Boston Globe, University of Virginia Professor Daniel Willingham and author of “Why Don’t Students Like School?’’ writes about the issue of “Merit Pay” for teachers as required by The President’s education reform initiative “Race to the Top.” While the majority of Orange County school districts opted out of committing to the California plan to accept Race to the Top funding, the issue of “merit pay isn’t going away.

Turning schools into Registry of Motor Vehicles

In an effort to improve public schools, President Obama wants to hold individual teachers accountable for student test scores; indeed, states that prohibit the practice are ineligible for the “Race to the Top’’ funds.

To a cognitive scientist, this is a strange line to draw in the sand. We do not have good tools to measure teachers, and when you hold people accountable with poor measures, things don’t just fail to improve. They get worse.

The reason is simple: Accountability changes workers’ focus from “do a good job’’ to “do a job that looks good according to the measure.’’

One approach to classroom accountability is to measure children’s learning and let the teacher do whatever they think is best. You simply administer a test in the fall and one in the spring and find the difference. That’s intuitive, but there are a number of conceptual and technical problems.

Obviously, teachers have little incentive to teach any topic that is not tested, or indeed, anything that will not be tested that year; why lay groundwork for improving next year’s scores? If you thought No Child Left Behind led to an overemphasis on testing, wait for the test-prep frenzy that follows linking salaries to test scores.

Another problem: not everything is in the teacher’s hands. Rowdy kids are harder to teach than well-behaved kids. And it’s easier to teach your class if your principal (and parents) are helpful and supportive. Several studies have shown that teacher evaluations based on test scores are unstable. About 25 percent of teachers pegged as terrific or terrible get the opposite designation the next year.

The logic underlying this approach is suspect. It assumes that teachers know what to do but just aren’t doing it or that they will figure out what to do once the pressure is on. It’s the equivalent of the frustrated parent shouting “I don’t care how you do it – just bring home better math grades!’’ No Child Left Behind should have taught us that improving student achievement doesn’t happen simply by mandating it.

So what if you do tell teachers how to improve? A second approach limits accountability to how teachers do their job. You observe teachers in the classroom and see whether they are using what are known to be good teaching practices. The problem is that people then become slavishly devoted to the rules, because it is to the rules that they are accountable. Call it RMV Syndrome.

I once waited in a long line at the Massachusetts Registry of Motor Vehicles only to be told that I needed an additional form. I saw the form about two feet from the clerk, but he insisted I wait in a different line for that form. Maddening for me, but perfectly sensible from his point of view. Why should he break the rules and risk punishment, just to save me a wait in line?

Social scientists have a technical term for this type of behavior. It’s called “covering your butt.’’ This type of accountability only works if the list of required behaviors is so intelligently constructed that in covering their butts people end up doing a good job. It can also work when the supervisor is knowledgeable and flexible; the RMV clerk might have known that his supervisor would understand that giving me the form was technically breaking a rule, but contributing to the larger goal of effective service.

There are ways of making accountability work. The two key elements are evaluations that take place over long periods of time, to increase stability, and evaluations that are conducted by people who are knowledgeable and are known by teachers to be knowledgeable. Unfortunately, neither element is part of the Obama administration’s plans.

Advocates of teacher accountability often acknowledge these problems, yet insist it’s better than nothing. Not true. A poor system could make teaching worse and a failed attempt will allow opponents to dismiss accountability as a failed policy. Accountability is a good idea, but we have to get the measures right.

It really isn’t a good idea for us to expand the failures of “No Child Left Behind” by tying teacher pay to their ability to teach students to pass a test, and nothing else. If our testing measures do not test sufficiently on student knowledge related to subjects other than reading, writing, and math, then we fail to teach students as individual people.

By teaching to a test we end up teaching as though we are programming a computer for limited and simple functions rather than teaching students the full spectrum of knowledge necessary to be productive and involved members of society.

Life is not a series of multiple-choice questions and our students are not monkeys trained to use a #2 pencil to fill in bubbles on a sheet of paper.

  9 comments for “Teacher performance should not be measured by a series of multiple-choice questions

  1. Slatemag
    February 8, 2010 at 9:20 am

    Another Demcrat’s big bang and they-know-better theory is going to doom soon. Look around, China and India and other third countries push their students to the limit by giving and taking standard tests for
    their students. They made leap and bound achievements while Democrats keep protecting the techer unions and let the students experimenting another failed social test.

    • JCTarzan
      February 8, 2010 at 4:41 pm

      Other countries weed out their “failing” students and only teach the ones with promise. We try to teach everybody in our nation. Also, ask the 40 percent or so of Republican teachers (at least in California) and they will tell you that the NCLB and RTTT are not good for improving education. Don’t make this a political party thing.

      • Slatemag
        February 9, 2010 at 1:38 pm

        I hope it’s not a political game, but with Obama and Democrats in control their Job One is to protect the teachers’ union or any unions. I have no disillusion here.

        PTA parent.
        If what you propose is to look for a Chinese scholar’s wisdom as our standard of education excellence then you miss the boat. Those commies love to send their offsprings to US to study so when they are back in the mainland they will automatically become overlords over their people. If Zhang thinks our education is perfect then why changes?

  2. PTA Parent
    February 8, 2010 at 12:06 pm

    To Slatemag,

    The article is not about protecting teachers’ unions; nor is it defending teachers’ unions. As a parent, I am very concerned about teaching to the test and narrowing the curriculum. Contrary to your assumption, China is looking to us for how to teach innovation (while we attempt to centralize and standardize). See Professor Yong Zhao’s new book, “Catching up or Leading the Way.” This remarkable book will forever change the debate about what’s wrong and what’s right with American education and where it should be going. Based on his own experience as a student in China and as a parent of children attending school in the United States, Zhao skewers conventional wisdom while setting straight the recent history and current state of US schools. To make his case, Zhao explains:
    Why the perceived weaknesses of American education are actually its strengths.

    How reform proponents, business executives, and politicians have misjudged American education.

    Why China and other nations in Asia are actually reforming their systems to be more like their American counterparts.

    What really matters for an education system and what really counts as educational excellence.

    With an extraordinary command of facts and thought leadership, Zhao describes how schools have to keep pace with a world that is being dramatically transformed by globalization, the “death of distance,” and digital technology. Instead of falling in line with mandates for standardization, his prescription is for educators to

    Expand the definition of success beyond math and reading test scores.

    Personalize schooling so that every student has opportunity to learn.

    View schools as enterprises that embrace globalization and digital technology.

  3. February 8, 2010 at 3:46 pm

    I have no idea what the solution is. I only know that my kid is in a public high school where the math teacher shows episodes of Lost instead of teaching math, he and his classmates cannot keep up later, the teacher is still employed (tenure) and i get to pay for tutors to teach him what he should have learned last year. If testing forces this twit to actually teach his students, do it. Those who are teaching need to be rewarded, there are great teachers out there who make the same money as this moron who runs TV programs in class, and it is demoralizing to them. Bonuses for those doing well, and pay cuts for those goofing off. Not perfect, but what is?

  4. PTA Parent
    February 8, 2010 at 6:34 pm

    To Colony Rabble,

    I agree that the teacher who is wasting everybody’s time should be fired. But I still have a problem measuring a teacher’s competency based on mutliple choice test scores, especially given that class sizes are now often above 40 students, and it is near impossible to keep everyone engaged and on task when the teachers are expected to follow a rigid pacing guide. Budgetcuts have decimated staff and morale at my child’s school is very low. The wheels are falling off and nobody’s driving.

  5. Deborah White
    February 9, 2010 at 11:54 am

    Love of and excitement for learning leads to academic success. No one loves to take tests, and many dread it. Fantastic test scores can be a reliable predictor of success in college, but not always. And some of the greatest successes were terrible test takers.

    Further, compensating teachers mainly based on their students test scores on one standardized test doesn’t reward either teaching or causing students to be interested in learning.

    From experience, I can tell you some of what it rewards: giving kids sugary snacks to juice-up their test performance; pushing bright kids into intense studying of advanced curriculum in order to up their test scores; pressurized memory drills; shunting aside kids less likely to produce higher test scores; and much more.

    And once the test is done, usually mid-to-late April, the last six or so weeks of school are a complete waste of time, academically.

  6. February 9, 2010 at 12:03 pm

    So what is the answer? I agree that tests are a difficult determination of class performance, but I have yet to hear a better answer. And I mean this genuinely. I would love to see true classroom reform in the public school system, and I hear lots of nagative about what does not work, but I truly want to know, what DOES work? Ideas?

  7. Toni Gilmont
    February 9, 2010 at 12:10 pm

    Your article covered a deep-seated problem found in so many schools. I’m in hopes your article made its way to the Administration who continue to encourage this posture which keeps the kids from actually getting what is needed in order to become productive adults in our society. Keep up the good work

Comments are closed.