Harry Potter and the Standardized Test

(I delivered this sermon at the Unitarian Universalist Fellowship of the Peninsula on July 7th 2013.)

Video: from “Harry Potter and the Order of the Phoenix

During the fifth installment of the Harry Potter series, the Ministry of Magic tries to take over the Hogwarts School of Wizardry and Witchcraft.  Some of this is driven by the supporters of the evil Lord Voldemort who work at the Ministry of Magic; they create a smear campaign to discredit Harry, who personally witnessed Voldemort’s rebirth, as well as Hogwarts headmaster Albus Dumbledore.  Some of it is driven by the personal paranoia of the Minister of Magic, Cornelius Fudge, who refuses to believe that Voldemort is back and is instead convinced that Dumbledore is raising a secret army to make himself the new Minister of Magic.

The agent of interference at Hogwarts is Fudge’s Senior Undersecretary, Dolores Umbridge, whose pinkly saccharine manner belies a cruel and vindictive soul.  At first she is on the staff as a teacher, but then becomes acting headmaster when Dumbledore is removed.  Through an ever-growing number of Educational Decrees issued by the Ministry of Magic, which are framed and hung on the wall outside the Great Hall, Umbridge imposes her draconian rule over the students, the teachers and all other aspects of school life.

In the end, though, it is the students themselves who fight back, with Harry secretly training the others in the defensive spells that Umbridge refuses to teach them, and the Weasley twins Fred and George generating mayhem where appropriate.

In this scene from the movie, things come to a head during an “Ordinary Wizarding Level” or OWL exam that Harry and the other fifth-years are taking under Umbridge’s watchful eye.

Sermon: “Harry Potter and the Standardized Test”

Some of you may not know this, but I actually went to school at Hogwarts.

Oh, I don’t literally mean the magical castle with its animated paintings, fantastic creatures, shifting staircases and energetic ghosts.  But from the age of seven until I was eighteen I did attend British boarding schools, with big halls where we ate our meals and took our exams, dormitories where we slept, and even houses into which we were sorted, though instead of being named Gryffindor, Hufflepuff, Ravenclaw and Slytherin, they had names such as Romans, Saxons, Vikings and Normans.  Such schools were, of course, the basis for J. K. Rowling’s creation of the Hogwarts School of Witchcraft and Wizardry which in a 2008 survey was, in spite of its fictional status, voted one of the best schools in Scotland.

And, of course, the British educational system generally was part of Rowling’s inspiration.  When I was in school we had two sets of nationwide exams that students took at the ages of sixteen and eighteen respectively.  At sixteen we all took the Ordinary or O-Levels in just about every school subject, which Rowling turned into the “Ordinary Wizarding Level” or OWL exams.  How each of us did in those determined which three or perhaps four subjects we might study for the next couple of years before taking the Advanced or A-Levels at eighteen, to which Rowling’s equivalents are the “Nastily Exhausting Wizarding Tests” or NEWT exams.

Now when I was working my way through the O- and A-Level system, there were, of course, other end-of-year exams, designed within the school, to test us on what we were supposed to have learned during the course of each school year.

Few of those exams, I should note, involved questions with multiple-choice answers.  Other than the occasional essay question, most questions required long answers, where we students were expected to provide not only an answer but to demonstrate the reasoning that went into figuring out that answer.  In fact, it was possible to get partial credit even for an incorrect answer, if some of the reasoning that went into it was still valid; on some exam questions the reverse might even be true, that a correct answer all by itself with no demonstration of how it was obtained would not receive full credit.

In college, too, there were similar exams, the final results of which were considered in regard to admission to graduate programs in the UK and by many potential employers, too.  But when I started down the path of applying to graduate schools in the US, I was told that I needed to take the Graduate Record Exam or GRE as part of the admission process.  The GRE, I discovered, was nothing but multiple-choice, with the answers marked by filling in these little circles on a computer-readable form using a number-two pencil.  Oh, and I’m guessing that part of the pretty high fee we had to pay to take the GRE went toward shipping those number-two pencils from the US because that’s not how pencils are categorized in the UK.

Now fifteen years after I went down to London one very cold October morning to take the GREs, I found myself teaching other college graduates how to prepare for the exams.

I worked for one of the big test preparation companies, which the legally binding agreement I signed to get that job prevents me from mentioning, and during my time with them I helped people prepare for a number of the standardized tests that are used to help determine admission to higher education in the US.  I got to revisit a lot of grade-school math and English, though I was disappointed to discover that essays had replaced the abstract reasoning section that had been part of the GRE when I had taken it.  More than that, though, I felt like I had become part of a privileged inner circle that had been given the secrets to unlocking these standardized tests.  And I guess that’s part of the reason why people who can afford to do so — or whose parents can afford to do so — pay hundreds if not thousands of dollars to take such test preparation courses and why the content of them is proprietary.

Now that’s not to say that being comfortable with arithmetic, algebra and geometry and having a good vocabulary and a good grasp of grammar aren’t important in these exams.  They certainly are.  But for most of the multiple-choice questions created for such standardized tests, being proficiently literate and numerate is very nearly besides the point.

For instance, knowing how to do long-division is actually a handicap on questions that look like they need long-division to answer.  Now I learned how to do long-division when I was eight, and I was so proud after the class where our teacher taught it to us, that I went back to my teacher from the previous year and showed her.  (Obviously she didn’t already know how or she’d have taught it to us herself.)  I remember whole sets of questions we were given for homework that involved doing long-division, as well as a more general emphasis in all of my school science classes that precision was something to be valued.  I don’t remember ever being taught with equivalent dedication about how it’s sometimes okay to estimate the answers to some long-division problems as opposed to calculating them, that in some circumstances estimation provides an answer that is good enough in its imprecision — or, as an old colleague of mine used to say, “close enough for government work” — but will at least do so faster.

So one example of a standardized test question is to find the answer to some horrible-looking division problem, like 2,393 over 607.  (I don’t use math to illustrate my sermons too often, and I hope this doesn’t induce any traumatic flashbacks in anyone!)  Now this problem is specifically designed for the student who knows how to estimate: that student quickly rounds the top up and rounds the bottom down and concludes that the correct answer is a little less than four, which of course matches just one of the possible answers on the test.  The student who knows how to do long-division, on the other hand, ends up with a more precise answer of 3.94…, which of course also matches just one of the multiple-choice answers, but they spent so long doing the long-division that they’re now four questions behind the student who estimated.  There’s also a bigger chance of making a mistake in calculating than in estimating.

So what is such a question actually testing?

In most situations in real life where long-division is actually needed, chances are it won’t lend itself nicely to estimation.  Remember that a problem like this is intentionally designed to benefit the student who knows how to estimate.  It’s an artificial problem in another way, too. Real life problems do not come with a pre-determined set of possible answers, one of which is guaranteed to be correct.  So the question is not testing the student’s ability to solve such a problem in anything like a realistic situation.

In these and all other such questions where there are tricks and tools for taking shortcuts to the correct answers, and even for improving your odds of simply guessing if that’s all you can do, the questions aren’t really testing students on what they appear, at first, to be about.  Most of the questions, in fact, are testing how well the students have learned to use the tools and the tricks, which means that what standardized tests are really testing is how good the students are at taking standardized tests.

Now I maintain that it is important for schools to assess students on what they’re learning, and when students from all over the country, even all over the world, need to be evaluated on as level a playing field as possible, it’s clear that tests that influence college admission decisions, for example, need to be standardized.  But let’s not kid ourselves that what standardized tests are really testing is anything other than the ability to do standardized tests.

And as a tool for evaluating teachers, when there are so many other factors at play such as the socioeconomic backgrounds of the students, the amount of support and encouragement that they’re getting from their families, standardized tests simply do not yield the accountability that was promised of them.  It’s not surprising that people have taken to referring to “No Child Left Behind” by other names such as “No Teacher Left Unshamed”.

All too many public school teachers find themselves “teaching to the test”, because that’s what matters most when it comes to their continued employment as teachers or even when it comes to the continued existence of their school.  Some school districts report that their teachers spend as much as forty-five days, in other words a quarter of the school year, in preparing and administering tests, even at the same time that the curriculum is dumbed down to be more suitable for standardized testing, sending higher level reasoning and critical thinking into the trash can right along with art and music.

Now in spite of the fact that there is no evidence of success when it comes to either student improvement or teacher accountability, the US relies upon standardized testing far more than any other economically developed nation.

The Texas program that was the prototype for “No Child Left Behind”, for instance, only appeared to be successful at the time because districts were fudging their numbers, such as by under-reporting dropout rates.  What’s more, reducing both students to test scores and teachers to test score producers gives the students incentives to cheat and gives the schools incentives to dump hard-to-teach students.  The culture of testing, in fact, enables what is known as the “school-to-prison pipeline”, where once again students of color, students with disabilities and students from low-income families are disproportionately impacted.

So the fact that Virginia’s testing regime is known by the initials S.O.L. is little more than a cruel irony.

Now I could continue by talking about how much standardized testing costs school districts, which now pay over a billion dollars a year to for-profit companies for the creation and administration of tests.  I could talk about how such tests fail to be teaching tools because they provide no evaluative feedback that closes the didactic loop in order to reinforce the original learning and guide continued improvement.  I could talk about how, with my daughter’s first birthday less than two weeks away, I’m conflicted about sending her, as and when, to public schools, given the culture of testing, and yet I believe supporting public education is the most important way we can resist the systematic destruction of the middle class that’s been taking place during the last two decades.

But I want to change gears and talk about how we, as Unitarian Universalists, do children’s religious education.  Aside from the fact that, right now, we’re doing a Summer RE program specifically based on the content of the Harry Potter stories, how we do RE may have more in common with J. K. Rowling’s fictional Hogwarts than with what goes on in public schools, even if that’s the parallel we tend to draw.

For all that we’ve embedded what we do for children in the larger process that has been named “life-span faith development”, I’ve come to the conclusion that what we do may be better termed religious exploration.  But there’s one aspect of the word “education” that still holds value, if only we can remember to hang onto it, and that’s because it comes from the Latin “educare” which means “to draw forth”.

While secular education consists, at least in theory, of the imparting of the facts and skills deemed necessary for life in today’s world, religious education is a drawing forth of one’s inner being, building upon personal and shared experiences to grow a soul that is capable of shining life into today’s world.  It is, as poet William Butler Yeats noted, “not the filling of a [bucket], but the lighting of a fire.”  Or in the words of Unitarian minister William Ellery Channing, “The great end in religious instruction is not to stamp our minds upon the young, but to stir up their own.”

It’s certainly important for religious education to include some didactic components — drawn, for example, from Unitarian Universalist history and theology as well as from the wisdom of the other faith traditions of the world — but they serve to support faith development in its largest sense, namely the formation of Unitarian Universalist identity at its best.  So within our Sunday school classes, we aim to give our children a basic understanding and appreciation of many different forms of spirituality and many different ways of approaching life, encouraging in them a respect for religious difference in general and for their own religious heritage in particular.

Religious education also needs to take place in age-appropriate ways and should take into account different learning styles.  I projected that long-division problem on the screen, for instance, because most of us are visual rather than auditory learners.  As Confucius is reputed to have said, “I hear and I forget. I see and I remember.”

Of course, Confucius completed that saying with “I do and I understand.”  We can’t just teach UU theory: we must teach UU practices, too.  Thus religious education extends beyond the congregation itself to the family setting as well.

I was actually stunned a few years ago when I realized that Sunday school alone only represents about forty hours of religious education each year.  That’s the equivalent of just one regular work week — fewer hours than were part of last week’s GoldMine youth leadership school — and yet we somehow think that that’s enough to teach our children about our faith and to help them grow up to be the sorts of adults we can only wish we were.  So no, the religious education of children and youth that takes place on Sunday morning should serve to support and enhance the religious education that they are receiving at home, rather than the other way around.  Parents are thus the primary religious educators of their own children and the congregation should provide them with the tools necessary to responsibly and successfully take on this role.  And, frankly, secular education should be viewed more like that, too.

Now I want to note that DRE Joanne does include activities for her RE classes that allow the lessons to be taken home and continued.  I’m pleased to know that many of you who are parents of children in RE here have used those activities at home and have given Joanne positive feedback on them.

A lot of how we do religious education ultimately comes back to the nature of Unitarian Universalism as a faith tradition.  While most religions have a basis in creed, which is a particular statement of belief, Unitarian Universalism inherits from its religious forbears a basis in covenant, which is a particular standard of behavior.  As generally interpreted for the purposes of religious education, this means that the way of approaching belief is more important than the content of belief. In other words, while we have a vision of ourselves as a community that “offer[s] a safe place for spiritual diversity and individual growth through lifelong religious education”, we nevertheless strive to place the pursuit of truth within a context of respect, kindness, responsibility and fairness.  As Universalist minister and educator Angus MacLean put it, “The method is the message.”

Weirdly enough, that’s actually one of the philosophies of standardized test preparation.  One of the things we teachers tried to get our students to remember was that it wasn’t the content of the example problems we worked through with them that mattered.  Rather, it was the tricks and tools we used for tackling the problems that we wanted them to remember.  Any specific problem, after all, would probably never come up in a test in that exact way, so it was the general means of solving the problem that needed to be remembered.  We assumed that higher level of reasoning as part of preparing for test questions that didn’t require it!

And in the Harry Potter stories, what matters to Harry and Hermione and Ron throughout their seven years of battling Voldemort and his minions isn’t really the specifics of the spells and the potions that they learned at Hogwarts, though they certainly help.  What made a difference definitely wasn’t the narrow curriculum approved by the Ministry of Magic.  What did make a difference to them were the resources for courage and hope that they found within themselves and within one another, and the love that made them and their friends stronger together than they would have been alone.

And in the religious education we do here, what matters isn’t whether we know all the details of Unitarian Universalist history or can recite by rote the words of all Seven Principles and all Six Sources.  What matters are things that can’t possibly be evaluated by multiple-choice problems: that we bring a willing spirit; that we offer one another an open mind, a loving heart and a helping hand; and that we engage together in this religious exploration by building upon our personal and shared experiences and dreams so that each of us may grow a soul that will shine life abundant into the world.

May it be so.



  1. […] July 7th: “Harry Potter and the Standardized Test” […]

  2. smking1 said

    Wonderfully written! What a wealth of understanding you provide. Thanks!

RSS feed for comments on this post · TrackBack URI

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: