Key Performance Indicators and the Heightening Contradictions of Academia

Chris Beneke

Of what value is your scholarship? Historians in Britain are receiving unsettling precise answers to that question. In case you hadn’t heard the news, British academics are now locked into a quality control regime that forces them to measure up against “Key Performance Indicators” over a 6 to 7 year span. The measures are largely determined by government officials though the actual measuring is done by historians.

Better minds have already suggested that the quantification of humanities scholarship through such mechanisms will dampen creativity, discourage ambitious long-term projects, and lower scholarly quality, while sucking much of the joy out of professional historical work. Randall blogged about funding-driven assessment tools a couple of weeks ago, Anthony Grafton’s AHA President’s column mentioned it in January, and Simon Head recently wrote a more extended analysis for the New York Review of Books. (I did some hand wringing myself in November.) Nonetheless, one paragraph in Simon Head’s account of the system struck me as especially noteworthy:

In the humanities the RAE (Research Assessment Exercise) bias also works in favor of the 180–200-page monograph, hyperspecialized, cautious and incremental in its findings, with few prospects for sale as a bound book but again with a good chance of being completed and peer-reviewed in time for the RAE deadline. A bookseller at Blackwell’s, the leading Oxford bookstore, told me that he dreaded the influx of such books as the RAE deadline approached.

In other words, the new British accountability systems seems perfectly designed to heighten the contradictions in the academic world, ensuring that the “research output” that scholars are dourly incentivized to produce is less accessible to the larger public and therefore less likely to contribute to the informed consideration of things that the public worries about—like, for instance, big social, political, and ethical problems. The argument can surely be made that rigorous research measures drive scholars toward more focused and more readily publishable research, which will ultimately makes a greater indirect contribution on the world. But I doubt that it would be convincing, especially when it comes to historical research.

Indeed, the RAE seems well designed to thwart scholars who believe that they should write engaging volumes that people who don’t care a whit about the distinction between social history and cultural history (the common term for them is “general readers”) might actually desire to read. Despite the precarious state of academic publishing, historians have enjoyed the indulgence of academic presses in recent years because of their faith that we will eventually write such books—if not the first time around, then the second, or third. It’s hard to imagine that system holding up if we’re flooding bookstore shelves with carefully calibrated units of research output, rather than, you know, books.

Rarely is the Question Asked: Is Our Professors Teaching? Part III

Randall Stephens

Guess what? Many college students do not learn analytical and writing skills during the four years they spend in college. Students don't study. Courses are not demanding. Collaborative learning does not work like professors think or hope it does. . .

Or, so argues a new book, Academically Adrift: Limited Learning on College Campuses (University of Chicago Press), by Richard Arum and Josipa Roksa. More and more students--more likely parents--are throwing down the cash for college. But the authors ask: "are undergraduates really learning anything once they get there?"

Last week the Chronicle highlighted Academically Adrift and the authors' controversial findings. (David Glenn, "New Book Lays Failure to Learn on Colleges' Doorsteps," Chronicle, January 18, 2011.) Arum and Roksa tracked 2,000 students at 24 four-year colleges. Thirty-six percent of these students who took the Collegiate Learning Assessment essay test showed no significant improvement from their freshman to senior year.

Arum and Roksa certainly have their critics. The study asked too few questions about collaborative learning, say some. Others say that the study, limited in scope, should not challenge the whole undergraduate enterprise.

But, overall, the findings should give us pause. "Mr. Arum and Ms. Roksa don't see any simple remedies for the problems they have identified," writes David Glenn in the Chronicle. "They discovered more variation in CLA-score gains within institutions than across institutions, and they say there are no simple lessons to draw about effective and ineffective colleges." Still, Glenn points out that business and education programs in Texas colleges require that students "take only a small number of writing-intensive courses." The path of least resistance.

Are students today less likely to major in history when the workload is high and the perceived payoff is so low? ("So I'm going to spend all this time reading primary and secondary works just so I can be unemployed after four years of reading, writing, and reading some more?") Five years ago Robert Townsend noted in Perspectives that: "Information from the latest Department of Education (DoE) report (pertaining to the years 1997–98 to 2001–02) suggests that in the competition for students, history lost ground while the total number of undergraduate students at colleges and universities grew quite quickly." I haven't see more recent data, but I can't help but think that there are fewer majors today then there were 20 years ago.

Perhaps history departments could do a better job of emphasizing the portable skills students learn in the major. Why not stress in clear terms that history trains students to think critically and to write clearly? I have my students read Peter Stearns excellent essay, "Why Study History," for this very reason. They learn that history students gain: "The Ability to Assess Evidence. . . . The Ability to Assess Conflicting Interpretations. . . . Experience in Assessing Past Examples of Change." Stearns ably shows that "Work in history also improves basic writing and speaking skills and is directly relevant to many of the analytical requirements in the public and private sectors, where the capacity to identify, assess, and explain trends is essential." I've also had students read Heather's excellent post on this subject from our blog. She noted: "History is the study of how and why things happen. What creates change in human society? What stops it? Why do people act in certain ways? Are there patterns in human behavior? What makes a society successful? . . . . When you study history, you’re not just studying the history of, for example, colonial America. You’ll learn a great deal about the specifics of colonial America in such a class, of course, but you’ll also learn about the role of economics in the establishment of human societies and about how class and racial divisions can either weaken the stability of a government or be used to shore it up."

Sounds like a cure for the "I-learned-little-in-four-years-of-college" blues.

Rarely is the Question Asked: Is Our Professors Teaching? Part II

Heather Cox Richardson

Randall asked a good question in his post wondering whether or not college and university professors are encouraged to improve their teaching. He has inspired me to blog about teaching issues in a more systematic way than I have before.

Today the topic that is consuming me is assessment. This is not a new obsession, either on my part or on that of the profession. We’ve talked about assessment for years. . . but what have we learned?

What, exactly, do we want our students to learn in our classes? Long ago, I figured out I should design my courses backward, identifying one key theme and several key developments that were students’ “takeaway” from a course. That seems to have worked (and I’ll write more on it in future).

But I’m still trying to figure out how to use assessments, especially exams, more intelligently that I do now. My brother, himself an educator who specializes in assessments, recently showed me this video (below), which—aside from being entertaining—tears apart the idea that traditional midterms and finals do anything useful in today’s world.

Shortly after watching the video, I happened to talk separately with two professors who use collaborative assignments and collaborative, open-book, take-home exams. They do this to emphasize that students should be learning the real-world skills of research and cooperation just as much—or more—than they learn facts. As one said,

facts in today’s world are at anyone’s fingertips . . . but people must know how to find them, and to use them intelligently. This is a skill we can teach more deliberately than we currently do.

These two people are from different universities and are in different fields, but both thought their experiment had generally worked well. One pointed out—as the video does—that the real world is not about isolation and memorization; it’s about cooperation to achieve a good result.

The other said she had had doubts about the exercise because she had worried that all the students would get an “A.” Then she realized that it would, in fact, be excellent news if all her students had mastered the skills she thought were important. When she actually gave the take-home, collaborative assignment, though, she was surprised—and chagrined—to discover the same grade spread she had always seen on traditional exams. She also saw that some of her student groups had no idea how to answer some very basic questions, and that she would have to go back over the idea that history was not just dates, but was about significance and change.

And that is maybe the most important lesson. The collaborative exam revealed that there were major concepts that a number of students simply weren’t getting. So she can now go back and reiterate them.

I’m still mulling this over, but I do think I’ll experiment with collaborative assessment techniques. Historians have some advantages doing this that teachers in other fields don’t. We can ask students to identify the significance of certain events, to write essays, and to analyze problems. With the huge amount of good—and bad—information on the web in our field, though, we could also ask students to research a topic, then judge their ability to distinguish between legitimate and illegitimate sources (something that might have helped Joy Masoff when she was writing her Virginia history textbook).

As I’ve been thinking this over, a third colleague has inadvertently weighed in on it. He discovered students had cheated on a take-home exam, working together and then slightly changing each essay to make them look original. At least an assigned collaboration would eliminate the problem of unapproved collaboration!

Rarely is the Question Asked: Is Our Professors Teaching?

Randall Stephens

Academics are, by nature, hand wringers. We worry about the decline in the humanities. We worry about grade inflation. We worry about the troubles of academic presses. Once in a while we worry about the state of teaching. Or, to paraphrase our former president, "rarely is the question asked: is our professors teaching"?

Quite often the appraisal of teaching is negative, though academics and non-academics offer different points of view. In the popular imagination, the old stereotypes persist, as Anthony Grafton points out, with tongue firmly in cheek:

We don’t teach undergraduates at all, even though we shamelessly charge them hundreds of dollars for an hour of our time. Mostly we leave them to the graduate students and adjuncts. Yet that may not be such a bad thing. For on the rare occasions when we do enter a classroom, we don’t offer students close encounters with powerful forms of knowledge, new or old. Rather, we make them master our “theories”—systems of interpretation as complicated and mechanical as sausage machines. However rich and varied the ingredients that go in the hopper, what comes out looks and tastes the same: philosophy and poetry, history and oratory, each is deconstructed and revealed to be Eurocentric, logocentric and all the other centrics an academic mind might concoct.*

Across the water, historian and filmmaker Tariq Ali and and Harvard historian and teledon Niall Ferguson speak to the BBC about what they see as the abysmal state of history teaching. (Hat tip to the AHA.) Students stop pursuing history in England at an early age, says Ferguson. And what history is taught is "too fragmentary." Ali agrees, saying that what is presented is, basically, "worthless," and hobbled by a chasing after so-called relevance. They both argue that the old anachronistic, triumphalist, island history of Britain, should be avoided, but students need a larger narrative. "It could hardly be worse than what is going on in schools today," concludes Ferguson.

How does history teaching fare in America's colleges and universities? Are teaching awards more than a feather in the cap? Do promotion and tenure committees value persistently good evaluations and commend teaching effectiveness in the same way that they reward scholarship? Do peers sit in on classes and make assessments? Do departments do anything when a professor continues to receive poor teaching evaluations one semester after another?

Nearly ten years ago Daniel Bernstein and Richard Edwards proposed that we need more peer review of teaching in the Chronicle. "[I]f educators are going to sustain the progress made, we will need to move toward a more rigorous and objective form of review," they wrote. "The goal of peer review has been to provide the same level of support, consultation, and evaluation for teaching that universities now provide for research." I can't imagine what the results of such efforts have been. Certainly, peer evaluation can turn into a messy, political business.

Does graduate training in history prepare men and women for classroom success? Budding historians spend far more time in graduate school working on research, parsing theory, and getting the historiography down. Less time is devoted to developing teaching skills and, at least as it was in my case, there is not much mentoring on teaching. (Most grad students I encountered came prewired with an interest in teaching. So, that was a plus.) Could graduate training be better oriented to prepare good history teachers? What would that look like?