So…. assessment

Through grad school and my visiting position at Kalamazoo, I was blissfully shielded from the idea of assessment. The veil was lifted when I came to Shimer, and I must confess that after our first meeting to discuss the issue, I was horrified. I remain suspicious of the “education reform” movement in general, but I have come to think that assessment — at least as it is conceived by Shimer’s accrediting body — might not be a bad thing. If I imagine that I’d never heard of Arne Duncan or charter schools and someone came up to me and suggested that my school might want to clarify its goals, gather information about how it’s doing in meeting those goals, and change its practices if it’s not working, I would find it hard to object. It bears a cost in terms of time and energy — but then so does plugging away at something that’s not working. Paying a reasonable number of hours up front in terms of data collection seems like a reasonable “hedge” against wasting countless hours on a pointless boondoggle.

One concern, particularly in liberal arts circles, surrounds the numerical aspect of assessment. I share the skepticism that numbers can really “capture” what we do, but we already use numerical forms of assessment all the time — we just mostly translate them into alphabetical form. It’s also widely acknowledged that professors as a whole have wavered in their dedication to rigorous grading, due in large part to the increasingly disproportionate real-world consequences grades can have on their students’ lives. The “grade inflation” trend reportedly began as professors were unwilling to condemn a student to die in Vietnam because their term paper was too short, and the financial consequences of grades likely play a similar role today. Hence it makes sense to come up with a parallel “internal” system of measurement so that we can be more objective (and also measure progress on a single scale, without the difficulty of deciding whether a B in an advanced seminar denotes an advance over an A in a freshman intro class, etc.).

When it comes to cross-institutional comparisons, however, we use a much more insidious numerical measure to signal educational quality: price. Education is a perverse market where everyone is in competition to charge more, because that’s how you signal that you’re an elite institution. There are other, more detailed measures such as those collected by the widely derided U.S. News and World Report ranking system — but those measures largely treat the educational process itself as a black box, and they’re incredibly easy to game. (For instance, I recently learned of an institution that decided to replace all its contingent faculty with full-time, non-tenure-track faculty at one fell swoop, due solely to a change in the U.S. News ranking algorithm.)

Obviously the problem with the assessment movement is that it’s embedded in a neoliberal logic whereby the government’s role is to make sure markets function as much as possible. Ideally, of course, there would be a robust “public option” in the form of well-funded state schools that provide a quality education for minimal cost, which would automatically bring down prices across the board. Unfortunately, we do not live in a world where that option is seriously entertained — everything has to be achieved via a “bank shot.” In this case, they’re trying to create a market based on demonstrable effectiveness rather than who can charge the most and/or offer the most extra-curricular amenities. The reasoning is presumably that once consumers have this more robust and relevant information, they won’t tolerate inflated tuition for a school that delivers the same basic outcome as a community college (for example). The government is also likely going to play a more forceful role by increasing the conditionality of financial aid, which would mean that one of the biggest drivers of tuition inflation, namely the unlimited government subsidy of whatever the school feels like charging, would be under control. (I assume this is part of what’s going on with the consolidation of control of the student loan program more directly in the hands of the Department of Education.)

It will almost certainly turn out to be the case that this elaborate “bank shot” strategy will not deliver the promised results overall. It is also possible that in the future, the assessment regime will be harnessed to force greater homogeneity in the educational market. As it stands now, however, all indications are that Shimer’s accrediting body is shielding institutions from any homogenizing force and encouraging them to create assessment programs that reflect their own unique goals and institutional ethos. In other words, for now the question is not whether we’re measuring up to some arbitrary standard, but whether each institution can make the case that they are delivering on what they promise.

Hence — leaving aside for the moment the market-making effects assessment is supposed to have, as well as suspicions about how it could be used in the future — it seems possible to come up with an assessment system that would actually be helpful for figuring out how to be faithful to each school’s own goals. I have to admit that part of my sanguine attitude stems from the fact that Shimer’s pedagogy embodies what independent researchers have already demonstrated to be “best practices” in terms of discussion-centered, small classes — and so if we take the trouble to come up with a plausible way to measure what the program is doing for our students, I’m confident the results will be very strong. I’m also sure that there are some things that we’re doing that aren’t working as well as they could, but we have no way of really knowing that currently.

We all have limited energy and time, especially at a place like Shimer where the faculty needs to take on a lot of the burden of administration of the school in addition to teaching, and so anything that can help us to make sure we’re devoting our energy to things that are actually beneficial seems all to the good. At the same time, though, I recognize that not every school has the strong shared governance that Shimer does, and so the assessment regime will most likely be more arbitrary and unhelpful when designed by the autonomous administrative class that wants to “run the college like a business.” That’s a separate question from assessment as such, however — and I’d even suggest that faculty resistence to the very idea of assessment may further weaken faculty governance and allow the “bad” administrative types even more of a free hand.

So there you have it. I look forward to hearing about what a horrifying neoliberal sell-out I am.

6 thoughts on “So…. assessment

  1. “The reasoning is presumably that once consumers have this more robust and relevant information, they won’t tolerate inflated tuition for a school that delivers the same basic outcome as a community college (for example).” Isn’t the outcome of going to an expensive school in large part having a diploma from an expensive school? For most employment purposes what you actually learned isn’t that relevant, so the only information you should need to measure the value of your degree is how expensive everyone thinks it is.

  2. You’re not a sell-out, just a faculty member concerned about how your institution can educate more efficiently and effectively.

    On a side note, it’s all well and good for Heidegger to bash technological modernity for being nihilistic and for Foucault to criticize contemporary webs of power/knowledge. But maybe utilitarianism is appropriate in small doses – especially in domains like education and medicine – since a lot of people would be harmed by inefficient resource use.

  3. I’m learning a lot about assessment, too, since I’m also relatively new at my job as a professor. I’m also quite happy with the idea of trying to figure out if we’re doing what we think we’re doing. But quantification is always going to be a very weird thing. And there’s the risk of self-assessment, which always might be more dependent on what the person just ate for lunch, or how much sleep they got, etc., than the actual outcome.

    But really I just want Cambyses to weigh in on this.

  4. I wonder if Jeff Nealon’s latest book, Post-Postmodernism, might make am contribution here. Nealon argues for an intensification of neoliberalism to either a.) push it to the limits of invention, or b.) force it to become something new. In terms of the university this means student and professor investment where middle management and administrators typically have stakes. In short, this collapses the ivory tower tropological economy and replaces it with a market reality. Grad student and profs become commodities to the institution, which allows leverage to do all sorts of things. Most importantly, ranking like us news goes away because it would no longer reflect market realities of “quality” but would reflect the amount of money an institution has and the access for professors and students. Assessment then becomes quite normalized – which is an advantage because there are clear value markers, rather than vague student likes and dislikes and amount of publication.

  5. As a staff member of a college of arts that has been in unresolved restructuring since 2003(!); in which it is almost impossible to fire poor teachers and non-existent researchers; in which ‘technology’ is seen as the answer yet students prefer lectures that stimulate questions and not just act as power-point slabs of information to be regurgitated, the reality of assesment is that they need to have real change outcomes . Too much ‘assesment’ without positive, actual change for the better becomes an on-going set of weberian bureaucracy that just disenchant all involved. What is needed is a down-sizing in managers and managerialism and an increase in the accountabilty of academics. Too often universities operate as a type of sheltered workshop- as my wife comments- ‘if you are one of the most capable and responsible people in your college then god help us all”( as an atheist catholic she speaks…)

Comments are closed.