Assessing assessment with an assessment rubric about assessment

For the past two days, the Shimer faculty was involved in an assessment workshop with a knowledgable consultant who was so enthusiastic about our program that he offered his services for free. One thing that struck me was that I had been thinking of assessment as a kind of “meta” layer on what we already do in terms of grading — coming up with parallel rubrics, etc. In reality, our consultant said, we’re assessing student learning all the time. This is all the more true of Shimer’s discussion-based curriculum, where we get vastly more insight into student learning than you would get in a lecture class. People tend to think of things like learning outcomes as artificial, and they really can be, but as I thought about it, I realized that no student would get an A in my class without demonstrating things like critical thinking, effective communication, etc.

One admittedly optimistic way of looking at the assessment movement is that it prompts college instructors to be more transparent and purposeful about what they are really asking of students. This serves multiple purposes: it can help us to do a better job of judging whether a student really is making progress, it can make for greater consistency in grading over time and across different instructors, and it can help us to provide more meaningful preparation and feedback for students so that they will get more out of our classes. Academics complain that students are obsessed with grades, but when that’s the only kind of feedback we give them and when they are often inscrutable to the students, can we really blame them?

I think that part of the reason I feel that assessment at least has the potential to be really helpful is that a lot of the things they’re asking us to do are things that I’ve more or less spontaneously been trying to do over the course of my teaching career — and in particular since I came to Shimer, where it is totally impossible to rely on the traditional model where as long as I do my best in presenting the information, the rest is really up to the students. For me it started with our writing intensive courses. I asked myself what skills I really hoped that a first year student would be able to take away from my class, and my written feedback was almost obsessively focused on that handful of qualities — and the students who worked seriously and did the rewrites actually improved. I’ve made similar efforts with discussion, trying to find one or two specific styles of thought in each class that I really want to be able to work with. I’ve probably been less successful in that regard, in part because discussion is inherently much more difficult to direct and comment on than written work, but hopefully I’ve made progress.

What makes me more willing to stick to these types of commitments is that I know that at Shimer, there will be someone to pick up the baton after the students leave my class — whereas in my previous teaching situation, it felt like I was working in a total void. I can imagine that after a few more years in a more monadic type of institution, I may have given up and just lectured. The idea of more systematically working through these types of expectations with my colleagues so that I can trust that someone who came from a particular course can be held accountable for some particular skill or habit of thought is even more encouraging in this regard.

The thing that makes the whole regime an artificial imposition is that faculty don’t actually want to sit down and discuss what is most important to them and how they want to hold students accountable for that — and so the goals become empty clichés that we grudgingly copy and paste into our syllabi. A place like Shimer has an advantage in that it’s small and the faculty has a profound investment in the community and institution, so that you don’t get the isolated monads who populate most colleges and universities. And that’s the paradox of the politics of assessment: it’s based on the model of the student as consumer, who should be given full information about the product they’re buying, but what it actually requires is more collaboration and open dialogue among faculty. In other words, it’s premised on the most objectionable liberal individualism, but actually implementing it requires turning our institutions of higher learning into more meaningful intellectual communities.

6 thoughts on “Assessing assessment with an assessment rubric about assessment

  1. In 1919 Count Harry Kessler noted in his Dairy (7 Feb, 1919):
    “All education is violence of a sort, just as every state is. Education, society and the state simply exist to sublimate the crider forms of force into more refined ones. That amounts to a difference in manner and degree, not of principle. The creature educated is no less violated, forced out of his orbit, denatured ( that is indeed the real object of culture) than if coerced by means of less lofty purpose. Nor can it be maintained that he in any case suffers less thereby. It is just that the onlooker is spared the revolting sight of physical violation, blood, pain-distorted features, screams of agony, and death. The process of social incorporation is no less cruel than war, but the onlooker’s sensitive nerves find it more bearable. That is the essential difference.”

    Learning outcomes are especially a type of totalitarian violence whereby there is a predetermination and rubric as to what will be, can be- and should be learned ( or rather- accepted as being learnt…) BEFORE the violence of education takes place. Consider the violence- physical, emotional, psychological and existential of any ‘re-education’ process. These are really the models for ‘Learning outcomes’ and why I do all I can to stand against such impositions and demands. Assessment is really meaningless for most students in mass-education unless it occurs as a type of re-educational violence- and to be party to such a process becomes increasingly distatsteful the longer I spend teaching in higher education. By this I mean all that most students increasingly want is to ‘pass’ the course and pass their degree because it is seen as a passport to a better paying job. This is why we now have the drive to make our courses and assesment activities ‘relevant’; that is ‘relevant’ as determined within a neo-liberal market economy. Yet any ideological claim of relevant assessment and education becomes the threat- and imposition of violence. Those of us in the humanities must be especially wary that we do not seek to impose such forms of re-educational violence in the name of approved and accepted learning outcomes that can signal an institional acceptance and “validity” to what we do. That is we assess in acceptable ways within learning outcomes so we can continue- and our success can be measured…

    A small faculty may be able to create a communal sense of assessment that is not a type of violent imposition on all; but in larger institutions collective and departmentallly-approved and imposed learning outcomes merely serve as ideological violence at “best” ( that is if there is a real and agreed ‘party line’ as to the assessment across the department) and insitutional neo-liberal bean-counting and validation at worst.

  2. That would be an ideal solution but of course is never going to occur. So the alternative is acts of internal resistance within the institutions we find ourselves. This is admittedly far easier when one has achieved tenure so we find ourselves in a double-bind. Younger colleagues starting out ‘play the game’ because they are understandably in pursuit of tenure and the security it brings. Older academics- ie those like myself who have been teaching for years- either tend not to think about such things- or become ‘teaching and learning committee’ zealots who see learning outcomes as the solution to all problems and the way to signal and award validation.
    So perhaps the way is to become an individual ‘shimer’ wherever we find ourselves- and seek out and support like-minded ‘shimers’, especially, for us older academics, those younger coleagues starting out and confronted by the institutional violence of demanded and imposed learning outcomes and ‘acceptable’ asessments…

  3. Okay, so as I return to this thread — what’s the ideal? We all individually do whatever we think is best in class and the students get what they can get? We don’t assess students in any way, so as to avoid violently imposing on them? I hate to break it to you, but grading is a form of assessment (albeit maybe not in the rarefied sense used in many academic admin circles) and I hope that you have some form of criteria that you use in assigning grades, even if they’re implicit. Unless you’re just handing out A’s to everyone, that means that you have goals for your students’ learning and hold them accountable for that in some way — i.e., you are an educator.

    The kind of attitude you display guarantees that assessment will be imposed in a top-down way, because how else can it happen if faculty refuse to participate in the dialogue? It’s a self-fulfilling prophecy.

  4. The ideal is to do away with the type of univeral rubric learning outcomes that are the norm is most universities whereby there are a series of teaching and learning committee buzzwords that get applied wholesale across every course.

    Of course we assess students- that is part and parcel of education, but my point is that learning outcomes actually work against the type of individualized assesment that I try to do. Secondly, don’t have someone else marking what you teach. I really feel that those who teach should mark the work of their classes. Learning outcomes often seem to be in place, alongside marking rubrics, for clases where someone other than the person teaching is marking. One important element in assessment that is often forgotten- or at least down-played, is that those teaching can self-assess how well what they are communicating is being understood- and responded to. So I have always marked all the work of all my students. I write all over their work in red pen, annotating in a conversation where they could improve, what they could add, where they get a fact wrong. The assessment is not in a spirit of pass/fail but rather how could this work be improved. The aim is to improve their understanding, improve what and how they write, think and argue. This is not to say work cannot fail- but if it is not up to a tertiary standard it is not just about failing but explaining why and then discussing with them why their work is failing to reach the expended tertiary standard. I did a period as an academic dean of faculty and this convinced me that too often assessments to a learning outcomes rubric are not working. I would deal with those students who had, in effect, been failed by the system. This is a problem is large classes where students exist as part of a collective and not as individuals who we, as educators, have a resposibilty for.

    My other concern is the type of assessment whereby students know the type of answers and statements that in that class will give them a pass or worse an ‘A’. Thsi is not learning, it is regurgitation- and the type of unthinking non-education that learning outcomes tend to encourage for that mass of disengaged, or intellectually lazy, tertiary teachers.

    Yes there is a violence in marking so the aim is not to slam students or to mark to a collective schedule but to mark each piece on its merits. There is always a criteria and i believe you should set a high bench-mark as we are in tertiary education. But we should not mark to a bell-curve because the students in each class are different. Some classes are far better and more able than others.

    The reality is that departments are composed of disparate individuals of differing abilities and engagement and we all suffer colleagues who pass students who shoudn’t pass, or give students inflated grades because they endorse that teacher’s buzz-word learning outcome style of teaching. The starting point is that we have to hold ourselves and our colleagues to account as you suggest occurs at shimer. Yet many departments are full of wars, feuds and antagonisms and individuals who don’t understand each other, let alone each other’s work or teaching philosophy.

    The ideal is small departments with small classes where academics have the time to discuss with each other. The other is the collaborative approach practised in science where almost everything is team taught and so assessment is undertaken with those you teach with. Talking with colleagues in science they tend to have better-functioning departments and more uniform, ground-up assessment practices than tradiationally exist in the sole-taught courses of humanities.

Comments are closed.