Transparency, Teaching, and Taking My Evaluations Public

August 4th, 2009 § 4 comments

I recently wrote about why I’m making even the earliest scraps of my research public. It’s a move, in theory, that most academics would not object to. Nobody is going to give me funny looks for suggesting we share our research problems. After all, scholarly collaboration is something we’re almost all willing to profess a belief in.

So here’s something that may send a few strange looks my way. In addition to my research, I believe the other half of my job — teaching undergraduate and graduate students — should be as public as possible. Even if I weren’t an employee of the Commonwealth of Virginia, working in a publicly funded state university, I would still argue that virtually all aspects of my job — what I earn, what I teach, what my students think about my teaching — should be transparent.

One of these areas — what I teach — has long been public, as all my syllabi, reading lists, and assignments are online. In the latest version of my videogame studies course I even used the class wiki to document and explain any changes I made to the syllabus during the semester.

RateMyProfessor.com tells us what a few self-selected students think about a professor, not what they think about a professor’s teaching.

Finding out how effective a teacher I am proves to be more difficult. Many professors and most students know about the informal ratings out there. MTV’s RateMyProfessor.com (I bet you didn’t know MTV was so dedicated to pedagogy) is the most popular site, but there are others. More often than not, though, these ratings are based upon a professor’s charisma or workload, rather than any kind of systematic statistical data. (Is a chili pepper statistically significant?) These sites tell us what a few self-selected students think about a professor, not what they think about a professor’s teaching.

My university’s own course evaluation system — salmon-colored forms students fill out anonymously at the end of every semester — is rigorous, qualitative, archived, and — happily for many faculty — almost completely invisible. I get these evaluations back, of course, and I have to share them with my salary and reappointment committees. But after that? In theory, George Mason makes the numerical score sheet for each set of teaching evaluations available to a wider audience.

In theory.

Good luck going to the university website and finding out information about the score sheets. Tracking down these evaluations reminds me of the scene early in The Hitchhiker’s Guide to the Galaxy, when Arthur Dent discovers the plans to build a highway bypass through his property. Dent eventually uncovers the designs on public “display” in the cellar of the local planning office, “in the bottom of a locked filing cabinet stuck in a disused lavatory with a sign on the door saying ‘Beware of the Leopard.’”

Here is George Mason’s own equivalent of the “Beware of Leopard” sign:

Off Campus Ratings Accessibility Message

The above message is what any off-campus visitor sees when he or she attempts to access the database of teaching evaluations. On campus, the wiew [sic] isn’t much better:

GMU's TypoTo be fair, I’m hoping that the typo has been corrected since I captured this screen shot in May. But I wouldn’t know for sure. You see, it’s August and I’m off-campus right now, as are most faculty and students, and I can’t even electronically access my own teaching evaluations, let alone those of other professors, unless I’m physically there.

In short, my teaching evaluations are all but hidden to the world. Off campus they are firewalled. On campus, you might be able to find them, but only if you know where to look (and have a Mason ID and password). And once you get past those hurdles, the university only provides the numerical scores — not the written comments students may have left.

So I’m moving beyond my professions of faith in scholarly transparency into clear, deliberate action. And this is where I start getting funny looks, if not totally horrified ones. I’m releasing all of my teaching evaluations, complete with every single enthusiastic or blistering or apathetic student comment, to the public under a Creative Commons Attribution-Noncommercial-Share Alike license.

I’ve begun with the most recent set of evaluations I have, from Fall 2008, and as soon as I have the Spring 2009 batch, I will upload those as well. And I’ll begin working my way backwards in time, adding teaching evaluations from every semester I’ve been at George Mason University. You’ll find the evaluations online at Scribd (“The YouTube for Documents”), but since they are embeddable, I’ll post them here as well.

Below are the evaluations for ENGL 343, a new media class dear to me but which encounters resistance from students who discount electronic literature. By the end of the semester I have many stragglers, evidenced in the thirteen students missing from class the day I distributed the evaluations.

[scribd id=14082871 key=key-4usnb5zi533ttsszn7a]

And here are evaluations for ENGL 414, a small seminar for exemplary undergraduate majors that focused on American Postmodernism.

[scribd id=18105809 key=key-1v0xyubce27t7fphkerc]

I’ve discussed the lofty minded “why” I’m doing this, and I want to end with the more practical “what” — What can someone do with these evaluations?

There are obvious answers: prospective students may find them valuable, other teachers of similar material might learn what works and what doesn’t, and my own colleagues may gain a better sense of what goes on in my classroom. But I’m interested in the less obvious answers. For instance, I can use the evaluations as the basis for a teaching portfolio, in which I perform my own reflective analysis of the students’ feedback. Or, more experimentally, because the evaluations are under a Share Alike license, they can be remixed. I have no idea what remixed teaching evaluations might look like, but I would love to see what someone comes up with.

Such transparency can be intimidating at first, as I am surrendering control over what many professors dread reading themselves when the forms are returned in their sealed envelopes weeks after classes are over. But it is also liberating. Both the public and myself can only gain from the availability of my teaching evaluations. Think of it as open source teaching.

§ 4 Responses to Transparency, Teaching, and Taking My Evaluations Public"

  • Jared C. says:

    This is pretty stellar stuff. And, yes, it is impossible to find those evaluations online. I have tried before…

  • [...] he is not only forward thinking about making his research open, Mark Sample yesterday wrote about making teaching evaluations public. Why–he reasons–allow a self-selected group of student to provide public feedback on a [...]

  • AramZS says:

    This is very admirable. I agree, Mason’s system of ratings lacks the transparency and accessibility to be used in any meaningful way by students. RateMyProfessor can be good for finding only the extremely good and bad professors, and sometimes not event that.  On the remix front, I’d love to see some more graphic visual representations of the statistics, which is really what Mason should be doing to begin with.
    I hope many more Mason professors follow your example!

  • [...] openness, there has been some discussion of open teaching evaluatoins at Prof Hacker and Sample Reality, and another interesting post by Mark Sample at Sample Reality about sharing research sources via [...]