Peer Reviews
Today I was hosted by ISOM to talk about they ways in which I have used the "Peer Review" feature of TurnItIn. This is a brief summary of my talk.
I've been using the Peer Review feature of TurnItIn for the past two years in my stage III course Business Policy & Strategy. The approach I have used consists of students submitting their assignments to TurnItIn, reviewing three of their peers' assignments (and in turn having their own assignment reviewed three times), and then revising their assignment (hopefully based on the feedback they have received).
The process takes a number of weeks. Usually, I make the assignment due in about week eight. I then allow the students one week to do three peer reviews. I then give them a further two weeks to revise their assignment. This means the final version of the assignment is handed in during week eleven -- giving me just enough time to mark it before any final examination. When the timing of other assignments are taken into account, scheduling of peer reviews is not a trivial matter.
TurnItIn allows the feedback to be structured in two ways. Firstly, the students can answer questions about the assignment they are reviewing; for example, "In what way could the introduction to this essay be improved?" Secondly, Likert like scales can be used; e.g. "On a scale of 1 to 5 where 1 is not effective at all, and 5 is very effective, how effective was the essay's introduction?" Finally, the student can be permitted to allocate a grade to the essay.
Overall, I have been very impressed at the quality of the feedback given by the students. In generally, the feedback provided, if used by the student, would lift the assignment by at least a whole grade (e.g. from a C to a B). Of course, there are some students who don't give such high quality feedback. However, I hope that the combination of each student receiving three sets of feedback and having seen three other essays (have reviewed them themselves) should compensate for the occasional weak reviewer.
There are three ways by which TurnItIn allows reviewers to be allocated. I’ve tended to use random allocation for all three reviewers. However, it is possible to allow people to choose who they review, or for the instructor to explicitly allocate reviewers. TurnItIn is flexible enough to allow any combination of these approaches. For example, it is possible to allocate three reviewers and allow the student to pick a third essay to review them self. However, I like to keep the reviewers (and "reviewees") anonymous as I think that leads to more open and frank feedback.
In my experience, the single largest concern students have about the peer review process is "What if someone steals my good idea?" I’ve not seen any evidence of this happening (and I have looked for it), but this observation does little to quell the students’ concerns. So, I tend to fall back on the policy for plagiarism as a deterrent (especially as TurnItIn should catch any replication).
Despite that concern, I think that the students like peer review. Why they like it is a little less clear to me. Is it because they get two bites of the cherry (and are able to check out if their assignment is on the 'right track')? Is it because they get more feedback about their assignment than I can reasonably give? Is it because the process of peer review gives them more time to think about their assignment? I don't know. Perhaps some students will comment on this.
How do I encourage students to actively participate? How do I stop them from handing in trivial "drafts" or failing to properly review others' work? My starting point is that I assume the student will "do the right thing" They will hand in a completed piece of work and they will be diligent in doing the reviews. Consequently, I don't give them marks for either of these parts of the class. Rather, I punish those who don’t "do the right thing"--by penalising them up to 30% of the value of final assignment. This is part of my view that often we reward (bribe) students with marks to motivate them for things they should do as a matter of course. i.e. there is no substantive pedagogical reason for some of the marks I've seen given to students (and I've done that as well).
There are a number of advantages to peer feedback. Firstly, and perhaps most importantly, it gives students a lot more feedback than they would normally receive. I accept that some (small portion) of the feedback may be of dubious quality. But I think the students can recognise that.