Over on the University of Huddersfield iPark blog, I’ve posted a summary of Dr Mike Reddy’s recent webinar for Plagiarismadvice.org, principally on redesigning assessments to reduce plagiarism but actually about good assessment design in general.
Category Archives: Turnitin
This post was edited on 01 Feb 2013 to add an additional point
I was interested to see the stats which Turnitin have released showing the sources which students’ work is most frequently matched against. In particular, they highlight that Wikipedia is a source which students’ work is frequently matched against. But is it really that clear cut?
The full paper, which is available on request, doesn’t say much about the methodology of the research, which leaves several questions unanswered.
One potential methodological criticism of this study is that it relies on Turnitin matches. More able students are likely to be paraphrasing the sources they have consulted rather than quoting verbatim – although this practice varies between disciplines – and such paraphrasing would not show up in a Turnitin originality report. Arguably, therefore, this study only reflects the sources that weaker students are using in their work.
A further criticism is that the paper does not make it clear whether they are looking at just the match which is initially shown when you open the originality report or every match that it finds.
Consider this example from an originality report, where the highlighted source is Wikipedia. Drilling down into the Match Breakdown section, you see that there are four other sources that have an equal match of 5%. Clicking them in turn highlights the exact same phrases in the student’s paper as were matched with Wikipedia.
So it seems that Turnitin may have a bias towards Wikipedia, listing it first amongst potential sources where there are multiple identical matches for the same content. (This is perhaps because of the nature of a wiki and the redirections on the site that mean each page has several URLs so it is listed multiple times, as in this example.)
Given that anybody can edit Wikipedia, it is perfectly plausible that the Wikipedia article has quoted (or plagiarised) the same source that student has used. It is also possible that students are citing Wikipedia content without knowing it, where it has been reused on other sites which might look more credible (which Wikipedia encourage by publishing under a Creative Commons license).
So while Turnitin’s advice in the full paper that students be guided to follow the links from Wikipedia to the original source is perfectly sound, there is every chance that it could still show up as a match against Wikipedia in the originality report, and thus in any future updates to these statistics.
For some time now, I’ve felt that the originality score in Turnitin is at best unhelpful, and at worst quite a dangerous statistic in the hands of people who don’t understand how to interpret it.
It’s a very blunt instrument, and there are plenty of tales of teachers setting an arbitrary cut off point beyond which they will reject work as being plagiarised, and below which they will consider it acceptable. This fails to take into account the difference between one large chunk of unoriginal work that has been copied and pasted verbatim and a collection of short attributed quotations which between them take the work over the arbitrary cut off point. Of course the latter may not be considered good academic practice, but that is an academic skills issue that can be addressed in the grade given and the feedback and feed forward rather than treating it as plagiarism, which it isn’t.
I have a search stream for turnitin on Twitter set up (in Hootsuite) and some of the comments from staff and students alike are eye opening, not to say eye watering.
- “It makes grading so easy. 56 percent on Turnitin. Big F. Moving on to next paper.”
- “On our course, we’re only allowed to submit coursework that has a Turnitin score less than 15% so try rewording some stuff.”
- “No #turnitin.com .. I’m pretty sure I didn’t plagiarize my own name”
- “Turnitin.com <<<<< I had 27% plagiarism"
- “Turnitin saying that 3% of my essay was plagiarized”
- “turnitin.com said that paper was 39% plagarized and i still got a high B? Looks like i’m changing my last name to Shakespeare”
These comments (which I have deliberately not attributed in fairness to the originators) demonstrate both teaching staff who are apparently using arbitrary cut off points rather than considering each originality report in depth and students who have been given access to the originality report apparently without them understanding the difference between originality and plagiarism and the limitations of Turnitin.
In an ideal world, the solution would be that staff and students receive appropriate training and instruction in how to interpret the originality report, but it’s the kind of thing that doesn’t have a natural home in the curriculum – particularly where Turnitin is introduced for continuing students who may not have the formal Study Skills modules that are common in the first year.
In my view, the percentage (and associated colour coding) serves no useful purpose – particularly with the advent of GradeMark and the ability to see text flagged as unoriginal highlighted in the background whilst marking – and tempts people to infer more from the headline percentage than is appropriate. Turnitin would be greatly improved by removing the percentage and colour coding.
I should say for clarity that I have no objection to showing the percentage match for specific sources within the main report view.