BY KEVIN HODGSON

I’m not sure what to think of this new site called Paper Rater, which is a free web-based service designed to be a tool for writing. It’s an automated tool which analyzes a piece of writing along a set of criteria, including possible plagiarism, grammar and spelling, vocabulary choices, style and more. The site is pretty easy to use. You paste your text into the box, click the button and wait for a second or two before Paper Rater spits out its interpretation of your writing. As an experiment, I had Paper Rater examine a feature article that I did for LEARN NC on teachers as writers.  I came away from the experience feeling a bit like someone had pulled out their red pen and scribbled along every margin of my paper. The report tells me that I likely plagiarized content (I didn’t — honest), that my vocabulary was less than average (ouch, that hurts), and my use of transitional phrases was severely lacking (double ouch).

The section that breaks down my use of words and sentence in the article, in part, reads:

  • 2380 words, average length 4.70 characters = 1.44 syllables
  • 95 sentences, average length 25.1 words
  • 50% (48) short sentences (at most 20 words)
  • 27% (26) long sentences (at least 35 words)
  • 1 paragraphs, average length 95.0 sentences
  • 1% (1) questions
  • 62% (59) passive sentences
  • I don’t think it’s a stretch to say that I lost sight of what I wrote by digging deep into these kinds of numbers. It’s an odd experience to have something you wrote reduced to some data analysis like this.

    In the Classroom

    I worry about the shift towards automated technology that is creeping into writing centers and high school classrooms as a (cost-effective) replacement for real teachers working with writers one-on-one. I can’t say the information I received from Paper Rater isn’t useful. I went back to look at my piece, particularly along the lines of the feedback about passive voice in my writing. It’s possible that Paper Rater is valuable as one step in the line of the process — maybe during the drafting or revision stage of writing. One area where I give Paper Rater credit is that it doesn’t just give criticism, it also provides information and examples of exemplary work around a concept of analysis.

    For example, under the Transitional Words section, Paper Rater explains:

    One sign of an excellent writer is the use of transitional phrases. Transitional words and phrases (e.g. however, or, finally, before ) contribute to the unity (cohesiveness) of a text. Indeed, without these words or phrases, a text will most likely seem disorganized and will often be difficult to understand. When used they provide greater organization within a text and lead to greater understanding and enjoyment on the part of the reader.

    Another thing to note: it’s not exactly clear who is running this site. The site notes that it is maintained by linguistic professionals and graduate students, but these folks are not identified. I wondered if the writing submitted is being used in some research, although there is no indication of that anywhere on the site, including the Terms of Use that a user must read and approve before submitting a piece of writing.

    Paper Rater

    The Past, Present, and Future of Automated Essay Scoring by Brigette Knudson

    Automated Scoring Versus Human Scoring: A Correlation Study by Jinhao Wang and Michelle Stallone Brown

    (Sidenote: I put the finished version of this article into Paper Rater as a sort of a meta-reading experiment, and it informed me that I was clear on plagiarism and had done well with my vocabularly. Phew.)