Computer Science CS 337
Intro to Semantic Information Processing Fall 2002
Reciprocal Grading
 

I have been thinking about the grading process. Basically, it sucks. I hate it. It is very hard to give appropriate feedback by writing comments on papers. Of course, if I were to make less open ended assignments, it would be much easier to grade them because the comments could be short and to the point. Besides, who reads the comments I write anyway? But I really believe that open ended assignments are the way to go. Since you made it to Northwestern, you can probably follow instructions (if you want to), and testing that is pointless. Also assignments with a tight specification, although easy to grade, often involve tricking the students in some way. So I am not going to change the assignment type

I used to write more extensive comments on people's assignments, only to find out that they really care about the grade. And, why shouldn't they? It is the bottom line. But it is also very weakly correlated with learning. 

It would be good if students would reflect on their past work. There is copious educational research that indicates that this is very effective. The question is how to motivate you. So, here is an experiment:

Every one of you will soon receive two inferencer submissions by two different people. You will write some comments on that inferencer based on the guidelines. You will send your comments to the person or people who worked on that inferencer and to cs337. The subject should be "inferencer comments." You will also assign a grade to the person.

This grade does not count. It will not be recorded.

It is here because I think everyone should know that grades are stupid. So now you are on the other side. Enjoy.

Why should you do this? Because I say so. Ok, but why should you do it well? Because you get points.

After receiving your comments, the person will have to respond to you and to me with "inferencer comments evaluation". That person submit the paper version. She will get 5 points for submitting it, and a possible bonus.

You will get points according to what she said:

0 - inadequate                      

3 - adequate, but I did not learn anything from the comments.

10 - I learned quite a bit, here's what (2 paragraphs) (bonus 5 points)

20 - I learned a lot, it changed my prospective on things (4 paragraphs) (bonus 5 points)

 

 

Example:

John receives Mary's inferencer file to comment on.

John comments on Mary's inferencer. He uses the inferencer questions guidelines. He  wants Mary to think highly of his comments, so he is kind, but perceptive. Insulting Mary's intelligence is right out, and yet he has to find some good comment to make. This is especially hard because Mary's inferencer is actually better than his own. But, he manages. He writes 7 paragraphs and gives Mary a grade of 42/50 which he thinks is generous.

He sends it to Mary and cs337.

Mary comments on Johns comments. She carefully reviews the comments. She would like to give John a lot of points, but she has to actually figure out what it is that she learned from them. Also, she thinks that John is wrong on some points. But then she realizes that this too can be a learning experience. 

She says "I learned quite a bit" from the comments. She sends it to John and cs337.

She prints a copy of the original comments and her evaluation, and puts it in my box.

John gets 10 points, Mary gets 5+5= 10 points.

 

I will talk about this in class.


Comments? Send mail to cs337@cs.northwestern.edu