Continuous Improvement

Considerations for Task-Based Rubric Development*

Getting Started: Determining the criteria that will be valued for a particular task.

• Brainstorm all the possible elements or criteria that could be assessed in the performance task.
• Determine which elements are "non-negotiable." Which criteria could be part of the task description as baseline requirements or provided as a checklist?
• Prioritize the elements that are left:
  

  What are the content and language goals of the unit? What do you really want the students to emphasize in their performance? How important is the overall "look" of the project (interest, appeal, creativity, neatness)? Is culture represented in the rubric, if applicable? Are the standards you targeted represented in the rubric?

• Determine 3 to 5 elements or criteria that will be incorporated in the rubric to define a quality performance. In trying to be thorough, an unwieldy rubric may be constructed, with so many elements being assessed that the rubric is time-consuming to fill out or oral performances will have to be taped (audio or video) in order to repeat them several times for the purposes of assessment.

Considering the Levels of the Rubric: Determining the number of levels and defining them.

• How many levels of performance do you wish to include in the rubric? How should they be defined? For example, "does not meet expectations," "meets expectations," "exceeds expectations." Or you may choose to use simply a 3, 4, or 5-point scale, noting, however, that a 3-point scale does not account for the fluctuation that exists within the average range. Some suggest that a 4-point scale is ideal, and that more than 4 points makes a scale cumbersome and difficult to use.
• Consider the elements or criteria you have chosen one at a time. Begin with the highest level of the scale to define top quality performance. This is the level that you want all students to achieve and it should be challenging. How would you describe a representation that exceeds expectations? meets expectations? does not meet expectations?
• Are the levels you have created parallel? That is, are the criteria present in all levels?
• Is there continuity in the difference between the criteria for exceeds vs. meets, and meets vs. does not meet expectations? The difference between a 2 and a 3 performance should not be more than the difference between a 3 and a 4 performance.
• Do the levels reflect variants in quality and not a shift in importance of the criteria?
• Is there an expectation of quality at the average (meets expectations) level of the scale?

Other issues to consider for rubric creation:

• Are the characteristics of each performance level described clearly? Will students be able to self-assess with the descriptors given? Will the descriptors give students enough information to know what they need to improve?
• Does the rubric adequately reflect the range of levels at which students may actually perform given tasks?
• Are the criteria at each level defined clearly enough to ensure that scoring is accurate, unbiased and consistent? Could several teachers use the rubric and score a student’s performance within the same range?
• Does the rubric attend to process as well as product?
• Are all criteria equally important, or does it make sense to weight an element more than the others?
• Are you attending carefully to the language used in the rubric? Use demonstrative verbs. Keep to observable behaviors. Avoid negatives ("begins without preparation" vs. "does not prepare"). Be specific. Instead of "many errors" you may want to specify "six or more errors." At the same time, be sure the rubric is generally qualitative in nature rather than quantitative.

Other issues to consider when using rubrics:

• Rubrics need to be piloted or field tested.
• Rubrics need to be discussed with students to create an understanding of expectations; you cannot write a paragraph defining each word in the rubric.
• Are the criteria at each level defined clearly enough to ensure that scoring is accurate, unbiased and consistent? Could several teachers use the rubric and score a student’s performance within the same range?
• There is a fine balance between modeling excellent work and creating a "template" that is replicated by the students ad nauseum to the detriment of creativity.
• "If a student can achieve a high score on all the criteria and still not perform well at the task, you have the wrong criteria" (Wiggins, cited in Clementi, 1999).
• Consider whether a rubric needs revision for a specific task. Do some of the criteria on the rubric go beyond this particular performance (that is, if you’ve created a rubric that is more "generic" and can be used for many tasks over time)?
• Make sure that the expectations in the rubric are directly aligned with the instruction of the lesson/unit. Students shouldn’t be expected to do what they haven’t been previously taught or shown.
• Some suggest that generic rubrics are more useful because creating rubrics is time-consuming and the more often they can be applied, the better. It is also more informative for students if the same rubric is used again and again, because they can see themselves making progress over time. On the other hand, generic rubrics are much less tied to the task and are not able to provide criteria for specific language use expectations or content knowledge.