Wednesday, April 1, 2015

Ditching "Effectiveness" - the Miracle of Page 18

This past weekend, I was fortunate to be included with a group of amazing Physics teachers from across Ontario as we began to re-examine the purpose of rich tasks in the senior physics eLearning courses. We met in Toronto for a weekend of curriculum work and a new look at assessment.

99% of the teachers I might tell this to would roll their eyes at the thought of spending a whole weekend on assessment. It has a bit of a reputation for being dry, boring and anything but inspirational. Anyone simply looking at pictures of us at work might come to the same conclusion...


Action shots from the weekend!

Assessment PD is not known for its excitement factor,
but it was VERY engaging and extremely worthwhile.
However, it was during our work on how to create assessments that I've had one of the bigger light bulb moments of my teaching career.

Lori Stryker, from the Ministry (who was amazing), led us through the creation of rich tasks on the first day, with the expectation that once learned, we would repeat the process on our own the following day. Briefly, here is the procedure we followed:


Creating a Rich Task

Starting with an overall expectation from the provincial curriculum, we "unpacked" what was expected of the students. What knowledge do they need to have in order to complete the expectation? What skills do the students need?
Unpacking the curriculum expectations

From these lists, we created learning goals for each of the knowledge pieces and skills (each beginning with "We are learning to..."), and then from those, we created success criteria (each beginning with "I know I can succeed, because I can..."). 

From these success criteria, we turned to the curriculum documents to figure out the assessment of the task (keeping in mind that we don't actually have the task in place yet!). We placed the success criteria within one of the four categories of assessment (Knowledge/Understanding, Thinking & Investigation, Communication, Application), and narrowed the focus down to one of the Ministry's subcategories. 

Here is the subcategory - under the assessment category of Thinking & Investigation - that our group chose for the success criteria of "I know I can succeed, because I can compare and contrast the environmental impacts of different energy transformation technologies," and the qualifiers for levels 1, 2, 3 (provincial standard, highlighted in green) and 4:



Normally, when making a rubric, what you see above is almost exactly what I use -  I cut and paste the ministry's wording right into the rubric I give the students. But I've never liked the word "effectiveness." How do you judge that? How is effectiveness even perceived from one person to another? Which aspect(s) of the student's work fall under being effective? It seems such a vague word, but since that's what the Ministry gave us, I just took it and used it. I know many Ontario educators feel the same way.

Game Changer

Here's where our minds were blown. Lori pointed us to page 18 of Growing Success, which states:

"What constitutes effectiveness in any given performance task will vary with the particular criterion being considered. Assessment of effectiveness may therefore focus on a quality such as appropriateness, clarity, accuracy, precision, logic, relevance, significance, fluency, flexibility, depth, or breadth, as appropriate for the particular criterion."

In other words... Teachers: Replace the word "effectiveness" in your rubric with one of the qualities suggested. Such a tiny little suggestion, but look what happens when the word is changed:

Success criteria: I know I can succeed, because I can compare and contrast the environmental impacts of different energy transformation technologies

Qualifier: To achieve a level 4, the student uses critical/creative thinking processes, skills and strategies with a high degree of breadth.

We (students, teachers, parents) now immediately know what we are looking for - has the student covered a wide variety of perspectives?

Change it again:

Success criteria: I know I can succeed, because I can compare and contrast the environmental impacts of different energy transformation technologies

Qualifier: To achieve a level 4, the student uses critical/creative thinking processes, skills and strategies with a high degree of fluency.

Now we are looking at the student's ability to communicate their process. Same task, COMPLETELY different way of assessing.
Of course, I had to share on Twitter... 
We were floored. This completely changes how we look at rubric design, task creation, and assessment as a whole. I am loving the fact that I can improve on the rubrics I've made up until now (but at the same time cringing at the number of rubrics I need to go back and tweak!).

After more practice, I'm looking forward to the rich task creation process becoming more streamlined, and seeing how this little change of ditching "effectiveness" impacts my students and my assessment of their work.

8 comments:

  1. I'm so glad everyone enjoyed the session! Love the term 'game-changer'!

    One further thing to consider... transform your success criteria into the criteria that you will assess and evaluate on your assessment tool:
    Example: Student can analyse the impact of environmental technologies with a high degree of relevance:) Replace the 'critical thinking' with the learning performance as you have it articulated through the success criteria, and you have easily and efficiently built your assessment tool!

    ReplyDelete
    Replies
    1. Wow! Thank you, Lori! I love how you push us to refine our work. It's only been a few days, but in conversations with my team, we've already found ourselves staring at our expectations and asking "What would Lori say?" :) Thanks again for an excellent session last weekend.

      Delete
  2. Heather I also had the opportunity to work with Lori. She is amazing and she pushes your thinking. The rubric should reflect the criteria that unpacks the overall expectations.

    ReplyDelete
    Replies
    1. I completely agree! It's nice to be reminded of that, too. I was surprised by how one's choice of words in the learning goals compared to the success criteria (as opposed to just restating the same thing) can hone what we expect from the students.

      Delete
  3. I am going to agree that this is game-changing for rubric writers everywhere! Thank you for sharing, Heather. (And, for the record, I don't think it is weird to spend a whole weekend on assessment...look what came out if it!)

    ReplyDelete
    Replies
    1. Thanks, Amy! Even weeks after going through this process, I still get a little giddy when I get to the point of making a rubric and I get to CHOOSE the word with which to replace "effectiveness!!" :D

      Delete
  4. Thank you for sharing! Glad to see Lori is getting the kudos she deserves! Such a pleasure to work with her as she challenges us to refine our thinking around assessment practice.

    ReplyDelete
    Replies
    1. Her enthusiasm was contagious at the workshop, and she continues to inspire us to really question how we assess our students!

      Delete