Published: 2019-08-20 02:43 |
Category: Grading | Tags: canvas, canvas lms, design, standards based grading
Every year, I'm building more support for standards based grading within our district. Though Canvas isn't really set up for SBG, there is a way to make it work, and it works well. In short, every assignment, whether it's a test/quiz, classwork, or even a conversation with students, can be linked to a standard which is aggregated and reported over time.
Current status of Instructure's outcomes roadmap.
With some of the recent Canvas updates, this system is eroding and Instructure is missing out on a huge opportunity to truly change the way we approach grading. This post is a breakdown of the changes and what could be done to fix them.
Reporting Outcomes
When Outcomes are attached to assignments, reporting is...okay. The Learning Mastery Gradebook view (which is toggled in the Course settings under Feature Options) is a helpful color-coded chart of any assessed outcome. Hovering the Outcome column title shows a context card with class averages and the outcome detail. But that's it.
If you switch to the Individual View in the gradebook, you can click a student name and then use the Learning Mastery tab to see all aligned Outcomes and their scores. This view gets closer to being helpful because it shows scores on assessments over time, which allows you to track progress (growth vs decline) in a chart.
To see reports organized by Outcome, you can go to Outcomes and then click on the title of the individual item. This shows the assignments it was assessed on and a list of students who were assessed. This list is not sortable and can be many, many pages long.
This is just to demonstrate that there is no consistency on where to find the information, which makes Outcomes less compelling to use.
Quizzes.Next and Outcomes
Canvas is building out a new Quiz engine called Quizzes.Next (though it will soon become 'Quizzes' and the old style will be 'Classic Quizzing'). While there are certainly some functional improvements, Outcomes are being left behind.
One benefit of Quizzes.Next is that Outcomes can be added to individual questions. This could be done in Classic Quizzing by using Question Banks, but the banks were never really exposed to teachers and alignment is a bit of a chore to find. I have more detail on our instructional tech blog if you want to see the process.
The Problems
There are three main issues:
- Quizzes.Next takes 24 hours to show course outcomes. I don't know why this limitation is in place, but I'm assuming it's because Quizzes.Next is built as a standalone LTI app and the data sync is metered. This hinders quick creation of reassessment.
- Question banks are now linked to the user, rather than the course. Individual teachers now maintain their banks rather than being attached to the course. In theory, teachers can share these banks like they would other items, but it adds a layer in the import process.
- This is the biggest: outcomes scores in Quizzes.Next do not report to the learning mastery gradebook. The mastery gradebook is the aggregate of all outcomes over time. Not reporting these assessments adds a step for teachers to have to process them in the course as a whole.
Removing the step of creating course-level question banks and moving it to the quiz creation step arguably makes outcome alignment more accessible. But, given that they do not report back erodes the value in aligning outcomes at all.
Solutions
While I'm not going to hold my breath, there are a few ways Canvas could make Outcomes more usable as they move forward with some of their development strategies.
- Make Outcome results easier to find. There is no reason they need to be buried. I submitted an idea with screenshots on how to display student results in the current structure...it's simply a link in the Outcome context card to take you to alignments. Canvas hasn't made any moves to make this easier.
- Keep Question Banks aligned to the course, not the user. I teach several courses and my banks are structured as such. When I'm building a quiz, I want my banks to stay in context. Searching through all of my question banks makes the process harder. Limiting banks to specific courses helps with organization and reporting.
- Push Quizzes.Next Outcome results to the Learning Mastery Gradebook. I'm still confused as to why this is a feature request. It is feature parity. If Canvas is going to remove functionality in a systems update, it makes the system less usable for instructors. The workaround, currently, would be to manually add a rubric to the quiz and then score it for each student based on the Quizzes.Next Outcome report. Workarounds are not solutions to problems.
If Canvas takes reporting seriously, they'll understand that it goes beyond assignment averages and test scores. Outcomes and their attachment to items in Canvas can really open doors to more accurate and equitable reporting. Unfortunately, most of the decisions coming from leadership are making this harder instead of easier.