Spit and Tape to Get Some Data

We're using Canvas LMS in our district. There are some things I appreciate and things I appreciate...less. One of which is when a main feature doesn't work.

Luckily, the Canvas API gives me some leeway in fixing things that Instructure doesn't seem know is broken.

I'm supporting teachers using Outcomes in their courses. Assignments can be tagged with relevant standards and recorded in a nice color-coded chart over time. I came across a big problem where students couldn't see their aggregated scores. There's no point in giving rubric feedback if students can't see those scores.

I thought it might be a fluke, so I tried reaching the student results through the API. It works as a teacher, but not as a student. There's a pipe broken somewhere on the Internet.

[File large bug report, crack knuckles.]

The approach

The easy fix would have been to hit the Canvas API to get this fixed, but student accounts came back with no data. So, I needed an intermediate, server-side account to do the work.

Earlier this semester, I set up an internal web app to manage reconciling grades. This was built in Flask so it was easy to add an endpoint as an API. I've been really digging the canvasapi Python package from UCF because it's so versatile and makes working with objects so easily.

So, I ended with:

  • JS client listents for the user to click on a tab in the gradebook.
  • Hit the Flask endpoint to start the server call.
  • Return the requested objects.
  • Process the objects in the client and append to the page.

The code

Here's the commented code for a simple implementation using Flask and some Javascript.

Canvas LMS is Missing a Big Opportunity with Outcomes

Cobblestone walk in France

Every year, I'm building more support for standards based grading within our district. Though Canvas isn't really set up for SBG, there is a way to make it work, and it works well. In short, every assignment, whether it's a test/quiz, classwork, or even a conversation with students, can be linked to a standard which is aggregated and reported over time.

Current status of Instructure's outcomes roadmap.

With some of the recent Canvas updates, this system is eroding and Instructure is missing out on a huge opportunity to truly change the way we approach grading. This post is a breakdown of the changes and what could be done to fix them.

Reporting Outcomes

When Outcomes are attached to assignments, reporting is...okay. The Learning Mastery Gradebook view (which is toggled in the Course settings under Feature Options) is a helpful color-coded chart of any assessed outcome. Hovering the Outcome column title shows a context card with class averages and the outcome detail. But that's it.

If you switch to the Individual View in the gradebook, you can click a student name and then use the Learning Mastery tab to see all aligned Outcomes and their scores. This view gets closer to being helpful because it shows scores on assessments over time, which allows you to track progress (growth vs decline) in a chart.

To see reports organized by Outcome, you can go to Outcomes and then click on the title of the individual item. This shows the assignments it was assessed on and a list of students who were assessed. This list is not sortable and can be many, many pages long.

This is just to demonstrate that there is no consistency on where to find the information, which makes Outcomes less compelling to use.

Quizzes.Next and Outcomes

Canvas is building out a new Quiz engine called Quizzes.Next (though it will soon become 'Quizzes' and the old style will be 'Classic Quizzing'). While there are certainly some functional improvements, Outcomes are being left behind.

One benefit of Quizzes.Next is that Outcomes can be added to individual questions. This could be done in Classic Quizzing by using Question Banks, but the banks were never really exposed to teachers and alignment is a bit of a chore to find. I have more detail on our instructional tech blog if you want to see the process.

The Problems

There are three main issues:

  1. Quizzes.Next takes 24 hours to show course outcomes. I don't know why this limitation is in place, but I'm assuming it's because Quizzes.Next is built as a standalone LTI app and the data sync is metered. This hinders quick creation of reassessment.
  2. Question banks are now linked to the user, rather than the course. Individual teachers now maintain their banks rather than being attached to the course. In theory, teachers can share these banks like they would other items, but it adds a layer in the import process.
  3. This is the biggest: outcomes scores in Quizzes.Next do not report to the learning mastery gradebook. The mastery gradebook is the aggregate of all outcomes over time. Not reporting these assessments adds a step for teachers to have to process them in the course as a whole.

Removing the step of creating course-level question banks and moving it to the quiz creation step arguably makes outcome alignment more accessible. But, given that they do not report back erodes the value in aligning outcomes at all.

Solutions

While I'm not going to hold my breath, there are a few ways Canvas could make Outcomes more usable as they move forward with some of their development strategies.

  1. Make Outcome results easier to find. There is no reason they need to be buried. I submitted an idea with screenshots on how to display student results in the current structure...it's simply a link in the Outcome context card to take you to alignments. Canvas hasn't made any moves to make this easier.
  2. Keep Question Banks aligned to the course, not the user. I teach several courses and my banks are structured as such. When I'm building a quiz, I want my banks to stay in context. Searching through all of my question banks makes the process harder. Limiting banks to specific courses helps with organization and reporting.
  3. Push Quizzes.Next Outcome results to the Learning Mastery Gradebook. I'm still confused as to why this is a feature request. It is feature parity. If Canvas is going to remove functionality in a systems update, it makes the system less usable for instructors. The workaround, currently, would be to manually add a rubric to the quiz and then score it for each student based on the Quizzes.Next Outcome report. Workarounds are not solutions to problems.

If Canvas takes reporting seriously, they'll understand that it goes beyond assignment averages and test scores. Outcomes and their attachment to items in Canvas can really open doors to more accurate and equitable reporting. Unfortunately, most of the decisions coming from leadership are making this harder instead of easier.

Adding Assignment Statuses to the Canvas SpeedGrader

Here's another little script I hammered out for Canvas today.

With the new gradebook, you can set assignment statuses like "late" and "missing." This is helpful in the gradebook for on paper assignments (digital assignments are automatically flagged) but you can only change the status in the gradebook grid.

This is a hacked together script to add the same buttons to the SpeedGrader controls.

The easiest way to add this is by adding an extension called Tampermonky. This essentially allows you to run code on websites you don't have access to edit.

After installing the extension, click here to install the script.

Last step: click on the Tampermonkey Icon, choose Dashboard, and then click on SpeedGrader Status. In the editor, update line 14 with your Canvas URL.