Columns and Colors

I’ve taken a big dive into test analysis lately. Spreadsheets, formulas, correlations, and color-coding have become the norm after an exam to help and identify misconceptions with my students as a whole and on a class-by-class basis.

One of the sheets I created (based on work done by Andy Schwen a few years back) color codes student responses in red if they got the question wrong, like this:

2015-11-24_6-35-27

I started wondering if there were significant patterns between classes looking at the scored matrix rather than statistical figures. I grabbed screenshots of each class, dropped the opacity of the image to ~35% and then layered them all to make this:

item_analysis_color

What made this doubly-interesting is that the darker red areas on the page do not always line up with the statistical figures. For instance, I needed to revisit 2, 4, 9, 11, and 12 based on the statistical analysis (biserial value, if you’re curious). They had discrepancies between students who got it right and did well on the test when compared to those who got it right and did poorly.

Looking at the picture, I also went back and revisited 8, 9, and 17 just because a lot of people missed them. I also looked at number 5 because very few people got it wrong.

Teaching is science and art. Even if the art is doing test analysis.

Language

The language I use around grading is very specific, especially when I’m speaking with students.

“Your grade is based on how much you learn. You must show me that you’ve learned.”

I also have a go-to response when students ask if a particular paper will be graded:

“The paper itself isn’t important. However, the work you do on the paper can demonstrate learning, which is how you’re graded. Second, and more importantly, mistakes you make on paper help me help you.”

Students usually agree with this sentiment. Until grades come around.

“I did all the papers! Why is my grade so low?”

“You haven’t demonstrated learning yet. What would you like to show me?”

“…but I did all the papers.”

The language is the same, but we’re speaking past one another. Until we can make clear the distinction between the process of learning and the demonstration (which can certainly happen in the process), we’ll continue to fight culture.

Close Reading – Day 1

We’re working on our close-reading skills. Indiana has changed the standardized test format (again) and students will be doing very Common Core-like reading and analysis tasks as part of the new format. (Sidebar: it’s amazing that critical thinking and analysis are terrible under Common Core, but totally okay and great when Indiana copies them.)

This means I need to take time in class to actually teach some serious reading strategies. I’d done similar things implicitly in the past, but it’s now more important than ever to make students are equipped to be successful on the exams coming in the spring.

This activity is the first time students have gone through the process, so it’s taking serious time. The school improvement team has built and tested a model that we’re to use in class with our students so each class gives an equivalent experience.

Today, I grabbed an article from the textbook on Robert Goddard. We’ve just finished Newton’s laws of motion and talking about spaceflight and the apparent paradox of the third law is a good way to round out the chapter. The text is on grade-level, so students should be able to approach it on relatively sure footing.

To start, I read the article out loud while they underlined anything of interest. Our students have struggled with writing coherent responses to prompts because they lacked a basis for their response. Finding points of interest can help build a frame of mind to respond to a question later. Listening to me read modeled the pace they should use when they’re reading alone. We talked about phrasing and hearing their own voice in their heads. Most seemed to see the importance of this because they actually had time to process what they were reading as it was being read. We also discussed points of interest to help those who struggled find a good starting point.

I also found at this point it was important to acknowledge that the interesting point did not (and really shouldn’t) be a direct quote. It can be an idea or a theme. It can be a frame of reference – putting themselves into the shoes of the subject. It helped some who didn’t find anything in particular at the surface interesting to find a point they could dig into later.

Second, they re-read the article silently (modeling appropriate pace) while circling words that they didn’t know or found confusing. I described this as, “figuring out what you don’t know so you can go learn it.” Not knowing something is too distracting. I wanted to embrace the fact that they wouldn’t know some of the terms and that it’s okay.

After finding their words, they grouped up and compared. Words in common were circled and left. Words that were unique (one person, maybe) were peer-taught.

Finally, we regrouped and started going around the room defining terms in common. Before class, I had gone through and identified potential terms, but there were a lot thrown out that I hadn’t anticipated. Doing this as a class promoted idea sharing and helped students to see that they were not alone in confusion.

Newton’s Laws Posters

I don’t know what day of class this is, but here’s what we did today.


We took a test a week ago that did not go well, to say the least. I already wrote about that process. We took this week to slowly go back over ideas and rehash them…not from scratch, but pretty thoroughly. I’ve been trying to be more explicit in the methods of interaction – reading, writing, kinesthetic (lab), drawing, etc. Today was the final day of Newton’s Laws (#thesecondone) and I decided to have them create posters and/or short comics on one of the laws.

These are some of my favorite student pieces:

2015-11-05 11.42.09

2015-11-05 11.42.36

2015-11-05 14.51.14

Yes, this is a brash Sir Isaac Newton breaking a tree with the apple.

2015-11-05 14.51.32

Are they perfect representations? No. And there are others that show inconsistency. But, they were talking about the ideas and creating their own models based on understanding. Drawing (sometimes silly) pictures helps to internalize the idea and it opens doors for discussion. It allowed me to enter a conversation about physics at their level and do some intense teaching one on one, which was the goal.

I also challenged myself to creating some stories. I’m a major fan of xkcd, you’ll notice the resemblance because that’s my level of artistic ability:

It was a fun, light day with lots of growth. Tomorrow, we’ll quiz one more time before moving on to work and energy next week.

Cultures

I gave a test a week or so ago. When I scored the test the first time, I didn’t think I made my key correctly. So, I double checked it and re-scored. Same result.

I immediately implicated myself in the low scores. My test may have been too hard. Perhaps I included things that weren’t taught. I went back and went over the entire test, item by item, and matched each to a set of notes, a lab, and the state standard I marked on my plans.

I taught everything on the test in at least three different modes. I did my job.

So, I turned to statistics to help me out. I ran item analysis on each question, both using Biserial values and Chronbach’s alpha test. It got nerdy fast.

With the exception of a few questions with low Biserial values (which I’d already identified as problematic, this just confirms) the test is reliable. All of the questions were within the accepted range of variance.

This confirms that the test was, in fact, valid and provided reliable results.

I can’t force students to engage. I can provide opportunities to engage and I can prod them that way, but at 15-18 years old, they need to make their own decision. Some classes have made that decision.

Two classes in particular are beginning to thrive. Two don’t know what they want, and two are continuing to really, really frustrating years.

I think it all comes down to the fact that students need to realize that everything they do as an individual affects every other person. That’s what a community is. We’re all affected, including me.

How do you teach that?