Deconstructing Hattie’s Effect Sizes

When taking the necessary in-depth look at Visible Learning with the eye of an expert, we find not a mighty castle but a fragile house of cards that quickly falls apart.

Source: HOW TO ENGAGE IN PSEUDOSCIENCE WITH REAL DATA: A CRITICISM OF JOHN HATTIE’S ARGUMENTS IN VISIBLE LEARNING FROM THE PERSPECTIVE OF A STATISTICIAN | Bergeron | McGill Journal of Education / Revue des sciences de l'éducation de McGill

Hattie's effect sizes are often thrown around as catch-all measurements of classroom methods. This reminds me of the learning styles discussions from several years ago. Both of these approaches have the same critical danger: reducing teaching and habits to single styles or single measures of effect is bad practice.

The idea of learning styles or effects on instruction are fine, but not when presented as scientific fact. A statistical breakdown of Hattie's effect sizes shows the clearly, as evidenced by this line:

Basically, Hattie computes averages that do not make any sense. A classic example of this type of average is: if my head is in the oven and my feet are in the freezer, on average, I’m comfortably warm.

Aggregating each category into a single effect size calculation disregards all of the other confounding variables present in a given population or individual. Learning styles has the same reductionist problem. In the mornings, reading works better for me. By the end of the day, I'm using YouTube tutorial videos for quick information. The style changes given the context and the idea of a single, best style ignores those context clues.

Use descriptors and measurements with care. Recognize the deficiencies and adjust for context as needed.

Google’s Narrow Focus on Digital Literacy

Google's Narrow Focus on Digital Literacy thumbnail
“We are helping them become more knowledgeable about what the digital landscape is like,” Mr. Hodgson said of his students, “so they can make choices about what they use and what they don’t use.”

Source: Google Is Teaching Children How to Act Online. Is It the Best Role Model? - The New York Times

The Times article on Google's "Be Internet Awesome" campaign does a great job of outlining why programs like this are needed, but not from companies who benefit from users within the program.

I tend to lean toward Kevin's (Mr. Hodgson) approach to teaching about the Internet: it's complex and you need to be aware of what you're using and why. The same is true for teachers. When I'm asked about apps or websites, I encourage them to read terms of service, privacy statements, and other conditions, especially if they want to use the thing with students.

Cutifying the practice of using the web wisely makes it exciting and easy to get into, but it's also easier to gloss over the deeper practices we should be teaching our students.

(Be sure to check out Kevin's blog for more of his writing.)


Conduits for Textbooks (and more!) flickr photo by Wesley Fryer shared under a Creative Commons (BY) license

Twitter’s Card Update is Bad For Users

Twitter's Card Update is Bad For Users thumbnail

Twitter made a UI change this week with it's card-styled link previews. It's subtle, but the changes have hurt accessibility and hurt the end user.

Twitter allows branded sites (anyone can do it, but brands use it the most) to create "cards" for links they post which include media previews, bold headlines, and a snippet of the article or post text. This gives the user a preview of what's in the link before they visit. The cards used to function like normal HTML anchor tags and included alternate text (the small popup when you hover your mouse) or showed the full URL of the link.

Since earlier this week, all of that is gone and it's led to some frustrating limitations.

First, headlines are cut off. And the alt text is gone from the mouse hover action.

screenshot missing hover text from Twitter card

A shortage of what? Cold drinks?

All browsers show URL paths in the bottom of the screen before you click on them. Twitter short links (t.co) would show first, but then resolve into the full path after a second or two. Not anymore.

A Twitter URL doesn't show the full path anymore.

Why is this less accessible?

The top-level domain shows, but that's all. Does this link take me to the registration page? Or to the article? Or neither?

Twitter cards no longer show the full link URL

Really, Twitter is making it harder to know what you click on, asking you, the user, to trust them, the company, to show us what we need to know. This update does nothing to help the usability on a platform that has done little to increase transparency and improve the quality of the platform for users.


Ripples of Glass B&W flickr photo by zeevveez shared under a Creative Commons (BY) license

Listening Has Changed My Coaching

Listening Has Changed My Coaching thumbnail

Aside from trying to be as productive as I can every day, listening - truly listening to people - has completely changed my coaching. Elena Agular talks about listening at length in The Art of Coaching (well worth the read, btw) and since I've made the conscious effort to listen first, I've seen fruits.

I stopped going in with preconceptions about progress or willingness to try new things. One unfortunate carryover from teaching is that I have expectations about specific teachers. So and so is hard to work with or this person will never change...I had shut down any possibility of working productively before I even walked in the room. Meeting people with the intent to listen rather than talk erased those expectations and allowed for positive conversations.

My ability to help people has increased. I don't limit this statement to technical help, which is certainly a component of my work these days. Asking questions and listening for context clues has allowed me to look beyond immediate problems and solve deeper issues, or at least identify issues to work toward solutions.

Summarizing the problem before offering solutions is critical. I stopped taking my computer to meetings with teachers because it leads to distractions. Or, if I do have it, I don't open it until we're working on a specific item. While we're talking, I have a notebook. I'm quietly making notes, looking for patterns and letting the teacher express their frustrations, ideas, or concerns without interrupting. I ask probing questions - "Why did you feel that way?" or, "What did [this thing] help you learn about your students' understandings?" - to draw out reflective thought. Before I start to talk, I take one minute to process my notes and state back, in my words, what they're experiencing. This catalyzes the rest of the conversation and helps us work together.

I can loop back to previous conversations and push toward growth. Since I have detailed notes (completely confidential notes) I can look back to previous meetings and probe next time we're together. Looping back to gently push toward growth on goals is easier because they're the teacher's own ideas. I'm there as a processing tool, not as the Owner Of Solutions.

In the end, I want to make sure I'm helping people in ways they want to be helped. I want to push them professionally by talking honestly about teaching - why we do what we do - to promote growth. If I can't understand what they're saying, it's a fool's errand.


Reflected Rocks flickr photo by Stanley Zimny (Thank You for 32 Million views) shared under a Creative Commons (BY-NC) license

The role of the faculty in the post-LMS world (opinion)

However, working outside the LMS, well-trained instructors will be able to do far more than meet the minimal requirements for moving college courses online...

Source: The role of the faculty in the post-LMS world (opinion)

I don't work at a university, but we're in the process of moving teachers into using Canvas in our district, so this resonates. I see two groups of people:

  • those who already had material online and are struggling to work backwards (essentially) to fit items into Canvas.
  • those who have nothing online and are struggling to make sense of what works well digitally and what doesn't.

The LMS is a weird stepping stone. I've had materials online for years, so I don't like the constriction an LMS brings to what I've done in the past, but I do appreciate the streamlined data I can grab from the system (I need to write more on using Outcomes in Canvas later...)

For the second group, it's a great intermediate step and I'm already seeing people look for more online on their own. They want to push the system now that they understand it more. They're seeing the benefit if using the Internet as a whole and not limiting their courses to the flow in Canvas.

Striking the balance between structure and variety is difficult. I'm not sure the LMS will ever completely go away, but I can see the influence waning as skills develop and alternatives becoming more accessible to teachers.

If We Don’t Allow A Redo, What Are We Teaching? – The Teacher And The Admins

Whatever the reason, he was afforded the opportunity to learn and apply. It hasn’t come easy, but that’s the point. Giving a chance to redo isn’t about being easy.

Source: If We Don’t Allow A Redo, What Are We Teaching? – The Teacher And The Admins

Still in my standards-based grading vein, this is inevitably the biggest sticking point for teachers I work with.

"What do you mean they can retake the test for full credit?"

Mentally, we can agree with the argument that redoing work or retaking tests makes sense in the scheme of student learning. The hold up, I find, is more with the work involved in making those opportunities reliable and valid more than the mental exercise of finding value in the habit.

There are ways to allow students to reassess work that does not include sitting an exam again, which opens more possibilities for authentic learning and demonstration of mastery.

Fair Isn’t Always Equal: Three Grading Practices to Avoid

Remember, though, that grades should not be used as rewards. Nor should they be used as affirmation, compensation, or validation. Grades should represent an honest report of evidence at this moment in time, nothing more. If we make them something more than that, we undermine the student’s maturation and any useful purpose for grading.

Source: Fair Isn't Always Equal: Three Grading Practices to Avoid

I'm working with several teachers on moving toward standards-based grading and we're starting to have conversations about grades themselves. I ask how they feel about zeroes, extra credit, completion, and makeup work. This article is a great primer and/or followup to those initial meetings.

Lesson Idea: Learning Objectives

Jessica Till asks a good question on Twitter:

and Graham's response stood out to me:

Turning this around (also mentioned by Will Dunn later in the thread), what would happen if we taught a lesson or went through an activity without positing the objective and then have students state the learning as an exit ticket or closing discussion. What insight could we glean?

Intentions are important, but implementation is harder.

H/T to Darren Burris and Dan Meyer for showing up in my timeline.

The Psychology of Classroom Discussions

I've never heard of the Asch Experiment, but this video is worth watching.

Kaplinsky's article gives some good classroom ideas on how to avoid groupthink in student responses, even highlighting the Desmos teacher view to anonymize responses (which also works outside of math.

During student teaching, questioning was the first thing my training teacher worked on with me. I was prone to asking, "Who knows..." which left the door open to zero responses. Moving to direct questions, "What is...?" or "How does...?" removes escape. I also recall another great post (I can't find it now) about leaving uncomfortable silence after asking a question. Letting several hands go up in the air allows for discussion as you can call on more than the first student.

Mixing Kaplinsky's ideas with leaving room for responses is a great way to help students feel comfortable with replying, even if they disagree with others.

Read the original article here.

Back to School

It's appropriate to write a back to school post nearly a month into the school year.

I took Twitter off my phone back in July in an effort to put my phone down more often. It had become too easy to lazily open and scroll when my interest waned. That bled into time with my kids, and that wasn't cool. I suppose I could have just "worked on my self control," but I'm more of a "pluck your eye out if it causes you to sin" kind of person. Cold turkey it was.

July bled into August which has raced into September. My phone is still Twitterless and I'm feeling more connected to my teachers and schools here. The social-edu world had eroded my sense of here-ness...I felt beckoned by people I've never met face to face. My desire to help and engage with the digital friends overrode the drive to get to know people here, now.

By extension, because I was online less, the blog has also cobwebbed up a little bit. I can't write volumes like Alan but I don't want to lose steam here entirely. Finding a balance has been hard. I need to write for me.

The evolving role of coach has pushed me to talk less and listen more. I think that's why I haven't really missed being online. Wanting to write, up until this point, feels like more of a drive to not waste the space than to share insight. Or maybe it's more that sharing insight shouldn't be the driving factor of having space on the Internet. Hubris kills just as quickly as stagnation.

Now that we're back, I'm planning on listening a lot. I'll probably do some processing here, in prose, just because it'll help me work through my own habits and thinking patterns. But I plan on listening more than talking.

Twitter still won't be on my phone.