Know Thy Impact, Improve Your Teaching & Boost Student Learning

John Hattie calls on teachers to ‘know thy impact‘. But what does this really mean?

Two of the most potent things you can do to improve student learning are to:

• Monitor the progress of each of their students
• Adjust their teaching accordingly

Or, in John Hattie’s words, to know thy impact.

When you know how far each of your students has progressed, you also know the impact your teaching has had.

How To Measure Impact

To know thy impact involves measuring progress, not A-E achievement. To do this, you need to use the basic pre-test/post-test format. You can use this format with any measurable assessment tasks. For example:

• Early reader text levels
• Draft writing sample scored with a numerical rubric (see attached sample)
• Actual tests (teacher developed or commercial)

A student may:

• Score 4/12 on their first attempt at using persuasive devices in a text (see above sample) and then 7/12 second attempt
• Be reading a level 3 text in week one, and a level 7 text at the end of the term
• May get 24/50 on a pre-test on Viking history and 40/50 on a subsequent test

The length of time between the pre and post assessments will vary with the nature of the learning.

• With discrete units, you could do the pre-test at the start of the unit and the post-test at the end. Examples of discrete units would include ones on Vikings, fractions or persuasive devices.
• For broader learning outcomes (e.g. reading levels, spelling levels), you may decide that longer lengths of time are appropriate.

How To Know If Thy Impact Is Enough

In the above examples, it is easy to see progress – the harder question to answer is whether that progress is enough. This is where the statistical measure of effect sizes is handy.

Effect sizes range from 0 to infinity, but most range between 0 and 1. After reviewing thousands of research articles, John Hattie found that the typical effect size of teaching is 0.4. That is, after teaching a unit on fractions, a pre-test/post-test should show progress of at least 0.4. If it doesn’t, the strategies being used by the teacher are not effective.

Effect sizes help you to know thy impact. Yet, there are limitations to using effect sizes, such as:

• Small sample sizes. Results for a single class or a single student aren’t as reliable as results from a larger sample.
• The average of 0.4 is based on experiments that lasted different amounts of time, from weeks up to a year. However, 0.4 is the best we have to go on.

The next question is how do you work out effect sizes, especially when you are a busy teacher. There are statistical formulas you can use. Thankfully, there is also an Excel spreadsheet called the Progress vs Achievement tool that does it all for you. It only works though if you punch in data for a whole class or at least a group. Open it up and type in some dummy data to get the idea – note just enter their score, not their score out of (e.g. 8 not 8/12).

Here is a graph the tool produces with a set of sample students.

The amount of progress is shown from left to right, so:

• Megan made the least amount of progress
• Jennifer made the most amount of progress

The dotted red line is your 0.4 benchmark, so:

• Max and Pablo just made it over the line
• Julia got close but not enough

The vertical axis (y) shows the class average (0.62), so Julio got just over the class average.

The graph shows achievement from bottom to top so:

• Max achieved the lowest level, despite making adequate (0.4) progress
• Megan achieved the highest level, despite making the lowest amount of progress

From this simple graph you can tell:

• The class as a whole made above average progress (0.62)
• Julia needs a little more or perhaps different support
• Megan is coasting and needs to pushed and extended to show progress, not just achievement
• Robert and Rodriguez also need to be pushed but not as much as Megan

The spreadsheet tool also gives you exact effect sizes (progress) for each student.

Know Thy Impact & Adjust Teaching As Necessary

So, you now have data on your students’ progress. In other words, you know thy impact. What next?

What you do with your knowledge of your impact is what matters most.

If the effect size for the whole class is lower than 0.4 you need to think about why.

• Was the time between assessments too short?
• Is their scope for trying new approaches to teaching?

If you think it’s option 2, then explore options with a trusted colleague.

You then repeat the exercise for any individual students who have progressed less than 0.4.

Every time the effect is higher than 0.4 is a time for congratulations and celebration, as your approaches to teaching are working.