John Hattie calls on teachers to ‘know thy impact‘. But what does this really mean?
Two of the most potent things you can do to improve student learning are to:
Or, in John Hattie’s words, to know thy impact.
When you know how far each of your students has progressed, you also know the impact your teaching has had.
How To Measure Impact
To know thy impact involves measuring progress, not A-E achievement. To do this, you need to use the basic pre-test/post-test format. You can use this format with any measurable assessment tasks. For example:
A student may:
The length of time between the pre and post assessments will vary with the nature of the learning.
How To Know If Thy Impact Is Enough
In the above examples, it is easy to see progress – the harder question to answer is whether that progress is enough. This is where the statistical measure of effect sizes is handy.
Effect sizes range from 0 to infinity, but most range between 0 and 1. After reviewing thousands of research articles, John Hattie found that the typical effect size of teaching is 0.4. That is, after teaching a unit on fractions, a pre-test/post-test should show progress of at least 0.4. If it doesn’t, the strategies being used by the teacher are not effective.
Effect sizes help you to know thy impact. Yet, there are limitations to using effect sizes, such as:
The next question is how do you work out effect sizes, especially when you are a busy teacher. There are statistical formulas you can use. Thankfully, there is also an Excel spreadsheet called the Progress vs Achievement tool that does it all for you. It only works though if you punch in data for a whole class or at least a group. Open it up and type in some dummy data to get the idea – note just enter their score, not their score out of (e.g. 8 not 8/12).
Here is a graph the tool produces with a set of sample students.
The amount of progress is shown from left to right, so:
- Megan made the least amount of progress
- Jennifer made the most amount of progress
The dotted red line is your 0.4 benchmark, so:
- Max and Pablo just made it over the line
- Julia got close but not enough
The vertical axis (y) shows the class average (0.62), so Julio got just over the class average.
The graph shows achievement from bottom to top so:
- Max achieved the lowest level, despite making adequate (0.4) progress
- Megan achieved the highest level, despite making the lowest amount of progress
From this simple graph you can tell:
- The class as a whole made above average progress (0.62)
- Julia needs a little more or perhaps different support
- Megan is coasting and needs to pushed and extended to show progress, not just achievement
- Robert and Rodriguez also need to be pushed but not as much as Megan
The spreadsheet tool also gives you exact effect sizes (progress) for each student.
Know Thy Impact & Adjust Teaching As Necessary
So, you now have data on your students’ progress. In other words, you know thy impact. What next?
What you do with your knowledge of your impact is what matters most.
If the effect size for the whole class is lower than 0.4 you need to think about why.
If you think it’s option 2, then explore options with a trusted colleague.
You then repeat the exercise for any individual students who have progressed less than 0.4.
Every time the effect is higher than 0.4 is a time for congratulations and celebration, as your approaches to teaching are working.