A guest post by Conrad Perry:
It looks like the new Julia being the real Julia campaign has kicked off with a bit of good old fashioned teacher bashing. This reminds me of one of the things that seems really ingrained in many people’s minds, and an assumption which a lot of this teacher bashing is based on, which is that there is a horde of teachers out there who are bored, can’t teach well and are too lazy to get another job that they might actually enjoy. The solution to this is obviously to spend large amounts of money developing scales, standardized tests and IT tracking systems to try and identify them. This seems to appeal a lot to some people, although anyone actually caught up in this sort of system may well realize how useless these sorts of measures can be, and indeed how easy they are to cheat if you are cynical and really want to.
If you get away from the idea that performance should be measured on little scales that generally cheese teachers off and cost huge amounts to administer (it’s been tried on and off in different educational settings for at least 150 years, and there’s no evidence that it’s a particularly successful strategy), it is in fact quite possible to measure whether there really is a great horde of bad teachers out there damaging our education system – it’s been done, it’s been done multiple times and it’s been done using methods that are far better than those that keep legions of micro-managers employed in the education system. A very recent study done in Australia (NSW) and the US that shows just how it can be done well is by Bryan Byrne and colleagues, “Teacher effects” in early literacy development: Evidence from a study of twins (2010; Journal of Educational Psychology, 102, 1, 32-42; unfortunately pay-walled), and here is the abstract:
It is often assumed that differences in teacher characteristics are a major source of variability in children’s educational achievements. We examine this assumption for early literacy achievement by calculating the correlations between pairs of twin children who either shared or did not share a teacher in kindergarten, Grade 1, and Grade 2. Teacher effects— or, more strictly, classroom effects—would show up as higher correlations for same-class than for different-class twin pairs. Same-class correlations were generally higher than different-class correlations, though not significantly so on most occasions. On the basis of the results, we estimate that the maximum variance accounted for by being assigned to the same or different classrooms is 8%. This is an upper-bound figure for a teacher effect because factors other than teachers may contribute to variation attributable to classroom assignment. We discuss the limitations of the study and draw out some of its educational implications.
You might like to compare this with some political fodder, the quote of which I’ll shamelessly steal from Byrne et al. also:
Anyone with an ounce of brains knows what must be done . . . . It’s time to move from identifying failing schools to identifying failing teachers. (Alter, 2007, para. 1) Studies claim that 40 per cent of the variation in student performance is the result of teacher quality. (Patty, 2008, para. 8; quoting John Della Bosca, New South Wales, Australia, Minister for Education, in the context of an article about performance based pay for teachers)
The important sentence in the abstract of the Byrne et al. article is the third last one. Basically, what it’s telling you is that if you run a really nice clean study looking at the effect of teacher performance on early literacy development, of all the things that affect how much your child learns at school, teacher performance is likely to account for 8% of it, and that’s an upper bound – i.e., it’s probably less because that 8% includes things that even a really good experiment can’t remove, like the effect of teachers’ aids, the other kids in the class and so on (this is why Byrne et al. call what they find classroom effects, not teacher effects). Whilst this number is still bigger than zero (i.e., the teacher has no effect), it’s vastly smaller than what some of our politicians are claiming. The quote from Della Bosca, for example, suggests that of all of the things that affect how much your child learns, teacher performance is likely to account for 40% of it, or five times as much as what Byrne et al. are claiming. You can think about this in terms of which percentile a child would fall into on a test. If a child was ranked at the 50th percentile on a test (i.e., was better than or equal to 50% of the other kids) and had the worst possible teacher, then Byrne et al. would predict that if they had the best possible teacher instead, it would have moved them up the scale 8 percentage points, so they would instead have been ranked at the 58th percentile (i.e., better than or equal to 58% of the other kids). Alternatively, Della Bosca would predict that in the same situation, changing from the worst to the best possible teacher would have moved the child up the scale 40 percentage points rather than 8, so they instead would have been ranked at the 90th percentile (i.e., better than or equal to 90% of the other kids). This is obviously a huge difference in what is being claimed. If teachers really did make the type of difference that Della Bosca is claiming they do, there would be every reason to try and find those who were performing poorly, and it wouldn’t be very hard. The obvious reason that this hasn’t been done is that the data doesn’t support the claim of Della Bosca, and Byrne et al. cite a number of other reputable studies both in early literacy and other educational areas that find similar results to their own study.
So if teacher performance only accounts for 8% of what is learnt in the classroom (or at least what can be measured easily), it leads one to wonder what might account for the rest. There are many different factors that are likely to be important, such as the extent that parents help their kids, different genetic predispositions kids have to different types of tasks, what kids learn from other kids outside school and so on. Indeed, one might wonder here why you would even bother about getting good teachers if the effect they have on learning is not especially large – perhaps what this is telling us is that it doesn’t really matter if we have bad teachers. This is, however, the wrong way to look at the problem. A more likely interpretation of what is going on is that most teachers are doing a reasonable job of teaching your kids, and the fact that most teachers are doing a reasonable job means that going from the best to the worst isn’t going very far, and that other factors are therefore more important in terms of explaining the differences between kids that you can find. Hopefully this shouldn’t be such a surprise, because these days teachers have good textbooks, university degrees teaching them how to teach, generally use the same curriculum and some probably have supportive workplaces when they do have problems. If you’re not convinced of this, then try and think what you would need to believe if you want to hold on to the assumption that there are lots and lots of bad teachers around. If you believe this, then what the data is telling you is that teachers have only a very minor influence on the kids that they teach. If this is so, then why would you want to spend large amounts of money with all these measures trying to find the bad ones anyway?