The release of the PISA testing data has engendered the usual media and political hysteria. I’m working on a comprehensive article about PISA and all its failings, which will be posted tomorrow. However there are some quick and probably not too well researched comments that need to be made about the results.
First, the question of New Zealand’s ranking on the PISA League tables.
Why has no one notice that in all the league tables, three of the ‘countries’ ranked above New Zealand are Shanghai-China, Hong Kong-China, and Macao-China. There’s no ranking for China as a whole country, and there’s no way that Hong Kong and Macao are representative of China as a whole. Maybe New Zealand should nominate certain cities to represent us in this competition? Also ranked above New Zealand are are Singapore, Chinese Taipei, Korea, and Japan. Why are these east Asian countries ranked so highly? That will discussed in tomorrow’s article. Another ‘country’ ranked above New Zealand is Lichenstein, a rather tiny country indeed. While I’ve not researched their demographics, I wouldn’t be surprised to discover that this is a very homogenous country with a relatively high level of income equality. I could be wrong though.
However that doesn’t explain New Zealand’s drop down the rankings in comparison to the 2009 results, and rather than going for doom and gloom headlines, we should undertake some indepth analysis to understand why this has happened. I’ve got my own pet theories but as I don’t have the evidence to back these up, I’ll won’t inflict them on you. However I would point out that we are still above the OECD average, while the heavily test based systems of England and United States are well below, while Australia is, on the average, just below New Zealand.
Another analysis, that the government will not be highlighting, shows that New Zealand is below the OECD average in equity in educational opportunities, and that this has deteriorated since 2009. That reflects the well established evidence that inequity is a major influence on children’s learning. Following this theme, these results reflect the trend from previous PISA tests – there’s a large gap between the haves and have nots. The bulk of the tested students performed way above average, but the tail end performed poorly. This trend, as in the past, reflects the disparity in socioeconomic backgrounds.
New Zealand is also well down the list in the allocation of educational resources. In other words, the government doesn’t adequately fund education yet expects miraculous results.
Similar issues apply to Reading, although this is obscured by the seemingly impossible task of developing reading tests for all the different languages involved. Removing the same East Asian countries as above, shows that New Zealand’s performance is better than the media would have us believe, and is towards the top of English speaking countries.
And so it goes. I’ve not discussed the Science results, as the lack of emphasis on Science education by successive governments going back a decade or so, explains any results. In fact I’m surprised we did reasonably well.
if you want to do your own analysis, have fun with this http://www.oecd.org/pisa/keyfindings/pisa-2012-results-volume-I.pdf
Tomorrow’s article will unpick the whole PISA testing programme and show why we must not take too much notice of many aspects. There’s more to PISA than we are told.
The lemmings headline word yesterday was “plummeted.”
The Minister of course would not use such emotive, and judgemental language. If she were the type to tell the truth, she would be saying, “What I can say to you, is that we have made great progress in our efforts to copy the best approaches of England and the USA which have seen our results rapidly moving towards theirs.” She definitely wouldn’t say “plunged!”
[…] to test the literacy abilities of so many nations who speak so many languages, or that fact that some countries may fudge their data by not releasing test results from poor performing areas, or that the hard-working statisticians […]
1) I would suspect that in the countries with “conforming” cultures, the kids all try their best in the tests; whereas the kids in “non-conforming” cultures don’t always try their best. It’s not like the tests are worth anything for the kids – neither the teachers nor parents nor kids get the marks back (although if you ask for them as a parent you should be able to get them back under our privacy laws).
I have some evidence on one side of the coin for this – a relation sat one of these tests and he said some of the kids who could have done really well just treated it like a joke.
2) My understanding is that NZ has introduced a novel statistics curriculum which other countries are watching to see how it goes (England, OZ). I assume that has meant taking out some pure maths content. As you would expect NZ did well in the stats section of the maths test but not so well in the pure maths section – but it’s hard to see that as a failing – we’re actually ahead of the game – it’s the testing reigime that the problem.
You could have a point there. I used to do any tests or exams as quick as humanly possible so I could sneak out for a smoke. I was often asked by teachers why I hadn’t stayed around to finish, to which I’d reply “Let’s wait and see my marks. I think I did enough to pass.”
Comments are closed.