Rats in the New Zealand Education Cupboard

In a previous blog posting I outlined that the Ministry of Education seems to have been manipulating the way that two standardised tests, STAR (Supplementary Test of Achievement in Reading) and e-asTTle (electronic – assessment Tools for Teaching and Learning), have been used to rank children’s performance. This will inevitably lead to increased ‘achievement’ against national standards, and which will be cited as proof of how wonderful the National led government has been in ‘raising achievement’ after ten years of failure under Labour.

Ho, ho, it seems that a nerve has been struck somewhere in Wellington. Schools using e-asTTle have now received a letter, accompanied by a website posting,
instructing them to downgrade the first term results, which, in some cases, has resulted in some children dropping two levels.

That rat I mentioned last time is smelling even more strongly. This now throws the whole e-asTTle testing programme into doubt. How can any results be claimed as having validity? Which is the better result? The initial one, or the regraded one, or neither?

The Ministry of Education presently has, on its collective desks, all schools’ National Standards reports for 2012, most of which will have used e-asTTle and STAR data to help inform teachers’ overall judgements (OTJ) about children’s ‘achievement against standards.’ National standards were dubious enough before, now this takes the farce to a new level. Garbage in, garbage out.

Nationwide data will be released, with percentages above, at, and below standards, at the end of May. This will be compared to 2011 results to see if student achievement is rising towards the arbitrary 85% ‘at or above’ standards target. Given that the standards were set at the approximately 65% against age appropriate norms (i.e being an ‘average’ achiever is below standard) this is a tall order.

At the end of June, the Ministry will release school specific data by year level, gender and ethnicity and we can expect the media to rework this into league tables of school performance, regardless of then Minister of Education Anne Tolley’s broken promise from 2009 that this would not be permitted. The fact is that league tables are an essential part of the privatisation of New Zealand education through ‘proving’ many schools are failing and that reform is justified – a classic use of disaster capitalism.

National Standards data is suspect enough as it is, without the extra issues that will flow from the flawed data from the standardised tests, and yet this is about to be dumped on the public accompanied by wondrous noises from the Beehive. Kelvin Smythe sums this up well:

‘The minister will make much of OTJ and trusting teachers and triangulation and other fluff, but she has also made much in the past of standardised tests, especially e-asTTle. It is mentioned, for instance, in that national standards legislation. E-asTTle is the only writing standardised test, and given its widespread use, the gross inflationary nature of the results must have had a significant effect on national standards.’

Turd polishing, anyone?

But… what about the 2013 National Standards results, now that the standardised test lark has been exposed? Will these now be lower than 2012, which will then show that national standards have failed? Never fear, this will be ‘sorted’ in time to prevent the government being embarrassed – so much for the pretence of ‘quality data on achievement.’

The National led government’s education policies have very little to do with education and much to do with their political agenda (e.g the mixed ownership model of schooling).

Hiding in the primary education cupboard, peeking out now and then, is another rat.

What is this rat? Is it a threat to educational health and wellbeing?

This rat is called PaCT (Progress and Consistency Tool).

‘The Progress and Consistency Tool (PaCT) is intended to support teachers’ overall judgments and assist with increasing the consistency of judgments across the country and across time periods. The tool is based on the same principles as the National Standards themselves, requiring a variety of evidence of learner achievement as the basis for overall judgments, and will enhance the ability to track learners’ progress in relation to the National Standards.’ Source.

Judgements, by definition, are alway subjective, yet PaCT aims to turn these into objective statements for ranking and comparative purposes. Square pegs and round holes?

PaCT will be an online system that will collect teachers’ judgements for each child against set criteria (known as rubrics) and then use psychometric analysis (Rasch modelling) to produce rankings against relevant national standards. Whether these numerical rankings actually count as education is an extremely contentious matter – a subject for future articles.

The Ministry of Education Business Plan, of November 2012, shows that the Mathematical part of PaCT will be online in July and that the Reading and Writing sections will be ready to go for the 2014 school year. The information provided to principals so far is very generalised and gives very little indication of how PaCT will work.

To be fair, PaCT will not be compulsory at this stage, and the Ministry are very vague about whether it will be made compulsory in the future. However, given the Ministry of Education’s desired aims as set out in their business plan,

‘The investment involves all state and private schools serving approximately 500,000 students in years 1 to 8. About 2,000 schools and 30,000 teachers are affected’

it will be made compulsory for state schools in the future.

Why, then, is the Ministry being vague about future intentions? If it is going to be compulsory, which seems obvious, why is this being hidden?

So that’s the basic PaCT. What kind of rat is it?

Consider:

PaCT will collect achievement information about every child in participating primary and intermediate schools, obtaining their names, schools, teachers and classrooms through a direct connection to the Ministry of Education’s ENROL database that is used to track every child’s school enrolment and other details throughout their school life.

PaCT will add to this data collection, so that every child’s educational progress, defined by their ranking against national standards, will be stored throughout all their school life.

I need to outline a few key concerns here. The first of these, acknowledged by the Ministry of Education in its business plan, is that PaCT, as designed, is a ‘world first initiative and unproven.’

The PaCT process of taking teacher judgements and running them through psychometric analysis processes to obtain national standards rankings, is untried and untested anywhere in the world, yet is expected to be ready to go starting from July this year. What are the chances that this will work as intended? Can we be sure that the results will be valid and reliable, especially since these will be kept and analysed for an unspecified period?

Remember, the Ministry of Education established the specifications for Novopay…

Are we looking at another waste of tax payer dollars?

Another issue is the whole notion of education based around standards and raising achievement, as the way to a ‘brighter future’;

‘High levels of achievement in reading, writing and mathematics means a more productive educated workforce and leads to increased international productivity. It also enhances international image.’

This claim, from the business plan, is similar to those made by all the other global education reform movement countries (GERM) yet goes against extensive educational research and expert opinion. In fact it is an economists’ theory of education, driven by the OECD, reinforced by a New Zealand Treasury Report to the Minister of Finance, (November 22, 2011 – the week before the general election), entitled ‘ Implementing Changes in schooling – key levers over the next three years’, and other financial sector organisations and businesses, unsupported by quality, objective research evidence.

Indeed, this claim is directly contradicted by experts such as Sir Ken Robinson (his 2006 TED talk is the most watched TED talk ever – if you’ve not watched this, your life is incomplete) and USA educator Yong Zhao:
“..what brings great test scores may hamper entrepreneurial qualities. Standardized testing and a focus on rote memorization, for example, are perhaps the biggest enemies of entrepreneurial capability.” Source.

As for enhancing international image, is this why New Zealand children go to school? New Zealand education has been held in high regard for many decades, and a number of highly regarded US educators, who have come here in the past to learn from us, have communicated their disbelief to me about New Zealand’s joining of the GERM brigade.

Against the very suspect PISA testing, New Zealand is at the top of the rankings, as I showed in this previous posting. Finally, the country most highly regarded educationally is Finland, who have developed their education system 180 degrees to the GERM ideologies. Something is very awry here.

So having discussed these key concerns, let’s return to how PaCT will affect schools, teachers, children and their parents.

PaCT will also connect with each school’s School Management System (approved databases that hold wide ranges of children’s data, including school based assessment results, attendance registers, medical, behavioural records, family issues and so on). Over the eight years of primary schooling, this combination of PaCT, ENROL and school based databases will aggregate to quite a substantial quantity of data on each child.

PaCT data will be made available to the Education Review Office, as it ‘provides clear information about student progress and achievement which can be used to identify areas that need improvement.’ This rings very loud alarm bells. Schools’ experiences with ERO over the whole period of their existence shows that this organisation ends up dictating how government policies are implemented through holding schools accountable to their own specifications.

This strongly suggest that ERO, who are already forming judgements of schools based on national standards targets, will take this to another level, through the interpretation of PaCT results as their primary accountability tool. Indeed, this possibility was also raised in the Treasury Report, p18.

This leads to only one outcome. Schools will be forced to ensure that their PaCT results are as high as possible so that ERO reports favourably on them, and that this will inevitably lead to an intense focus on the 3Rs of reading, ‘riting, and ‘rithmetic, to the detriment of all other curriculum areas. This is a situation well known in countries with a nationalised standardised testing regime, particularly England and USA, where art and physical education are being neglected, in order to achieve high scores and thus favourable league table rankings.

Is this what New Zealand parents want for their children?

What happens to the data when a child leaves school? Good question.

Who owns this data? Another good question.

Parents will be able to request copies of their child’s data, but will not have direct access to the database – this will be limited to authorised users, such as principals and teachers. Given that parents will not have any rights to request that their children’s achievement data not be uploaded to the PaCT database it follows that parents have no ownership. It is pertinent to note here that overseas parents have the right to opt children out of standardised testing but this right is denied to New Zealand parents.

As the data for each child is independent of their school (it will follow children as they move schools) then schools do not own this either.

As one of the aims of national standards is to help the Ministry identify schools who are less effective in raising achievement, analysing PaCT data will help provide this information. Indeed the business plan states that PaCT will enable ‘improved target setting for school charters and the monitoring of improvements against targets’ and this suggests that the Ministry will own and control the data and its usage. Are we, as a country, comfortable with this?

The issue of data ownership now brings overseas developments into play. In the United States, a student database called inBloom has been developed by the usual players such as Bill Gates, News Corp, Scholastic, Promethean and so on – full list here. Concerns have been raised, for example here, here, and here, about inBloom’s intention to sell data to private companies such as McGraw Hill and Pearson Group, to help them develop materials to sell to schools. Other databases are also in development in other US states, and concerns about the ‘Orwellian’ nature of these are discussed here.

While PaCT differs in its methodology, it is similar in storing children’s personal and achievement data and using this for other purposes, and therefore the ‘Orwellian’ aspect, in the New Zealand context, must be considered as well.

At this point McKinsey & Co. reappear on the scene (I introduced them in this previous article) with an article entitled ‘Big data: The next frontier for innovation, competition, and productivity.’ :

‘..big data can unlock significant value by making information transparent and usable at much higher frequency. Second, as organizations create and store more transactional data in digital form, they can collect more accurate and detailed performance information on everything from product inventories to sick days, and therefore expose variability and boost performance.’

Not rocket science to see where PaCT is heading, wouldn’t you agree? Even bigger rockets can be easily constructed by looking at other government databases (e.g health and social welfare) and linking these to PaCT to provide even bigger data sources. Will PaCT data be sold to private companies, as is happening with inBloom and the other US databases?

Connecting the dots … Since 1990, Learning Media (a SOE) has had the exclusive contract to develop curriculum materials (especially the School Journals that you will all remember) for the Ministry of Education. This exclusivity has now been revoked. Learning Media now has to compete with private companies for curriculum development contracts and is now listed as a struggling SOE, ripe for being sold off to the highest bidder.

Deliberate sabotage? Will PaCT and other national standards data be sold to competing bidders for curriculum development contracts? What do you think?

The more extensively PaCT data is shared, the greater the chances of privacy breaches, and while it is theoretical possible to construct a very secure system, recent experiences would cast doubts on the government’s abilities to ensure privacy. Similar concerns are raised in the US article (above) about national databases. Is this a risk New Zealand parents are prepared to take?

The effect on teacher workload is an unknown. How long will it take for a teacher to collect and enter judgements against specified criteria (rubrics) in each of the three subjects? The present plan is for this to be done twice a year, but I’ve also been made aware that this could be increased over time to maybe once per term, and in fact, for PaCT to achieve its goals, more frequent inputting of data is almost essential.

Underpinning this is a huge assumption that teachers will have the computer aptitude and time to enter the data quickly and efficiently.

Hiding away here, denied by the Ministry in their business plan but highlighted in the November 2011 Treasury document, p8, is the possible use of PaCT data as a tool to measure teacher effectiveness (known in the USA as VAM – Value Added Measurement).

‘Creation of value add data to support school-level accountability changes in the future and to underpin a more objective and data-driven approach to teacher appraisal.’

PaCT could easily be adapted to monitor the achievement of all children in one class over a year, thus provided a tool to judge teacher’s effectiveness in raising achievement.

The issues around VAM are many and varied, regardless of superficial appeal to renowned experts such as Bill Gates; however the immediate concern would be that teachers would be forced to focus all their energies on raising achievement against standards, to the detriment of all other curriculum areas and the myriad of other tasks that constitute a teacher’s workload.

Again, what do New Zealand parents want for their children? Do they want their children educated to work or educated to live a full and rich life?

Note also that the Ministry is putting a great effort into improving the measurement of achievement, and neglecting the other side of the equation – promoting excellent holistic school educational programmes. There’s a well used cliche in education – ‘repeatedly weighing the pig doesn’t make it any fatter.’

Over the past couple of decades of both National and Labour led governments we’ve seen the destruction of the school advisory system that provided rich school and teacher professional development across the whole curriculum, to the present extremely narrow focus on ‘raising achievement.’ I’d suggest that much more would be gained by reinstating a high quality advisory service.

While all these discussions are very much worst case scenarios, the risk factor is very high and the issues raised here:

dubious goals
suspect educational ideologies and methodologies
data gathering and security
use of data
value added measurement
impact on teachers’ time
neglect of the rest of the curriculum

all lead to the conclusion that PaCT is indeed an unpleasant rat.

All schools, principals, teachers and parents should be very wary about catching educational rabies from its bite. Changing the government in 2104 will be the only cure.