GUEST BLOG: Ian Powell – Data cleansing of unmet patient need hides a national health scandal


In the mid to late 2000s, while Executive Director of the Association of Salaried Medical Specialists, I used the term ‘data cleansing’ to describe a policy under the then Labour-led government when Pete Hodgson was health minister.

When general practitioners considered that their patients required further investigations by hospital specialists, they would refer them to the relevant district health boards (DHB).

These further investigations are called ‘first specialist assessments’ (FSA’s). Those patients who could be assessed were then placed on a waitlist for their treatment such as surgery. The prescribed maximum waiting time was four months.

The issue was the growing number of patients requiring further investigation who were not placed on the waitlist. Instead, due to a mix of hospital underfunding and understaffing, they were referred back to their GPs from whence they had come.

Inevitably patients were placed under stress as a result of this continued uncertainty about their health and when their necessary further investigations would be undertaken.

- Sponsor Promotion -

Unsurprisingly, their worried GPs were also stressed. Few things are more worrying for a GP than knowing that their patients are fretting and at increased risk of harm or death because further investigations into their conditions have been denied.

Cleansing data

Unfortunately there was a conscious decision not to report (and most likely record) data on the number of patients who were referred but denied access to FSAs. Data reporting was confined to just those patients who accessed them.

This is what I called data cleansing. The official data looked good; the real situation for many was dire; for some fatally so.

Strictly speaking this was not technically accurate. In the IT world, data cleansing is the process of finding and removing errors, inconsistencies, duplications, and missing entries from data to increase data consistency and quality.

However, levering off the expression ‘ethnic cleansing’, I adapted the term accordingly. The bad news is that this practice has gotten worse as the crisis in Aotearoa New Zealand’s health system further deteriorates.

The relatively good news is that “groundbreaking” academic research has made this transparent. In the words of the researchers it reveals a “national scandal”.

Revealing a national health scandal

Undertaken by the University of Otago, in conjunction with General Practice New Zealand (GPNZ), the research was released on 4 June: Impact of unmet patient need on general practices.

Professor Robin Gauld co-authored national scandal revealing research study

It is positive to see this sort of collaboration between university academia (Professor Robin Gauld, Dr Jerram Bateman and Dr Nick Bowden) and a major general practice body such as GPNZ, which represents the primary care organisations.

They are to be congratulated for doing what the Ministry of Health and Health New Zealand (Te Whatu Ora) have failed to take responsibility for; the duplicity of data cleansing.

GPNZ released its own statement on the research two days later: Research reveals hidden pressures on general practices.

The same day Radio New Zealand’s health correspondent  Rowan Quinn also covered the findings: Over 85,000 GP referred patients annually prevented from seeing hospital specialists for necessary investigations.

Drilling down to the study’s findings

The study was based on both quantitative and qualitative analysis leading it to conclude that unmet health needs are jeopardising patient healthcare and straining general practices.

It drilled down further to conclude that:

  • access to hospital specialist services was declining;
  • there were equity and regional disparities;
  • patients were experiencing delayed treatments, poorer health outcomes, and exacerbating health inequities;
  • already stretched GPs were being overburdened; and
  • there were negative financial implications for both patients and general practices.

GPNZ Chair Dr Bryan Betty: more and more New Zealanders unable to access specialist care

In the words of Porirua GP and Chair of GPNZ, Dr Bryan Betty:

The study shows more and more New Zealanders are unable to access specialist care in a timely way, leaving GPs to manage these complex cases without extra resources. This not only affects patients, who suffer from delayed treatments, but also puts immense pressure on GPs, making it harder for them to provide quality care.

Some of these at least 85,000 referred patients a year are being turned away from seeing a hospital specialist; worse still some of them are dying as a result.

In the words of Professor Gauld it was “appallingly bad” for patients who could be in pain or disabled because their condition could not be treated. “We have thousands and thousands of people around the country not living life to their full potential.”

Time to end this health scandal

Phil Bagshaw: tireless campaigner for investigating full extent of unmet patient need

Retired Christchurch general surgeon Phil Bagshaw has been campaigning for years for a full investigation into the level of unmet health need in New Zealand but has been batted off by successive governments.

This research is not the full investigation he and his colleagues wanted. But it is an important step in the right direction and provides some badly needed transparency. It has indeed exposed a national health scandal. This work needs to continue.

More importantly, the political and bureaucratic leadership of our health system need to focus as a matter of exigency on addressing the causes of this scandal beginning with across-the-board workforce shortages.


Ian Powell was Executive Director of the Association of Salaried Medical Specialists, the professional union representing senior doctors and dentists in New Zealand, for over 30 years, until December 2019. He is now a health systems, labour market, and political commentator living in the small river estuary community of Otaihanga (the place by the tide). First published at Otaihanga Second Opinion


  1. This is one of the results of increasing the poulation by 20 percent in 10 years through over immigration. Has the capacity of the health service increased by 20% in the last 10 years?

    The nation needs an immigration moratorium rahui for 10 years. A catch up period to get health housing education back to a world class standard.

    • not to mention refusal to meet pay rises and consistantly cutting funding…hollowing a service out won’t get us world class health…with or without fact there is an arguement that without immigrant nurses etc our hospitals would shut down joe

      and yes I agree end low skill/family migration

  2. Part of the problem is a lot of these medical people only work part time .I can not get an appointment for weeks because my GP only works 2 days a week at most .And no she is not nearing retirement age .Then too many so called consultants are mostly only interested in trolling hospitals for patients who are willing to pay stupid money for treatment even though they may not be as ill as others who are shunted to make room for the payers .

  3. Our gratitude to these people. I am Speaking out as one of these people who languished at home for over 2 years and sure I was dying. Two doctors being unable to diagnose my pain etc. and be turned down 3 times for a specialist appointment, to personally plea for an appointment then no diagnosis, then turned down again by the heart specialist, go to Thames Hospital myself only to be turned away by an obnoxious American Dr telling me this was an ongoing condition he can’t help me, how dare I go without referral. i then finally collapsed with heart failure. I have been through a horrific experience, pushed to get the treatment I need (bi-ventricular pacemaker on Monday) and all up now fear our health system is third world. The system needs the input of a compulsory ‘public health scheme insurance’ to top up what we have.

  4. The issue is we do not have enough junior doctors been trained by our universities in decades.

    The shortage of doctors means we overpay existing registrars to provide same capacity, when we could pay more doctors with the same money to provide more care. Same applies to senior doctors.

    • because training costs money nobody wants to pay it has been cheaper in the past to poach staff….save a quid today forget it’ll cost you 100 down the road…look at my current balance sheet in wonder

  5. in my experience – I have been referred to specialists for no apparent reason. seems there is a good old-boys network amongst medical professionals that “play” the referral game and I’m guessing they select on medical insurance grounds to assuage their guilt. If we are going to do immigration – lets do it right. Health professionals at the top of a very short list.


Please enter your comment!
Please enter your name here