LSE's evidence-based research into health services has shown how and when ambitious targets and tough sanctions can improve performance of public service

What was the issue?

In the late 1990s, long hospital waiting times were symptomatic of a crisis in the quality of health services throughout the United Kingdom.

From 2000, the government sought to transform the quality of health services in England by substantial annual increases in NHS funding. The devolved governments of Scotland, Wales and Northern Ireland followed suit.

A second government policy – adopted only in England – also scrapped the old governance model that rewarded failure with extra resources. They replaced it with a regime of annual 'star ratings' for hospitals and ambulance services based on ambitious targets, with tough sanctions for failure. From 2000 to 2005, failing authorities could be 'named and shamed' and their Chief Executives sacked under an approach dubbed 'targets and terror'.

This 'natural experiment' provided a unique opportunity to compare the effects of two systems of governance for performance in relation to targets: where failure resulted in sanctions in England and was perceived to result in rewards in Wales and the other devolved countries.

What did we do?

When the government in England implemented these changes, LSE Professor Gwyn Bevan was on secondment as a Director at the Commission for Health Improvement (CHI), a new body set up to regulate NHS quality in England and Wales. Bevan led the Commission's efforts to contribute to the development of ‘star ratings’ for England. Through his secondment, Bevan was perfectly positioned to appreciate the ‘natural experiment’ developing between England and Wales.

The results of that ‘natural experiment’ were clear-cut. By 2002, hospital waiting times had decreased in England but increased in Wales. Work by colleagues at CHI, however, suggested that English hospitals and ambulance services had engaged in 'gaming' to improve their ratings.

On his return to LSE in 2003, Professor Bevan continued to investigate comparative NHS performance of hospitals and ambulance services across all four health services and between England and Wales, and he further explored the effect of 'naming and shaming' policies.

Crucially, since funding levels and performance goals remained broadly similar, Bevan's research could relate outcomes directly to different models of governance, making his policy conclusions genuinely evidence-based.

His research showed that linking high-powered incentives to targets will result in improvements in reported performance and also in attempts to game the system, such as ‘hitting the target and missing the point’. This highlighted two strategic lessons for policymakers on when such an approach will and will not result in improved performance outcomes for public services. First, targets with sanctions can work when the targets accurately measure key dimensions of performance that cannot easily be manipulated by gaming. Examples include hospital waiting times, ambulance response times and school league tables.

Second, where targets can only be framed as proxies for important dimensions of performance, the manipulative effects of gaming may undermine their use. An example is policing, where the nature of the targets means that pressure to meet them can result in such dysfunctional behaviour that performance deteriorates.

What happened?

Professor Bevan's research has significantly influenced policy and practice across health services, and helped to define the terms of the debate about performance targets for public services.

Within the UK, BBC Radio 4 devoted an Analysis programme to Bevan's research. Bevan also co-presented a paper on models of governance for health services and schools in England and Wales to a House of Lords seminar on benchmarking public services.

The British Medical Journal invited Bevan to participate in a debate about targets for the English NHS and also asked him to review the evidence that competition between hospitals improves clinical quality.

Internationally, Bevan featured in a World Health Organization report on assessing the performance of health systems and contributed to World Bank seminars on measuring performance in developing countries.

The many references to Bevan's research by other respected research bodies and leading national journalists show how its influence is striking home within the field of health and reaching into other areas of public life. For example, Tim Harford in the Financial Times devoted an article to Bevan’s typology of models of governance.

Simon Guilfoyle, a serving police inspector, used Bevan's findings on gaming to help frame his own account of improving performance within his police force, while a team of researchers from Bristol used the 'natural experiment' to investigate English and Welsh school examination performance at age 16. Their research concluded that devolution had produced worse grades in Wales, which did not publish school league tables, suggesting ‘naming and shaming’ as an effective strategy for improving school performance.

The real test of policy research lies in the changes it provokes. The regime of 'star ratings' undoubtedly reduced waiting times in England, where they continued to fall more sharply than in Scotland, Wales or Northern Ireland.

The most recent study evaluating the UK’s ‘natural experiment’ between the four national health systems reported improved performance in the devolved countries with reductions in hospital waiting times and quicker ambulance response times which has followed the improvements in England.  The most striking finding is the way in which Scotland has transformed its performance to match that of England by strengthening accountability against targets. The coalition government also reaffirmed its commitment to waiting time targets for the NHS in England after initially appearing to abandon them.

Search all impact case studies.