Why performance indicators fail

PIs fail because they succeed! They are designed to separate a normal from an abnormal state in a ‘system under management’. Yet because they are brought into the system, into the way the system is managed, not merely to give an outward measure, they reduce or distort the capacity of the system to adapt; perhaps in inconsequential ways; perhaps in collusive state dependency.

Examples are easy to find, whether in relation to mechanical systems – the malfunctioning probe that overrides normal system adjustment; or more diffuse systems such as a finanicial system running on normal activity but with artificially maintained valuations. In such cases the detachment of the system of measurement from the nature of the system being measured is obvious after the event: shocking perhaps, but there is in fact no guarantee that certitude in the performance measure translates to macroscopic performance: to the quality of governance. Nor that an absence of evidence of performance translates to an absence of performance.

Yet ‘high performance’ is the currency of work contracts, of individuals as of organisations. It is how we judge managers, and how managers regulate their own behaviour. KPIs are the public face of managerial ability, how rewards are determined; how strategic pathways mapped; and how political programs framed. The ‘gap’ in public discourse is as real as any moral imperative, and in fact the more to be trusted because it lies outside moral or intellectual failures of the past. Closing the gap has moral urgency, because it has subsumed the debate on responsibility for the past, and the continuing failures of comprehension in the policy frameworks adopted.

The gaps – the contrast in outcomes according to social state, a form of social state determinism so relic of class consciousness perhaps if one were to enter into a psychoanalytical interpretation – show up social performance in a large sense. What happens then is outside of any effort or ingenuity in the construction. The ‘gap’ so revealed can be interpreted as a managerial lever. How to most effectively repair policy shortcomings, is to act on the elements of the indicator – reading rates, school performance scores, income poverty levels, crowding and so forth. And as such they have the quality of moving forward, while keeping intact the apparatus that lead to the gap.

That is the danger. Of course how is the citizenry or their body of servants and representatives to know that the system is healthy or not? The levers of government function as legislated; proximate effects are manifest.

Enter official statistics. Without fear or favour, it reflects the nation to itself. What is important, what is simply activity? OS rests on consent – on the authority resting in published measures outside performance within the programs of government – and on privileged access. OS, as expounded by NSOs, labours under a cloud of irrelevence if not illegitmacy, ironically a complement of the fatal success of the KPIs on which much management theory now seems to rely.

My presentation at the forthcoming ASC-IMS conference relates this heuristic to an emerging foundational account of inference within the reality of multiple data sourcing, drawing on the ever fecund concept of a learning organisation from the engineering literature. It nevertheless is foundationally and linguistically statistical.