The Tyranny of Metrics

Jerry Z. Muller's "The Tyranny of Metrics" (2018) has a title suggestive of an essential read, but in detail it runs thin. The author may have aimed for a broad audience, providing a brief overview. For those versed in the challenges of metrics, this is not your book. Nonetheless, there are some interesting points, including:

  • "gaming the metrics occurs in every realm: in policing; in primary, secondary, and higher education; in medicine; in nonprofit organizations; and, of course, in business. And gaming is only one class of problems that inevitably arise when using performance metrics as the basis of reward or sanction. There are things that can be measured. There are things that are not worth measuring. But what can be measured is not always worth measuring; what gets measured may have no relationship to what we really want to know. The costs of measuring may be greater than the benefits. The things that get measured may draw effort away from the things we really care about. And measurement may provide us with distorted knowledge - knowledge that seems solid but is actually deceptive." (p. 3)
  • "What has largely gone unnoticed is the recurrence of the same unintended negative consequences of performance metrics, accountability, and transparency across a wide range of institutions... while they are a potentially valuable tool, the virtues of accountability metrics have been oversold, and their costs often underappreciated. It [the book] offers an etiology and diagnosis, but also a prognosis for how metric fixation can be avoided, and its pains alleviated." (p. 6)
  • Education: "The unintended consequences of NCLB's [No Child Left Behind] testing-and-accountability regime are more tangible, and exemplify many of the characteristic pitfalls of metric fixation. Under NCLB, scores on standardized tests are the numerical metric by which success and failure are judged. And the stakes are high for teachers and principals, whose raises in salary and whose very jobs sometimes depend on this performance indicator. It is no wonder, then, that teachers (encouraged by their principals) divert class time toward the subjects tests - mathematic and English - and away from other subjects, such as history, social studies, art, music, and physical education. Instruction in math and English is narrowly focused on the sorts of skills required by the test, rather than broader cognitive processes: that is, students too often learn test-taking strategies rather than substantive knowledge." (p. 92)
  • "Ultimately, the issue is not one of metrics versus judgement, but metrics as informing judgement, which includes knowledge how much weight to give metrics, recognizing their characteristic distortions, and appreciating what can't be measured. In recent decades, too many politicians, business leaders, policymakers, and academic officials have not sight of that." (p. 183)
Black Skin, White Masks
Social Movements and Market Transformations
Subscribe to receive new blog posts via email