Jul
21

Measuring Tomorrow

There are piles of critiques of economic indicators driving decision making, and a range of proposals for alternatives. In 2018 Eloi Laurent published "Measuring Tomorrow: Accounting for Well-Being, Resilience, and Sustainability in the Twenty-First Century", which proposes new metrics. The book gives concrete examples of metrics and existing data sets - for those interested in this subject, this is a great introduction.

"The starting point of this book is therefore that, contrary to their etymology, data are the product of values, which in turn influence human attitudes and behaviors via policy when they become indicators by the combined action of scholars and policy makers." (p. 1)

"What happened to economic analysis to make all these key insights largely forgotten for so long? The shift away from well-being and sustainability happened in two crucial steps. First, at the beginning of the twentieth century, economists decided to divorce their study from philosophy – or, more precisely, from ethics – and make it a science of efficiency, modeled on physics… Then, after the Second World War, it purported to become the science of growth. Both metamorphoses were symbolized by a single indicator: gross domestic product." (p. 5).

"…trust is the bedrock of economic activities: without it, no bank, no business, no government can long remain, let alone prosper. Trust reduces the uncertainty inherent in human behavior, turning it into acceptable or unacceptable risk… To measure trust, it is important first to define its different forms. There are essentially three of these: trust in institutions or organizations (which is by far the most important in contemporary societies), trust between people, and finally the problematic notion of "trust in the future," widely used in economic forecasting." (p. 88-89)

"We should therefore not confuse different dimensions of trust with one another, especially what political scientists Pascal Perrineau calls "vertical trust" (e.g., trust in institutions) and "horizontal trust" (e.g., trust between people)." (p. 94)

"The challenge posed to developed countries is that, yes, they are often more advanced in terms of good environmental practices on their own soil, but in so doing they are only paying attention to part of their environmental footprint, one that is visible and directly under their control. As their level of economic development increases, countries reduce the levels of natural resource extraction in their own territory but do not reduce their consumption of natural resources. Instead they outsource the environmental damage caused by their economic development to countries that are willing to pay the environmental costs in exchange for pay. But this cost is often paid by the poorest people who see little of the actual money." (p. 115)

"We have thus arrived, before concluding this book, at an interesting paradox. If sustainability is best measured globally, resilience and well-being are best assessed locally." (p. 189) 

  614 Hits
Dec
08

Measuring What Counts

In "Measuring What Counts: The Global Movement for Well-Being" (2019), Stiglitz, Fitoussi and Durand build upon the work that was conducted following a 2009 commission to re-think what measures are used to assess the health of the economy (particularly GDP). The financial crises forced reflections on how the vulnerabilities were not understood; to which these authors say that most "fundamentally, policy-makers ignored these warning signals because of the ideological blinders that prevented them (and their economic advisors) from seeing the dangers ahead" (p. 6). The 2009 work was led by Stiglitz, Fitoussi and Sen and the results were published in the book "Mismeasuring Our Lives" (2010). Nearly a decade later, this book broadens the areas of work, summarizes progress made, and highlights areas that require much more work / research.

On metrics: "That GDP didn't do all that was hoped of it shouldn't be a surprise: no single number can summarize anything as complex as the economy" (p. 9). Yet, there is a slippery slope in metrics, some are usefully contextualized, but then not globally relevant, or vice versa. "There is some tension, though, between the desire to have metrics that reflect the particular situation within a country and the need to have metrics that enable cross-country comparisons, i.e., to give a picture of how a country is doing relative to others. Both perspectives are important: we all want to know how well we are doing (in one dimension or another) relative to our past or relative to what is occurring everywhere" (p. 19). The authors conclude, on this tensions that "more needed to be done at the international level" (p. 19).

On inequality: First, we need to do a better job understanding the multiplication / layering of vulnerability, rather than as stand-alone metrics: "Many of these inequality indicators are correlated, with the same households or individuals experiencing disadvantage in many of these dimensions. The report argued that one should focus on the household (or even better, the individual) as the unit of analysis, looking at all the dimensions that affect well-being at the same time" (p. 67). Second, we do not have the data, nor even the definitions for the data, to provide a basis for these understandings: "Some of the criteria for comparing horizontal inequalities across countries also lack well-established statistical conventions and definitions. An example is provided by disability status where, despite decade-long discussions, no generally accepted definition applied across official surveys exists yet. In other cases, no statistical criteria exists simply because these types of horizontal inequalities (for example, those linked to sexual orientation) have only recently entered public discussions" (p. 83-84). Even more challenging, they are, is assessing inequality of opportunity, not only outcomes (p. 88). How to do that? The authors recommend that: "Data should be disaggregated by age, gender, disability, sexual orientation, education, and other markers of social status in order to describe group differences in well-being outcomes; and metrics to describe within-household inequalities, such as those related to asset ownership and the sharing of resources and financial decisions within the household, should be developed." (p. 156)

On trust: "If the measures we rely on are out of sync with how citizens experience their lives, a lack of trust in government will develop" (p. 9). Later, they continue, "trust is negatively correlated with income inequality. And rising income inequality has also been related to lower trust in institutions. High-trusting societies have lower levels of income inequality, measured by Gini coefficients, while low-trusting societies show typically higher levels of income inequality. Trust is undermined by things that run against people's sense of fairness. Since, at least in many countries, there is a general sense that income is inequitably distributed, it is not a surprise that economic equality is consistently identified as one of the strongest predictors of generalized trust, and that countries with highest levels of trust rank highest on economic equality (e.g. Nordic countries, the Netherlands, Canada; OECD, 2018a)." (p. 126)

  1071 Hits
Jul
30

The Tyranny of Metrics

Jerry Z. Muller's "The Tyranny of Metrics" (2018) has a title suggestive of an essential read, but in detail it runs thin. The author may have aimed for a broad audience, providing a brief overview. For those versed in the challenges of metrics, this is not your book. Nonetheless, there are some interesting points, including:

  • "gaming the metrics occurs in every realm: in policing; in primary, secondary, and higher education; in medicine; in nonprofit organizations; and, of course, in business. And gaming is only one class of problems that inevitably arise when using performance metrics as the basis of reward or sanction. There are things that can be measured. There are things that are not worth measuring. But what can be measured is not always worth measuring; what gets measured may have no relationship to what we really want to know. The costs of measuring may be greater than the benefits. The things that get measured may draw effort away from the things we really care about. And measurement may provide us with distorted knowledge - knowledge that seems solid but is actually deceptive." (p. 3)
  • "What has largely gone unnoticed is the recurrence of the same unintended negative consequences of performance metrics, accountability, and transparency across a wide range of institutions... while they are a potentially valuable tool, the virtues of accountability metrics have been oversold, and their costs often underappreciated. It [the book] offers an etiology and diagnosis, but also a prognosis for how metric fixation can be avoided, and its pains alleviated." (p. 6)
  • Education: "The unintended consequences of NCLB's [No Child Left Behind] testing-and-accountability regime are more tangible, and exemplify many of the characteristic pitfalls of metric fixation. Under NCLB, scores on standardized tests are the numerical metric by which success and failure are judged. And the stakes are high for teachers and principals, whose raises in salary and whose very jobs sometimes depend on this performance indicator. It is no wonder, then, that teachers (encouraged by their principals) divert class time toward the subjects tests - mathematic and English - and away from other subjects, such as history, social studies, art, music, and physical education. Instruction in math and English is narrowly focused on the sorts of skills required by the test, rather than broader cognitive processes: that is, students too often learn test-taking strategies rather than substantive knowledge." (p. 92)
  • "Ultimately, the issue is not one of metrics versus judgement, but metrics as informing judgement, which includes knowledge how much weight to give metrics, recognizing their characteristic distortions, and appreciating what can't be measured. In recent decades, too many politicians, business leaders, policymakers, and academic officials have not sight of that." (p. 183)
  1108 Hits
Subscribe to receive new blog posts via email