Run by the numbers – Wendy Espeland on quantification and discipline

Rankings impose discipline on markets: Wendy Espeland's A Different Kind of Quantitative Sociology talk at ICOS in 2008:

Third is discipline (here comes the Foucault!). Most readers will probably know where this line of argument goes – rankings and other metrics are internalized and begin to alter our behavior without any direct action on the part of the rankings-makers. Espeland draws here on her recent work on US News and World Report law school rankings. For example, because the rankings counted median LSAT scores of only regular full-time students, law schools began to push weaker applicants into doing part-time and night programs. In response to a proposal by US News to start including those programs in the median LSAT calculations, law schools formed committees to ponder reforming or closing their night and part-time programs. Organizational discipline in action.

The whole of the argument has 5 bullet points

  • Quantification takes work
  • Quantification changes what it measures
  • Quantification imposes discipline on those who are measured
  • Numbers are seen as authoritative
  • Numbers are powerful for their aesthetic qualities

The National Law Journal takes on the impact of this in May 2010:

U.S. News does maintain a law school "diversity index," but those data are not a factor in the overall rankings. 

"A lot of people we interviewed expressed concern over the impact of rankings on diversity," said Wendy Espeland, an assistant professor of sociology at Northwestern University. "The overarching tension between catering to the rankings and doing what you think is best for your school is intense." 

Every once in a while I invoke the phrase "quality without a number"; most recently, I guess, in a post on Reification is an Ontological Imperative, also from 2008.  Ed Yourdon's The Politics of Metrics has the same theme.


3 thoughts on “Run by the numbers – Wendy Espeland on quantification and discipline

  1. Edward

    An example:
    Nefarious Numbers
    Douglas N. Arnold, Kristine K. Fowler
    (Submitted on 1 Oct 2010)
    We investigate the journal impact factor, focusing on the applied mathematics category. We demonstrate that significant manipulation of the impact factor is being carried out by the editors of some journals and that the impact factor gives a very inaccurate view of journal quality, which is poorly correlated with expert opinion.
    Comments: 5 pages, 1 figure
    Subjects: History and Overview (math.HO); Digital Libraries (cs.DL)
    Cite as: arXiv:1010.0278v1 [math.HO]

  2. Edward

    the pull quote from that article:
    Because of the (misplaced) emphasis on impact factors, this measure has become a target at which journal editors and publishers aim. This has in turn led to another major source of problems with the factor. Goodhart’s law warns us that “when a measure becomes a target, it ceases to be a good measure” (in the succinct formulation of Strathern). This is precisely the case for impact factors. Their limited utility has been further compromised by impact factor manipulation, the engineering of this supposed measure of journal quality, in ways that increase the measure, but do not add to—indeed subtract from—journal quality.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s