The barking dog that bites: Test score volatility and school rankings in Punjab, Pakistan

Felipe Barrera-Osorio, Alejandro J. Ganimian

Research output: Contribution to journalArticlepeer-review


This paper is the first to explore the implications of test score volatility for school accountability policies in a lower-middle-income country. Using two new datasets from the province of Punjab, Pakistan, we find that rankings based on school-level averages of students' test scores fluctuate considerably from one year to the next due to factors unrelated to school quality. First, most variation in test scores is within, not between schools, allowing noise to play a large part in determining school rankings. Second, variation in school test scores is partially attributable to fluctuations in the characteristics of student cohorts. Third, an even larger share is explained by one-time shocks. Fourth, these problems are more pronounced in small schools. As a result, rankings that rely on test score levels or changes often fail to identify top-performing or rapidly-improving schools, since they are influenced by factors beyond schools' control.

Original languageEnglish (US)
Pages (from-to)31-54
Number of pages24
JournalInternational Journal of Educational Development
StatePublished - Jul 1 2016


  • Accountability
  • School rankings
  • Test score volatility

ASJC Scopus subject areas

  • Education
  • Development
  • Sociology and Political Science


Dive into the research topics of 'The barking dog that bites: Test score volatility and school rankings in Punjab, Pakistan'. Together they form a unique fingerprint.

Cite this