TY - JOUR
T1 - Quality Review of Mass Adjudication
T2 - A Randomized Natural Experiment at the Board of Veterans Appeals, 2003-16
AU - Ho, Daniel E.
AU - Handan-Nader, Cassandra
AU - Ames, David
AU - Marcus, David
N1 - Publisher Copyright:
© 2019 The Author(s) 2019.
PY - 2019/7/1
Y1 - 2019/7/1
N2 - We study a unique natural experiment, during which 5-10% of draft opinions by judges of the Board of Veterans Appeals (BVA) were randomly selected for "quality review (QR)" by a team of full-Time staff attorneys. The express goals of this performance program were to measure accuracy and reduce remands on appeal. In cases of legal error, the QR team wrote memoranda to judges for correction of draft opinions. We use rich internal administrative data on nearly 600,000 cases from 2003 to 2016 to conduct the first rigorous evaluation of this program. With precise estimates, we show that QR had no appreciable effects on appeals or remands. Based on internal records, we demonstrate that this inefficacy is likely by design, as meeting the performance measure of "accuracy" conflicted with error correction. These findings inform longstanding questions of law, organization, and bureaucracy, including performance management, standards of review, and institutional design of adjudication.
AB - We study a unique natural experiment, during which 5-10% of draft opinions by judges of the Board of Veterans Appeals (BVA) were randomly selected for "quality review (QR)" by a team of full-Time staff attorneys. The express goals of this performance program were to measure accuracy and reduce remands on appeal. In cases of legal error, the QR team wrote memoranda to judges for correction of draft opinions. We use rich internal administrative data on nearly 600,000 cases from 2003 to 2016 to conduct the first rigorous evaluation of this program. With precise estimates, we show that QR had no appreciable effects on appeals or remands. Based on internal records, we demonstrate that this inefficacy is likely by design, as meeting the performance measure of "accuracy" conflicted with error correction. These findings inform longstanding questions of law, organization, and bureaucracy, including performance management, standards of review, and institutional design of adjudication.
KW - H11
KW - K23
KW - K40
UR - http://www.scopus.com/inward/record.url?scp=85096238575&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85096238575&partnerID=8YFLogxK
U2 - 10.1093/jleo/ewz001
DO - 10.1093/jleo/ewz001
M3 - Article
AN - SCOPUS:85096238575
SN - 8756-6222
VL - 35
SP - 239
EP - 288
JO - Journal of Law, Economics, and Organization
JF - Journal of Law, Economics, and Organization
IS - 2
M1 - ewz001
ER -