Judging Judgment

Philip E. Tetlock and I agree that forecasting tools are best evaluated in peer-reviewed settings and in comparison not only to expert judgments, but also to alternative modeling strategies. Applying his suggested standards of assessment, however, certain forecasting models not only outperform expert judgments, but also have gone head-to-head with alternative models and outperformed them. This track record demonstrates the capability to make significant, reliable predictions of difficult, complex events. The record has unfolded, contrary to Tetlock's contention, not only in government and business applications, but also in numerous peer-reviewed publications containing hundreds of real-time forecasts. Moreover, reliable prediction is achieved while avoiding significant false-positive or false-negative rates.