By Mark Himmelstein, Pavel Atanasov, David V. Budescu

« A growing body of research indicates that forecasting skill is a unique and stabletrait: forecasters with a track record of high accuracy tend to maintain this record. Buthow does one identify skilled forecasters effectively? We address this question usingdata collected during two seasons of a longitudinal geopolitical forecastingtournament.Our first analysis, which compares psychometric traits assessed prior toforecasting,indicates intelligence consistently predicts accuracy. Next, using methods adapted fromclassical test theory and item response theory, we model latent forecasting skill basedon the forecasters’ past accuracy, while accounting for the timing of theirforecastsrelative to question resolution. Our results suggest these methods performbetter atassessing forecasting skill than simpler methods employed by many previous studies.By parsing the data at different time points during the competitions, we assess therelative importance of each information source over time. When past performanceinformation is limited, psychometric traits are useful predictors of future performance,but, as more information becomes available, past performance becomes the strongerpredictor of future accuracy. Finally, we demonstrate the predictive validity of theseresults on out-of-sample data, and their utility in producing performance weights forwisdom-of-crowds aggregations. »

%d blogueurs aiment cette page :