Now that the regular season has finished in the NBA, our Wisdom of Crowds predictions (see the mid-season post below, linked here) can be evaluated. We had predictions of the orders the 15 teams would finish in the Eastern and Western conferences, collected from 100s of people. Using these, we formed a standard aggregate prediction using the Borda count method, and a psychologically motivated prediction using what we call the Thurstone method. The actual predictions were detailed in our earlier post.
To measure the accuracy of the rankings, we used the standard Kendall Tau measure. This is basically a count of the number of pair-wise swaps of teams you need to turn to convert a prediction to the true final order. Lower taus are better, with zero being perfect prediction. The figure below shows a histogram of all the taus for the individual people’s predictions, and for the Borda and Thurstone aggregate predictions, for both conferences.
Out of the 172 people predicting the East, the Borda aggregate does as well or better than 126 (73%), while the Thurstone aggregate does as well or better than 156 (91%). Out of the 156 people predicting the West, the Borda aggregate does as well or better than 135 (87%), and the Thurstone aggregate does as well or better than 144 (92%). So, both Wisdom of Crowds methods did well, relative to people’s performance, and the Thurstone approach was better than the Borda approach.