Drs. Michael Lee and Mark Steyvers of MADLab have been recently featured in an article on the UCI webpage. The article discusses one of their current projects, which involves aggregating intelligence from crowds to produce reliable forecasts. To read more about the project, follow this link to the article:

http://www.uci.edu/features/2011/09/feature_forecast_110919.php.

Madlab currently has three undergraduates working as research assistants.

Annie Ditta is a third year psychology major and biology minor. The research she does in Madlab investigates collaborative group memory. Instead of using a standard wisdom of crowd approach, Annie is using an iterative learning condition where each participant is allowed to see and alter the immediately preceding participant’s response. So far, Annie’s reseach has proposed that the group aggregate seems to be yielding better performance at the end of a chain.

Mindy De Young is a fourth year psychology major. Mindy’s research in Madlab involves sports predictions. Through her online experiments, she found that you can identify experts through wisdom of the crowds approaches, and that this approach is more accurate than self-assessment of expertise. Mindy is also graduating this academic year and will be enrolling in graduate school at Cal State Long Beach September 2011.

Daniela Amado is a fourth year psychology major. Her research in Madlab investigates the conscious awareness of memory performance, or metamemory, in Alzheimers patients.  Daniela compared patients’ estimated to actual memory performance and found that metamemory decays as soon as Alzheimers disease is present. Daniela is also graduating this year and plans to take a year off before pursuing a higher education in law.

Joachim Vandekerckhove is visiting the OC from Leuven, Belgium. His research primarily involves the application of various diffusion models. Recently, he applied hierarchical diffusion models on speeded semantic categorization task data to investigate human reaction time. He found that applying a diffusion model can provide more profound insight into data. The models used in his research were found to have the ability to predict information uptake rate and stimulus encoding time.

For those who want to pick on Joachim’s brain, have drinks, hang out, etc…, he will be here for 2 more weeks.

This year, Madlab is getting the spotlight at the 33rd annual Cognitive Science Conference! Current lab members Brent Miller, James Pooley, Sean Tauber, and Shunan Zhang will all be presenting talks on their recent research projects, and Tim Rubin will present his research findings with a poster.

Brent will be discussing how cognitive models can accurately measure expertise on rank ordering tasks. In addition, he will also be presenting research on the result of iterative learning in the Wisdom of Crowds. James’ research aims to use cognitive models to assess memory performance in Alzheimer’s patients. Sean will be talking about how people can infer the goals and intentions of others by using certain cues, such as spatial orientation and the presence or absence of visual obstructions. And finally, Shunan’s research involves applying a new machine learning method, Space Instance Representation, to the problem of category identification.

Madlab alumni Matt Zeigenfuse will also be presenting a talk on categorization and similarity judgment, research mostly drawn from his dissertation. Where would science be without us? The conference will be held in Boston, Massachusetts in July 2011.

Snaps for Shunan Zhang, who recently published several papers on human decision-making, one pertaining to the Wisdom of Crowds and another integrating hierarchical Bayesian analysis with bandit problems.  Next Shunan is visiting the Netherlands to collaborate with several of our Dutch colleagues on more decision-making research. You’ll be missed, Shunan!

Congratulations to James Pooley, 3rd year graduate student of Madlab, who successfully advanced on March 7, 2011. James’ research involves using Bayesian analysis to assess the stages of Alzheimer’s disease and related disorders (ADRD). One of his recent papers (published in the Journal of Mathematical Psychology) proposes that hierarchical Bayesian models can not only account for individual differences among ADRD patients, but can also efficiently explain the patients’ performances on memory recall tasks. Figure 1 shows that the observed serial position curves (indicated by black lines) are proportional to the model’s posterior predictions (indicated by areas of squares).  Next, James is working on a model with more detailed processing assumptions to gain a better insight into human memory. So stay tuned.

…to Matt Zeigenfuse (left) and Mike Yi (right) who both defended their theses in January and February 2011, respectively. Matt, whose research primarily focuses on categorization and features, has continued on with a post-doc in Michigan. Mike, with his research in optimal decision-making and problem solving, is still contemplating which corporation to invade.

Congratulations to Pernille Hemmer for getting not just a good, but great doctoral degree in Cognitive Science. She succesfully defended her dissertation on Feb 17, 2011 at UC Irvine. Pernille has written numerous papers on how prior knowledge interacts with episodic memory. She has also actively presented her work in various research conferences.  She will be continuing her academic endeavours with a post-doc at Syracuse University in New York starting mid-March 2011. (We’ll miss you Barbz!)

Now that the regular season has finished in the NBA, our Wisdom of Crowds predictions (see the mid-season post below, linked here) can be evaluated. We had predictions of the orders the 15 teams would finish in the Eastern and Western conferences, collected from 100s of people. Using these, we formed a standard aggregate prediction using the Borda count method, and a psychologically motivated prediction using what we call the Thurstone method. The actual predictions were detailed in our earlier post.

To measure the accuracy of the rankings, we used the standard Kendall Tau measure. This is basically a count of the number of pair-wise swaps of teams you need to turn to convert a prediction to the true final order. Lower taus are better, with zero being perfect prediction. The figure below shows a histogram of all the taus for the individual people’s predictions, and for the Borda and Thurstone aggregate predictions, for both conferences.

Out of the 172 people predicting the East, the Borda aggregate does as well or better than 126 (73%), while the Thurstone aggregate does as well or better than 156 (91%). Out of the 156 people predicting the West, the Borda aggregate does as well or better than 135 (87%), and the Thurstone aggregate does as well or better than 144 (92%). So, both Wisdom of Crowds methods did well, relative to people’s performance, and the Thurstone approach was better than the Borda approach.