Find It If You Can

This page provides source code, data, and the game rules for the paper:

Find It If You Can: A Game for Modeling Different Types of Web Search Success Using Interaction Data.
Mikhail Ageev, Qi Guo, Dmitry Lagun, and Eugene Agichtein.
In Proceedings of the 34th annual international ACM SIGIR conference on Research and development in information retrieval (SIGIR'2011). ACM, Beijing, China.

Full text PDF

To cite out paper:
author = {Mikhail Ageev, Qi Guo, Dmitry Lagun, and Eugene Agichtein}
title = {Find It If You Can: A Game for Modeling Different Types of Web Search Success Using Interaction Data.}
booktitle = {Proceeding of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval({SIGIR} 2011)},
year = {2011},
pages = {xxx--xxx},
keywords = {User studies, query log analysis, web search sucess}

A better understanding of strategies and behavior of successful searchers is crucial for improving the experience of all searchers. However, research of search behavior has been struggling with the tension between the relatively small-scale, but controlled lab studies, and the large-scale log-based studies where the searcher intent and many other important factors have to be inferred. We present our solution for performing controlled, yet realistic, scalable, and reproducible studies of searcher behavior. We focus on difficult informational tasks, which tend to frustrate many users of the current web search technology. First, we propose a principled formalization of different types of "success" for informational search, which encapsulate and sharpen previously proposed models. Second, we present a scalable game-like infrastructure for crowdsourcing search behavior studies, specifically targeted towards capturing and evaluating successful search strategies on informational tasks with known intent. Third, we report our analysis of search success using these data, which confirm and extends previous findings. Finally, we demonstrate that our model can predict search success more effectively than the existing state-of-the-art methods, on both our data and on a different set of log data collected from regular search engine sessions. Together, our search success models, the data collection infrastructure, and the associated behavior analysis techniques, significantly advance the study of success in web search.