This page provides source code, data, and the game rules for the paper:
Find It If You Can: A Game for Modeling Different Types of Web Search Success Using Interaction Data.
Mikhail Ageev, Qi Guo, Dmitry Lagun, and Eugene Agichtein.
In Proceedings of the 34th annual international ACM SIGIR conference on Research and development in information retrieval (SIGIR'2011).
ACM, Beijing, China.
@inproceedings{sigir2011/ageev-ufindit,
author = {Mikhail Ageev, Qi Guo, Dmitry Lagun, and Eugene Agichtein}
title = {Find It If You Can: A Game for Modeling Different Types of Web Search Success Using Interaction Data.}
booktitle = {Proceeding of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval({SIGIR} 2011)},
year = {2011},
pages = {xxx--xxx},
keywords = {User studies, query log analysis, web search sucess}
}
ABSTRACT:
A better understanding of strategies and behavior of successful searchers is crucial for improving
the experience of all searchers. However, research of search behavior has been struggling with the
tension between the relatively small-scale, but controlled lab studies, and the large-scale
log-based studies where the searcher intent and many other important factors have to be inferred.
We present our solution for performing controlled, yet realistic, scalable, and reproducible studies
of searcher behavior. We focus on difficult informational tasks, which tend to frustrate many users
of the current web search technology.
First, we propose a principled formalization of different types of "success" for informational search,
which encapsulate and sharpen previously proposed models. Second, we present a scalable game-like
infrastructure for crowdsourcing search behavior studies, specifically targeted towards capturing
and evaluating successful search strategies on informational tasks with known intent.
Third, we report our analysis of search success using these data, which confirm and extends previous findings.
Finally, we demonstrate that our model can predict search success more effectively than the existing
state-of-the-art methods, on both our data and on a different set of log data collected from regular
search engine sessions. Together, our search success models, the data collection infrastructure,
and the associated behavior analysis techniques, significantly advance the study of success in web search.
ADDITIONAL INFORMATION: