TY - GEN
T1 - Workload sampling for enterprise search evaluation
AU - Rowlands, Tom
AU - Hawking, David
AU - Sankaranarayana, Ramesh
PY - 2007
Y1 - 2007
N2 - In real world use of test collection methods, it is essential that the query test set be representative of the work load expected in the actual application. Using a random sample of queries from a media company's query log as a 'gold standard' test set we demonstrate that biases in sitemap-derived and top n query sets can lead to significant perturbations in engine rankings and big differences in estimated performance levels.
AB - In real world use of test collection methods, it is essential that the query test set be representative of the work load expected in the actual application. Using a random sample of queries from a media company's query log as a 'gold standard' test set we demonstrate that biases in sitemap-derived and top n query sets can lead to significant perturbations in engine rankings and big differences in estimated performance levels.
KW - Information retrieval evaluation
KW - Query sampling
UR - http://www.scopus.com/inward/record.url?scp=36448970053&partnerID=8YFLogxK
U2 - 10.1145/1277741.1277959
DO - 10.1145/1277741.1277959
M3 - Conference contribution
SN - 1595935975
SN - 9781595935977
T3 - Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR'07
SP - 887
EP - 888
BT - Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR'07
T2 - 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR'07
Y2 - 23 July 2007 through 27 July 2007
ER -