Abstract
Retrieval performance can often be improved significantly by using a number of different retrieval algorithms and combining the results, in contrast to using just a single retrieval algorithm. This is because different retrieval algorithms, or retrieval experts, often emphasize different document and query features when determining relevance and therefore retrieve different sets of documents. However, it is unclear how the different experts are to be combined, in general, to yield a superior overall estimate. We propose a method by which the relevance estimates made by different experts can be automatically combined to result in superior retrieval performance. We apply the method to two expert combination tasks. The applications demonstrate that the method can identify high performance combinations of experts and also is a novel means for determining the combined effectiveness of experts.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Brian T. Bartell. Optimizing Ranking Functions: A Connectionist Approach to Adaptive Information Retrieval. PhD thesis, Department of Computer Science & Engineering, The University of California, San Diego, 1994.
Brian T. Bartell, Garrison W. Cottrell, and Richard K. Belew. Learning the optimal parameters in a ranked retrieval system using multi-query relevance feedback. In Proceedings of the Symposium on Document Analysis and Information Retrieval,Las Vegas, 1994. in press.
Nicholas J. Belkin, C. Cool, W. Bruce Croft, and James P. Callan. Effect of multiple query representations on information retrieval system performance. In Proc. SIGIR 1993, pages 339–346, Pittsburgh, PA, June 1993.
I. Borg and J. Lingoes. Multidimensional Similarity Structure Analysis. Springer-Verlag, New York, 1987.
Edward A. Fox, M. Prabhakar Koushik, Joseph Shaw, Russell Modlin, and Durgesh Rao. Combining evidence from multiple searches. In Donna K. Harman, editor, The First Text REtrieval Conference (TREC-1), pages 319–328, March 1993. NIST Special Publication 500–207.
Norbert Fuhr and Chris Buckley. A probabilistic learning approach for document indexing. ACM Transactions on Information Systems, 9 (3): 223–248, 1991.
Michael Gordon. Probabilistic and genetic algorithms in document retrieval. Communications of the ACM, 31 (10), October 1988.
L. Guttman. What is not what in statistics. The Statistician, 26: 81–107, 1978.
Donna Harman. Overview of the first Text REtrieval Conference. In Proceedings of the ACM SIGIR, pages 36–48, Pittsburgh, PA, June 1993.
J. Katzer„M. J. McGill, J. A. Tessier, W. Frakes, and P. DasGupta. A study of the overlap among document representations. Information Technology: Research and Development, 1 (4): 261–274, Oct 1982.
David D. Lewis and W. Bruce Croft. Term clustering of syntactic phrases. In Proceedings of the ACM SIGIR, Brussels, Sept 1990.
William H. Press, Brian P. Flannery, Saul A. Teukolsky, and William T. Vetterling. Numerical Recipes in C: The Art of Scientific Computing. Cambridge University Press, 1988.
T. Saracevic and P. Kantor. A study of information seeking and retrieving. III. Searchers, searches, overlap. Journal of the ASIS, 39 (3): 197–216, 1988.
Paul Thompson. A combination of expert opinion approach to probabilistic information retrieval, part 1: The conceptual model. Information Processing El Management, 26 (3): 371–382, 1990.
Paul Thompson. Description of the PRC CEO algorithm for TREC. In Donna K. Harman, editor, The First Text REtrieval Conference (TREC-1), pages 337–342, March 1993. NIST Special Publication 500–207.
Howard Turtle and W. Bruce Croft. Evaluation of an inference network-based retrieval model. ACM Transactions on Information Systems, 9(3):187–222, july 1991
S. K. M. Wong, Y. J. Cai, and Y. Y. Yao. Computation of term associations by a neural network. In Proceedings of SIGIR, Pittsburgh, PA, June 1993.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1994 Springer-Verlag London Limited
About this paper
Cite this paper
Bartell, B.T., Cottrell, G.W., Belew, R.K. (1994). Automatic Combination of Multiple Ranked Retrieval Systems. In: Croft, B.W., van Rijsbergen, C.J. (eds) SIGIR ’94. Springer, London. https://doi.org/10.1007/978-1-4471-2099-5_18
Download citation
DOI: https://doi.org/10.1007/978-1-4471-2099-5_18
Publisher Name: Springer, London
Print ISBN: 978-3-540-19889-5
Online ISBN: 978-1-4471-2099-5
eBook Packages: Springer Book Archive