Predicting Information Retrieval Performance

Information Retrieval performance measures are usually retrospective in nature, representing the effectiveness of an experimental process. However, in the sciences, phenomena may be predicted, given parameter values of the system. After developing a measure that can be applied retrospectively or can be predicted, performance of a system using a single term can be predicted given several different types of probabilistic distributions. Information Retrieval performance can be predicted with multiple terms, where statistical dependence between terms exists and is understood. These predictive models may be applied to realistic problems, and then the results may be used to validate the accuracy of the methods used. The application of metadata or index labels can be used to determine whether or not these features should be used in particular cases. Linguistic information, such as part-of-speech tag information, can increase the discrimination value of existing terminology and can be studied predictively.

This work provides methods for measuring performance that may be used predictively. Means of predicting these performance measures are provided, both for the simple case of a single term in the query and for multiple terms. Methods of applying these formulae are also suggested.

Robert Losee has been a professor at the University of North Carolina at Chapel Hills School of Information and Library Science since 1986, after receiving a Ph.D. from the University of Chicago. He has taught courses in Information Retrieval, including an introductory graduate course, and an advanced Artificial Intelligence for Information Retrieval course. He has also taught courses in Information Theory, including a doctoral seminar in the area. His most recent monograph is Information from Processes: About the Nature of Information Creation, Use, and Representation (Springer, 2012).