Abstract
This paper presents an empirical study on four techniques of language model adaptation, including a maximum a posteriori (MAP) method and three discriminative training models, in the application of Japanese Kana-Kanji conversion. We compare the performance of these methods from various angles by adapting the baseline model to four adaptation domains. In particular, we attempt to interpret the results given in terms of the character error rate (CER) by correlating them with the characteristics of the adaptation domain measured using the information-theoretic notion of cross entropy. We show that such a metric correlates well with the CER performance of the adaptation methods, and also show that the discriminative methods are not only superior to a MAP-based method in terms of achieving larger CER reduction, but are also more robust against the similarity of background and adaptation domains.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Bellagarda, J.: An Overview of Statistical Language Model Adaptation. In: ITRW on Adaptation Methods for Speech Recognition, pp. 165–174 (2001)
Collins, M.: Ranking Algorithms for Name-Entity Extraction: Boosting and the Voted Perceptron. In: ACL (2002)
Collins, M.: Discriminative Training Methods for Hidden Markov Models: Theory and Experiments with Perceptron Algorithms. In: EMNLP (2002)
Gao, J., Yu, H., Xu, P., Yuan, W.: Minimum Sample Risk Methods for Language Modeling (2005) (to Appear)
Manning, C.D., Schütze, H.: Foundations of Statistical Natural Language Processing. The MIT Press, Cambridge (1999)
Dagan, I., Lee, L., Pereira, F.: Similarity-based models of cooccurrence probabilities. Machine Learning 34(1-3), 43–69 (1999)
Lee, L.: Measures of distributional similarity. In: ACL, pp. 25-32 (1999)
Roark, B., Saraclar, M., Collins, M.: Corrective Language Modeling for Large Vocabulary ASR with the Perceptron Algorithm. In: ICASSP, pp. 749–752 (2004)
Bacchiani, M., Roark, B., Saraclar, M.: Language Model Adaptation with MAP Estimation and the Perceptron Algorithm. In: HLT-NAACL, pp. 21–24 (2004)
Bacchiani, M., Roark, B.: Unsupervised language model adaptation. In: ICASSP, pp. 224–227 (2003)
Och, F.J.: Minimum error rate training in statistical machine translation. In: ACL, pp. 160–167 (2003)
Gao, J., Goodman, J., Li, M., Lee, K.F.: Toward a unified approach to statistical language modeling for Chinese. ACM Transactions on Asian Language Information Processing l-1, 3–33 (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Yuan, W., Gao, J., Suzuki, H. (2005). An Empirical Study on Language Model Adaptation Using a Metric of Domain Similarity. In: Dale, R., Wong, KF., Su, J., Kwong, O.Y. (eds) Natural Language Processing – IJCNLP 2005. IJCNLP 2005. Lecture Notes in Computer Science(), vol 3651. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11562214_83
Download citation
DOI: https://doi.org/10.1007/11562214_83
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-29172-5
Online ISBN: 978-3-540-31724-1
eBook Packages: Computer ScienceComputer Science (R0)