Abstract
Most memory-based methods use encoded retrieved pairs as the translation memory (TM) to provide external guidance, but there still exist some noisy words in the retrieved pairs. In this paper, we propose a simple and effective end-to-end model to select useful sentence words from the encoded memory and incorporate them into the NMT model. Our model uses a novel memory selection mechanism to avoid the noise from similar sentences and provide external guidance simultaneously. To verify the positive influence of selected retrieved words, we evaluate our model on the single-domain dataset namely JRC-Acquis and multi-domain dataset comprised of existing benchmarks including WMT, IWSLT, JRC-Acquis, and OpenSubtitles. Experimental results demonstrate our method can improve the translation quality under different scenarios.
J. Yang and J. Wan—Equal contribution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Akoury, N., Krishna, K., Iyyer, M.: Syntactically supervised transformers for faster neural machine translation. In: ACL 2019, pp. 1269–1281 (2019)
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. In: ICLR 2015 (2015)
Bapna, A., Firat, O.: Non-parametric adaptation for neural machine translation. In: NAACL 2019, pp. 1921–1931 (2019)
Biçici, E., Dymetman, M.: Dynamic translation memory: using statistical machine translation to improve translation memory fuzzy matches. In: Gelbukh, A. (ed.) CICLing 2008. LNCS, vol. 4919, pp. 454–465. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-78135-6_39
Bulté, B., Tezcan, A.: Neural fuzzy repair: integrating fuzzy matches into neural machine translation. In: ACL 2019, pp. 1800–1809 (2019)
Cao, Q., Xiong, D.: Encoding gated translation memory into neural machine translation. In: EMNLP 2018, pp. 3042–3047 (2018)
Dahlmann, L., Matusov, E., Petrushkov, P., Khadivi, S.: Neural machine translation leveraging phrase-based models in a hybrid search. In: EMNLP 2017, pp. 1411–1420 (2017)
Farajian, M.A., Turchi, M., Negri, M., Federico, M.: Multi-domain neural machine translation through unsupervised adaptation. In: WMT 2017, pp. 127–137 (2017)
Firat, O., Cho, K., Bengio, Y.: Multi-way, multilingual neural machine translation with a shared attention mechanism. In: NAACL 2016, pp. 866–875 (2016)
Gehring, J., Auli, M., Grangier, D., Yarats, D., Dauphin, Y.N.: Convolutional sequence to sequence learning. In: ICML 2017, pp. 1243–1252 (2017)
Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. CoRR arXiv:1312.6211 (2013)
Gu, J., Lu, Z., Li, H., Li, V.O.K.: Incorporating copying mechanism in sequence-to-sequence learning. In: ACL 2016 (2016)
Gu, J., Wang, Y., Cho, K., Li, V.O.K.: Search engine guided neural machine translation. In: AAAI 2018, pp. 5133–5140 (2018)
Hewavitharana, S., Vogel, S., Waibel, A.: Augmenting a statistical translation system with a translation memory. In: EAMT 2005, vol. 5 (2005)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: ICLR 2015 (2015)
Koehn, P., et al.: Moses: open source toolkit for statistical machine translation. In: ACL 2007, pp. 177–180 (2007)
Koehn, P., Och, F.J., Marcu, D.: Statistical phrase-based translation. In: NAACL 2003 (2003)
Koehn, P., Senellart, J.: Convergence of translation memory and statistical machine translation. In: AMTA 2010, pp. 21–31 (2010)
Koehn, P., Senellart, J.: Fast approximate string matching with suffix arrays and A* parsing. In: AMTA 2010 (2010)
Li, J., Xiong, D., Tu, Z., Zhu, M., Zhang, M., Zhou, G.: Modeling source syntax for neural machine translation. In: ACL 2017, pp. 688–697 (2017)
Luong, M.T., Manning, C.D.: Stanford neural machine translation systems for spoken language domains. In: IWSLT 2015, pp. 76–79 (2015)
Maruf, S., Martins, A.F.T., Haffari, G.: Selective attention for context-aware neural machine translation. In: NAACL 2019, pp. 3092–3102 (2019)
Ortega, J.E., Sánchez-Martınez, F., Forcada, M.L.: Fuzzy-match repair using black-box machine translation systems: what can be expected. In: AMTA 2016, vol. 1, pp. 27–39 (2016)
Sennrich, R., Haddow, B., Birch, A.: Neural machine translation of rare words with subword units. In: ACL 2016 (2016)
Simard, M., Isabelle, P.: Phrase-based machine translation in a computer-assisted translation environment. In: MT Summit XII, pp. 120–127 (2009)
Simard, M., Langlais, P.: Sub-sentential exploitation of translation memories. In: Machine Translation Summit, vol. 8, pp. 335–339 (2001)
Vaswani, A., et al.: Attention is all you need. In: NIPS 2017, pp. 5998–6008 (2017)
Wang, K., Zong, C., Su, K.: Integrating translation memory into phrase-based machine translation during decoding. In: ACL 2013, pp. 11–21 (2013)
Wuebker, J., Green, S., DeNero, J.: Hierarchical incremental adaptation for statistical machine translation. In: EMNLP 2015, pp. 1059–1065 (2015)
Xia, Y., et al.: Deliberation networks: sequence generation beyond one-pass decoding. In: NIPS 2017, pp. 1784–1794 (2017)
Xu, J., Crego, J.M., Senellart, J.: Boosting neural machine translation with similar translations. In: ACL 2020, pp. 1580–1590 (2020)
Yang, B., Li, J., Wong, D.F., Chao, L.S., Wang, X., Tu, Z.: Context-aware self-attention networks. In: AAAI 2019, pp. 387–394 (2019)
Zhang, J., Utiyama, M., Sumita, E., Neubig, G., Nakamura, S.: Improving neural machine translation through phrase-based forced decoding. In: IJCNLP 2017, pp. 152–162 (2017)
Zhang, J., Utiyama, M., Sumita, E., Neubig, G., Nakamura, S.: Guiding neural machine translation with retrieved translation pieces. In: NAACL 2018, pp. 1325–1335 (2018)
Zhou, L., Hu, W., Zhang, J., Zong, C.: Neural system combination for machine translation. In: ACL 2017, pp. 378–384 (2017)
Zhu, C., Yu, H., Cheng, S., Luo, W.: Language-aware interlingua for multilingual neural machine translation. In: ACL 2020, pp. 1650–1655 (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Yang, J. et al. (2021). Learning to Select Relevant Knowledge for Neural Machine Translation. In: Wang, L., Feng, Y., Hong, Y., He, R. (eds) Natural Language Processing and Chinese Computing. NLPCC 2021. Lecture Notes in Computer Science(), vol 13028. Springer, Cham. https://doi.org/10.1007/978-3-030-88480-2_7
Download citation
DOI: https://doi.org/10.1007/978-3-030-88480-2_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-88479-6
Online ISBN: 978-3-030-88480-2
eBook Packages: Computer ScienceComputer Science (R0)
