CCG Supertagging Using Morphological and Dependency Syntax Information | SpringerLink
Skip to main content
Springer Nature Link
Log in

CCG Supertagging Using Morphological and Dependency Syntax Information

  • Conference paper
  • First Online:
Computational Linguistics and Intelligent Text Processing (CICLing 2019)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13451))

  • 422 Accesses

Abstract

After presenting a new CCG supertagging algorithm based on morphological and dependency syntax information, we use this algorithm to create a CCG French Tree Bank corpus (20,261 sentences) based on the FTB corpus by Abeillé et al. We then use this corpus, as well as the Groningen Tree Bank corpus for the English language, to train a new BiLSTM+CRF neural architecture that uses (a) morphosyntactic input features and (b) feature correlations as input features. We show experimentally that for an inflected language like French, dependency syntax information allows significant improvement of the accuracy of the CCG supertagging task, when using deep learning techniques.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
JPY 3498
Price includes VAT (Japan)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
JPY 11439
Price includes VAT (Japan)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
JPY 14299
Price includes VAT (Japan)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Abeillé, A., Clément, L., Toussenel, F.: Building a treebank for French. In: Treebanks: Building and Using Parsed Corpora, pp. 165–187. Kluwer (2003)

    Google Scholar 

  2. Ambati, B.R., Deoskar, T., Steedman, M.: Shift-reduce CCG parsing using neural network models. In: Proceedings of NAACL 2016, pp. 447–453 (2016)

    Google Scholar 

  3. Baldi, P., Brunak, S., Frasconi, P., Pollastri, G., Soda, G.: Bidirectional dynamics for protein secondary structure prediction. In: Sun, R., Giles, C.L. (eds.) Sequence Learning. LNCS (LNAI), vol. 1828, pp. 80–104. Springer, Heidelberg (2000). https://doi.org/10.1007/3-540-44565-X_5

    Chapter  Google Scholar 

  4. Bengio, Y., Frasconi, P., Simard, P.: The problem of learning long-term dependencies in recurrent networks. In: IEEE International Conference on Neural Networks 1993, pp. 1183–1188. IEEE (1993)

    Google Scholar 

  5. Bos, J., Basile, V., Evang, K., Venhuizen, N.J., Bjerva, J.: The Groningen meaning bank. In: Ide, N., Pustejovsky, J. (eds.) Handbook of Linguistic Annotation, vol. 2, pp. 463–496. Springer, Dordrecht (2017). https://doi.org/10.1007/978-94-024-0881-2_18

    Chapter  Google Scholar 

  6. Candito, M., Crabbé, B., Denis, P.: Statistical French dependency parsing: treebank conversion and first results. In: Proceedings of LREC 2010, pp. 1840–1847 (2010)

    Google Scholar 

  7. Candito, M., Crabbé, B., Denis, P., Guérin, F.: Analyse syntaxique du français: des constituants aux dépendances. In: Proceedings of TALN 2009 (2009)

    Google Scholar 

  8. Cho, K., Van Merriënboer, B., Bahdanau, D., Bengio, Y.: On the properties of neural machine translation: encoder-decoder approaches. arXiv:1409.1259

  9. Chollet, F.: Deep Learning with Python. Manning Publications (2018)

    Google Scholar 

  10. Clark, S., Curran, J.R.: Wide-coverage efficient statistical parsing with CCG and log-linear models. Comput. Linguist. 33(4), 493–552 (2007)

    Article  MATH  Google Scholar 

  11. Collobert, R.: Deep learning for efficient discriminative parsing. In: Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, pp. 224–232 (2011)

    Google Scholar 

  12. De Marneffe, M.C., MacCartney, B., Manning, C.D., et al.: Generating typed dependency parses from phrase structure parses. In: Proceedings of LREC 2006, vol. 6, pp. 449–454 (2006)

    Google Scholar 

  13. Fauconnier, J.P.: French word embeddings (2015). http://fauconnier.github.io

  14. Gers, F.A., Schmidhuber, J., Cummins, F.: Learning to forget: continual prediction with LSTM. In: Proceedings of ICANN 1999. IET (1999)

    Google Scholar 

  15. Goller, C., Kuchler, A.: Learning task-dependent distributed representations by backpropagation through structure. Neural Netw. 1, 347–352 (1996)

    Google Scholar 

  16. Graves, A., Schmidhuber, J.: Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Netw. 18(5–6), 602–610 (2005)

    Article  Google Scholar 

  17. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  18. Hockenmaier, J., Steedman, M.: CCGbank: a corpus of CCG derivations and dependency structures extracted from the Penn Treebank. Comput. Linguist. 33(3), 355–396 (2007)

    Article  MATH  Google Scholar 

  19. Honnibal, M., Johnson, M.: An improved non-monotonic transition system for dependency parsing. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1373–1378 (2015)

    Google Scholar 

  20. Honnibal, M., Kummerfeld, J.K., Curran, J.R.: Morphological analysis can improve a CCG parser for English. In: Proceedings of Coling 2010, pp. 445–453 (2010)

    Google Scholar 

  21. Huang, Z., Xu, W., Yu, K.: Bidirectional LSTM-CRF models for sequence tagging. arXiv:1508.01991

  22. Kadari, R., Zhang, Y., Zhang, W., Liu, T.: CCG supertagging via Bidirectional LSTM-CRF neural architecture. Neurocomputing 283, 31–37 (2018)

    Article  Google Scholar 

  23. Kadari, R., Zhang, Y., Zhang, W., Liu, T.: CCG supertagging with bidirectional long short-term memory networks. Nat. Lang. Eng. 24(1), 77–90 (2018)

    Article  Google Scholar 

  24. Lafferty, J., McCallum, A., Pereira, F.C.: Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: Proceedings of ICML 2001, pp. 282–289 (2001)

    Google Scholar 

  25. Lewis, M., Lee, K., Zettlemoyer, L.: LSTM CCG parsing. In: Proceedings of NAACL 2016, pp. 221–231 (2016)

    Google Scholar 

  26. Lewis, M., Steedman, M.: Improved CCG parsing with semi-supervised supertagging. Trans. ACL 2, 327–338 (2014)

    Google Scholar 

  27. Ma, X., Hovy, E.: End-to-end sequence labeling via bi-directional LSTM-CNNS-CRF. arXiv:1603.01354

  28. McDonald, R., Pereira, F., Ribarov, K., Hajič, J.: Non-projective dependency parsing using spanning tree algorithms. In: Proceedings of EMNLP 2005, pp. 523–530 (2005)

    Google Scholar 

  29. Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Proceedings of NIPS 2013, pp. 3111–3119 (2013)

    Google Scholar 

  30. Nivre, J., Hall, J., Nilsson, J.: MaltParser: a data-driven parser-generator for dependency parsing. In: Proceedings of LREC 2006, pp. 2216–2219 (2006)

    Google Scholar 

  31. Pascanu, R., Mikolov, T., Bengio, Y.: On the difficulty of training recurrent neural networks. In: International Conference on Machine Learning, pp. 1310–1318 (2013)

    Google Scholar 

  32. Pennington, J., Socher, R., Manning, C.D.: Glove: global vectors for word representation. In: Proceedings of EMNLP 2014, pp. 1532–1543 (2014)

    Google Scholar 

  33. Schuster, M., Paliwal, K.K.: Bidirectional recurrent neural networks. IEEE Trans. Signal Process. 45(11), 2673–2681 (1997)

    Article  Google Scholar 

  34. Steedman, M.: Surface Structure and Interpretation. MIT Press, Cambridge (1996)

    Google Scholar 

  35. Steedman, M.: The Syntactic Process. MIT Press, Cambridge (2000)

    MATH  Google Scholar 

  36. Steedman, M., Baldridge, J.: Combinatory categorial grammar. In: Non-transformational Syntax: Formal and Explicit Models of Grammar, pp. 181–224. Wiley-Blackwell (2011)

    Google Scholar 

  37. Tieleman, T., Hinton, G.: Lecture 6.5-rmsprop: divide the gradient by a running average of its recent magnitude. COURSERA: Neural Netw. Mach. Learn. 4(2), 26–31 (2012)

    Google Scholar 

  38. Turian, J., Ratinov, L., Bengio, Y.: Word representations: a simple and general method for semi-supervised learning. In: Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pp. 384–394 (2010)

    Google Scholar 

  39. Vaswani, A., Bisk, Y., Sagae, K., Musa, R.: Supertagging with LSTMs. In: Proceedings of NAACL 2016, pp. 232–237 (2016)

    Google Scholar 

  40. Wu, H., Zhang, J., Zong, C.: A dynamic window neural network for CCG supertagging. In: Proceedings of AAAI 2017, pp. 3337–3343 (2017)

    Google Scholar 

  41. Xu, W.: LSTM shift-reduce CCG parsing. In: Proceedings of EMNLP 2016, pp. 1754–1764 (2016)

    Google Scholar 

  42. Xu, W., Auli, M., Clark, S.: CCG supertagging with a recurrent neural network. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics, vol. 2, pp. 250–255 (2015)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. IMT Atlantique & UMR CNRS 6285 Lab-STICC, CS 83818, 29238, Brest Cedex 3, France

    Luyện Ngọc Lê & Yannis Haralambous

Authors
  1. Luyện Ngọc Lê
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yannis Haralambous
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yannis Haralambous .

Editor information

Editors and Affiliations

  1. Instituto Politécnico Nacional, Mexico City, Mexico

    Alexander Gelbukh

Rights and permissions

Reprints and permissions

Copyright information

© 2023 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Lê, L.N., Haralambous, Y. (2023). CCG Supertagging Using Morphological and Dependency Syntax Information. In: Gelbukh, A. (eds) Computational Linguistics and Intelligent Text Processing. CICLing 2019. Lecture Notes in Computer Science, vol 13451. Springer, Cham. https://doi.org/10.1007/978-3-031-24337-0_43

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-24337-0_43

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-24336-3

  • Online ISBN: 978-3-031-24337-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation