Automatic Classification and Comparison of Words by Difficulty | SpringerLink
Skip to main content
Springer Nature Link
Log in

Automatic Classification and Comparison of Words by Difficulty

  • Conference paper
  • First Online:
Neural Information Processing (ICONIP 2020)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1332))

Included in the following conference series:

  • 2539 Accesses

Abstract

Vocabulary knowledge is essential for both native and foreign language learning. Classifying words by difficulty helps students develop better in different stages of study and gives teachers the standard to adhere to when preparing tutorials. However, classifying word difficulty is time-consuming and labor-intensive. In this paper, we propose to classify and compare the word difficulty by analyzing multi-faceted features, including intra-word, syntactic and semantic features. The results show that our method is robust against different language environments.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
JPY 3498
Price includes VAT (Japan)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
JPY 11439
Price includes VAT (Japan)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
JPY 14299
Price includes VAT (Japan)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    The Corpus of Contemporary American English: https://www.english-corpora.org/coca/.

  2. 2.

    CEFR defines 6 difficulty levels {A1, A2, B1, B2, C1, C2} where A1 represents the minimum difficulty and C2 represents the highest difficulty.

  3. 3.

    https://github.com/f-e-l-i-x/deuPD.

  4. 4.

    https://github.com/LoraineYoko/word_difficulty.

  5. 5.

    https://www.twinword.com/api/language-scoring.php.

References

  1. Little, D.: The common European framework of reference for languages: a research agenda. Lang. Teach. 44(3), 381–393 (2011)

    Article  Google Scholar 

  2. Breland, H.M.: Word frequency and word difficulty: a comparison of counts in four corpora. Psychol. Sci. 7(2), 96–99 (1996)

    Article  Google Scholar 

  3. Hiebert, E., Scott, J., Castaneda, R., Spichtig, A.: An analysis of the features of words that influence vocabulary difficulty. Educ. Sci. 9(1), 8 (2019)

    Article  Google Scholar 

  4. Koirala, C.: The word frequency effect on second language vocabulary learning. In: Critical CALL-Proceedings of the 2015 EUROCALL Conference, Padova, Italy, p. 318. Research-publishing.net (2015)

    Google Scholar 

  5. Culligan, B.: A comparison of three test formats to assess word difficulty. Lang. Test. 32(4), 503–520 (2015)

    Article  Google Scholar 

  6. Schuster, S., Manning, C.D.: Enhanced English universal dependencies: an improved representation for natural language understanding tasks. In: LREC, Portorož, Slovenia, pp. 23–28 (2016)

    Google Scholar 

  7. Toutanova, K., Klein, D., Manning, C.D., Singer, Y.: Feature-rich part-of-speech tagging with a cyclic dependency network. In: Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology, vol. 1, pp. 173–180. Association for Computational Linguistics (2003)

    Google Scholar 

  8. Evan, S.: The New York Times Annotated Corpus LDC2008T19. DVD. Linguistic Data Consortium, Philadelphia (2008)

    Google Scholar 

  9. Lahiri, S.: Complexity of word collocation networks: a preliminary structural analysis. In: Proceedings of the Student Research Workshop at the 14th Conference of the European Chapter of the Association for Computational Linguistics, pp. 96–105. Association for Computational Linguistics, Gothenburg, April 2014. http://www.aclweb.org/anthology/E14-3011

  10. Kominek, J., Black, A.W.: The CMU arctic speech databases. In: Proceedings of the 5th ISCA Speech Synthesis Workshop (SSW5), Pittsburgh, PA, pp. 223–224 (2004)

    Google Scholar 

  11. Koehn, P.: Europarl: a parallel corpus for statistical machine translation. In: MT Summit, vol. 5, pp. 79–86. Citeseer (2005)

    Google Scholar 

  12. Manning, C., Surdeanu, M., Bauer, J., Finkel, J., Bethard, S., McClosky, D.: The Stanford Corenlp natural language processing toolkit. In: Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pp. 55–60 (2014)

    Google Scholar 

  13. Nakanishi, K., Kobayashi, N., Shiina, H., Kitagawa, F.: Estimating word difficulty using semantic descriptions in dictionaries and web data. In: 2012 IIAI International Conference on Advanced Applied Informatics, pp. 324–329. IEEE (2012)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai, China

    Shengyao Zhang & Qi Jia

  2. Leyan Technologies, Shanghai, China

    Libin Shen & Yinggong Zhao

Authors
  1. Shengyao Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qi Jia
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Libin Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yinggong Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shengyao Zhang .

Editor information

Editors and Affiliations

  1. Department of AI, Ping An Life, Shenzhen, China

    Haiqin Yang

  2. Faculty of Information Technology, King Mongkut's Institute of Technology Ladkrabang, Bangkok, Thailand

    Kitsuchart Pasupa

  3. City University of Hong Kong, Kowloon, Hong Kong

    Andrew Chi-Sing Leung

  4. Department of Computer Science and Engineering, Hong Kong University of Science and Technology, Hong Kong, Hong Kong

    James T. Kwok

  5. School of Information Technology, King Mongkut's University of Technology Thonburi, Bangkok, Thailand

    Jonathan H. Chan

  6. The Chinese University of Hong Kong, New Territories, Hong Kong

    Irwin King

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhang, S., Jia, Q., Shen, L., Zhao, Y. (2020). Automatic Classification and Comparison of Words by Difficulty. In: Yang, H., Pasupa, K., Leung, A.CS., Kwok, J.T., Chan, J.H., King, I. (eds) Neural Information Processing. ICONIP 2020. Communications in Computer and Information Science, vol 1332. Springer, Cham. https://doi.org/10.1007/978-3-030-63820-7_72

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-63820-7_72

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-63819-1

  • Online ISBN: 978-3-030-63820-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation