<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="derby-etal-2018-representation">
    <titleInfo>
        <title>Representation of Word Meaning in the Intermediate Projection Layer of a Neural Language Model</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Steven</namePart>
        <namePart type="family">Derby</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Paul</namePart>
        <namePart type="family">Miller</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Brian</namePart>
        <namePart type="family">Murphy</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Barry</namePart>
        <namePart type="family">Devereux</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2018-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Tal</namePart>
            <namePart type="family">Linzen</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Grzegorz</namePart>
            <namePart type="family">Chrupała</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Afra</namePart>
            <namePart type="family">Alishahi</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Brussels, Belgium</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Performance in language modelling has been significantly improved by training recurrent neural networks on large corpora. This progress has come at the cost of interpretability and an understanding of how these architectures function, making principled development of better language models more difficult. We look inside a state-of-the-art neural language model to analyse how this model represents high-level lexico-semantic information. In particular, we investigate how the model represents words by extracting activation patterns where they occur in the text, and compare these representations directly to human semantic knowledge.</abstract>
    <identifier type="citekey">derby-etal-2018-representation</identifier>
    <identifier type="doi">10.18653/v1/W18-5449</identifier>
    <location>
        <url>https://aclanthology.org/W18-5449</url>
    </location>
    <part>
        <date>2018-11</date>
        <extent unit="page">
            <start>362</start>
            <end>364</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Representation of Word Meaning in the Intermediate Projection Layer of a Neural Language Model
%A Derby, Steven
%A Miller, Paul
%A Murphy, Brian
%A Devereux, Barry
%Y Linzen, Tal
%Y Chrupała, Grzegorz
%Y Alishahi, Afra
%S Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP
%D 2018
%8 November
%I Association for Computational Linguistics
%C Brussels, Belgium
%F derby-etal-2018-representation
%X Performance in language modelling has been significantly improved by training recurrent neural networks on large corpora. This progress has come at the cost of interpretability and an understanding of how these architectures function, making principled development of better language models more difficult. We look inside a state-of-the-art neural language model to analyse how this model represents high-level lexico-semantic information. In particular, we investigate how the model represents words by extracting activation patterns where they occur in the text, and compare these representations directly to human semantic knowledge.
%R 10.18653/v1/W18-5449
%U https://aclanthology.org/W18-5449
%U https://doi.org/10.18653/v1/W18-5449
%P 362-364

Markdown (Informal)

[Representation of Word Meaning in the Intermediate Projection Layer of a Neural Language Model](https://aclanthology.org/W18-5449) (Derby et al., EMNLP 2018)

• Representation of Word Meaning in the Intermediate Projection Layer of a Neural Language Model (Derby et al., EMNLP 2018)

ACL

• Steven Derby, Paul Miller, Brian Murphy, and Barry Devereux. 2018. Representation of Word Meaning in the Intermediate Projection Layer of a Neural Language Model. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 362–364, Brussels, Belgium. Association for Computational Linguistics.