<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="melamud-etal-2017-simple">
    <titleInfo>
        <title>A Simple Language Model based on PMI Matrix Approximations</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Oren</namePart>
        <namePart type="family">Melamud</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Ido</namePart>
        <namePart type="family">Dagan</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Jacob</namePart>
        <namePart type="family">Goldberger</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2017-09</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Martha</namePart>
            <namePart type="family">Palmer</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Rebecca</namePart>
            <namePart type="family">Hwa</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sebastian</namePart>
            <namePart type="family">Riedel</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Copenhagen, Denmark</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>In this study, we introduce a new approach for learning language models by training them to estimate word-context pointwise mutual information (PMI), and then deriving the desired conditional probabilities from PMI at test time. Specifically, we show that with minor modifications to word2vec’s algorithm, we get principled language models that are closely related to the well-established Noise Contrastive Estimation (NCE) based language models. A compelling aspect of our approach is that our models are trained with the same simple negative sampling objective function that is commonly used in word2vec to learn word embeddings.</abstract>
    <identifier type="citekey">melamud-etal-2017-simple</identifier>
    <identifier type="doi">10.18653/v1/D17-1198</identifier>
    <location>
        <url>https://aclanthology.org/D17-1198</url>
    </location>
    <part>
        <date>2017-09</date>
        <extent unit="page">
            <start>1860</start>
            <end>1865</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T A Simple Language Model based on PMI Matrix Approximations
%A Melamud, Oren
%A Dagan, Ido
%A Goldberger, Jacob
%Y Palmer, Martha
%Y Hwa, Rebecca
%Y Riedel, Sebastian
%S Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing
%D 2017
%8 September
%I Association for Computational Linguistics
%C Copenhagen, Denmark
%F melamud-etal-2017-simple
%X In this study, we introduce a new approach for learning language models by training them to estimate word-context pointwise mutual information (PMI), and then deriving the desired conditional probabilities from PMI at test time. Specifically, we show that with minor modifications to word2vec’s algorithm, we get principled language models that are closely related to the well-established Noise Contrastive Estimation (NCE) based language models. A compelling aspect of our approach is that our models are trained with the same simple negative sampling objective function that is commonly used in word2vec to learn word embeddings.
%R 10.18653/v1/D17-1198
%U https://aclanthology.org/D17-1198
%U https://doi.org/10.18653/v1/D17-1198
%P 1860-1865

Markdown (Informal)

[A Simple Language Model based on PMI Matrix Approximations](https://aclanthology.org/D17-1198) (Melamud et al., EMNLP 2017)

• A Simple Language Model based on PMI Matrix Approximations (Melamud et al., EMNLP 2017)

ACL

• Oren Melamud, Ido Dagan, and Jacob Goldberger. 2017. A Simple Language Model based on PMI Matrix Approximations. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1860–1865, Copenhagen, Denmark. Association for Computational Linguistics.