<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="kashani-motlagh-etal-2024-assessing">
    <titleInfo>
        <title>Assessing the Role of Imagery in Multimodal Machine Translation</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Nicholas</namePart>
        <namePart type="family">Kashani Motlagh</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Jim</namePart>
        <namePart type="family">Davis</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Jeremy</namePart>
        <namePart type="family">Gwinnup</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Grant</namePart>
        <namePart type="family">Erdmann</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Tim</namePart>
        <namePart type="family">Anderson</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the Ninth Conference on Machine Translation</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Barry</namePart>
            <namePart type="family">Haddow</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Tom</namePart>
            <namePart type="family">Kocmi</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Philipp</namePart>
            <namePart type="family">Koehn</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Christof</namePart>
            <namePart type="family">Monz</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Miami, Florida, USA</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>In Multimodal Machine Translation (MMT), the use of visual data has shown only marginal improvements compared to text-only models. Previously, the CoMMuTE dataset and associated metric were proposed to score models on tasks where the imagery is necessary to disambiguate between two possible translations for each ambiguous source sentence. In this work, we introduce new metrics within the CoMMuTE domain to provide deeper insights into image-aware translation models. Our proposed metrics differ from the previous CoMMuTE scoring method by 1) assessing the impact of multiple images on individual translations and 2) evaluating a model’s ability to jointly select each translation for each image context. Our results challenge the conventional views of poor visual comprehension capabilities of MMT models and show that models can indeed meaningfully interpret visual information, though they may not leverage it sufficiently in the final decision.</abstract>
    <identifier type="citekey">kashani-motlagh-etal-2024-assessing</identifier>
    <identifier type="doi">10.18653/v1/2024.wmt-1.130</identifier>
    <location>
        <url>https://aclanthology.org/2024.wmt-1.130</url>
    </location>
    <part>
        <date>2024-11</date>
        <extent unit="page">
            <start>1428</start>
            <end>1439</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Assessing the Role of Imagery in Multimodal Machine Translation
%A Kashani Motlagh, Nicholas
%A Davis, Jim
%A Gwinnup, Jeremy
%A Erdmann, Grant
%A Anderson, Tim
%Y Haddow, Barry
%Y Kocmi, Tom
%Y Koehn, Philipp
%Y Monz, Christof
%S Proceedings of the Ninth Conference on Machine Translation
%D 2024
%8 November
%I Association for Computational Linguistics
%C Miami, Florida, USA
%F kashani-motlagh-etal-2024-assessing
%X In Multimodal Machine Translation (MMT), the use of visual data has shown only marginal improvements compared to text-only models. Previously, the CoMMuTE dataset and associated metric were proposed to score models on tasks where the imagery is necessary to disambiguate between two possible translations for each ambiguous source sentence. In this work, we introduce new metrics within the CoMMuTE domain to provide deeper insights into image-aware translation models. Our proposed metrics differ from the previous CoMMuTE scoring method by 1) assessing the impact of multiple images on individual translations and 2) evaluating a model’s ability to jointly select each translation for each image context. Our results challenge the conventional views of poor visual comprehension capabilities of MMT models and show that models can indeed meaningfully interpret visual information, though they may not leverage it sufficiently in the final decision.
%R 10.18653/v1/2024.wmt-1.130
%U https://aclanthology.org/2024.wmt-1.130
%U https://doi.org/10.18653/v1/2024.wmt-1.130
%P 1428-1439

Markdown (Informal)

[Assessing the Role of Imagery in Multimodal Machine Translation](https://aclanthology.org/2024.wmt-1.130) (Kashani Motlagh et al., WMT 2024)

• Assessing the Role of Imagery in Multimodal Machine Translation (Kashani Motlagh et al., WMT 2024)

ACL

• Nicholas Kashani Motlagh, Jim Davis, Jeremy Gwinnup, Grant Erdmann, and Tim Anderson. 2024. Assessing the Role of Imagery in Multimodal Machine Translation. In Proceedings of the Ninth Conference on Machine Translation, pages 1428–1439, Miami, Florida, USA. Association for Computational Linguistics.