No More Cost in Translation: Validating Open-Source Machine Translation for Quantitative Text Analysis

Hauke Licht (), Ronja Sczepanski, Moritz Laurer and Ayjeren Bekmuratovna
Additional contact information
Hauke Licht: University of Cologne
Ronja Sczepanski: Sciences Po Paris
Moritz Laurer: Hugging Face; Vrije Universiteit Amsterdam
Ayjeren Bekmuratovna: DHL

No 276, ECONtribute Discussion Papers Series from University of Bonn and University of Cologne, Germany

Abstract: As more and more scholars apply computational text analysis methods to multilingual corpora, machine translation has become an indispensable tool. However, relying on commercial services for machine translation, such as Google Translate or DeepL, limits reproducibility and can be expensive. This paper assesses the viability of a reproducible and affordable alternative: free and open-source machine translation models. We ask whether researchers who use an open-source model instead of a commercial service for machine translation would obtain substantially different measurements from their multilingual corpora. We address this question by replicating and extending an influential study by de Vries et al. (2018) on the use of machine translation in cross-lingual topic modeling, and an original study of its use in supervised text classification with Transformer-based classifiers. We find only minor differences between the measurements generated by these methods when applied to corpora translated with open-source models and commercial services, respectively. We conclude that “free” machine translation is a very valuable addition to researchers’ multilingual text analysis toolkit. Our study adds to a growing body of work on multilingual text analysis methods and has direct practical implications for applied researchers.

Keywords: machine translation; multilingual topic modeling; multilingual Transformers (search for similar items in EconPapers)
JEL-codes: C45 (search for similar items in EconPapers)
Pages: 46 pages
Date: 2024-02
New Economics Papers: this item is included in nep-ain and nep-big
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.econtribute.de/RePEc/ajk/ajkdps/ECONtribute_276_2024.pdf First version, 2024 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:ajk:ajkdps:276

Access Statistics for this paper

More papers in ECONtribute Discussion Papers Series from University of Bonn and University of Cologne, Germany Niebuhrstrasse 5, 53113 Bonn, Germany.
Bibliographic data for series maintained by ECONtribute Office ().