Lesan -- Machine Translation for Low Resource Languages
A. Hadgu, A. Aregawi, and A. Beaudoin. (2021)cite arxiv:2112.08191Comment: 4 pages, 2 figures, 35th Conference on Neural Information Processing Systems (NeurIPS 2021) demonstrations track.
Abstract
Millions of people around the world can not access content on the Web because
most of the content is not readily available in their language. Machine
translation (MT) systems have the potential to change this for many languages.
Current MT systems provide very accurate results for high resource language
pairs, e.g., German and English. However, for many low resource languages, MT
is still under active research. The key challenge is lack of datasets to build
these systems. We present Lesan, an MT system for low resource languages. Our
pipeline solves the key bottleneck to low resource MT by leveraging online and
offline sources, a custom OCR system for Ethiopic and an automatic alignment
module. The final step in the pipeline is a sequence to sequence model that
takes parallel corpus as input and gives us a translation model. Lesan's
translation model is based on the Transformer architecture. After constructing
a base model, back translation, is used to leverage monolingual corpora.
Currently Lesan supports translation to and from Tigrinya, Amharic and English.
We perform extensive human evaluation and show that Lesan outperforms
state-of-the-art systems such as Google Translate and Microsoft Translator
across all six pairs. Lesan is freely available and has served more than 10
million translations so far. At the moment, there are only 217 Tigrinya and
15,009 Amharic Wikipedia articles. We believe that Lesan will contribute
towards democratizing access to the Web through MT for millions of people.
Description
Lesan -- Machine Translation for Low Resource Languages
%0 Generic
%1 hadgu2021lesan
%A Hadgu, Asmelash Teka
%A Aregawi, Abel
%A Beaudoin, Adam
%D 2021
%K amharic lesan machinetranslation mt myown tigrinya
%T Lesan -- Machine Translation for Low Resource Languages
%U http://arxiv.org/abs/2112.08191
%X Millions of people around the world can not access content on the Web because
most of the content is not readily available in their language. Machine
translation (MT) systems have the potential to change this for many languages.
Current MT systems provide very accurate results for high resource language
pairs, e.g., German and English. However, for many low resource languages, MT
is still under active research. The key challenge is lack of datasets to build
these systems. We present Lesan, an MT system for low resource languages. Our
pipeline solves the key bottleneck to low resource MT by leveraging online and
offline sources, a custom OCR system for Ethiopic and an automatic alignment
module. The final step in the pipeline is a sequence to sequence model that
takes parallel corpus as input and gives us a translation model. Lesan's
translation model is based on the Transformer architecture. After constructing
a base model, back translation, is used to leverage monolingual corpora.
Currently Lesan supports translation to and from Tigrinya, Amharic and English.
We perform extensive human evaluation and show that Lesan outperforms
state-of-the-art systems such as Google Translate and Microsoft Translator
across all six pairs. Lesan is freely available and has served more than 10
million translations so far. At the moment, there are only 217 Tigrinya and
15,009 Amharic Wikipedia articles. We believe that Lesan will contribute
towards democratizing access to the Web through MT for millions of people.
@misc{hadgu2021lesan,
abstract = {Millions of people around the world can not access content on the Web because
most of the content is not readily available in their language. Machine
translation (MT) systems have the potential to change this for many languages.
Current MT systems provide very accurate results for high resource language
pairs, e.g., German and English. However, for many low resource languages, MT
is still under active research. The key challenge is lack of datasets to build
these systems. We present Lesan, an MT system for low resource languages. Our
pipeline solves the key bottleneck to low resource MT by leveraging online and
offline sources, a custom OCR system for Ethiopic and an automatic alignment
module. The final step in the pipeline is a sequence to sequence model that
takes parallel corpus as input and gives us a translation model. Lesan's
translation model is based on the Transformer architecture. After constructing
a base model, back translation, is used to leverage monolingual corpora.
Currently Lesan supports translation to and from Tigrinya, Amharic and English.
We perform extensive human evaluation and show that Lesan outperforms
state-of-the-art systems such as Google Translate and Microsoft Translator
across all six pairs. Lesan is freely available and has served more than 10
million translations so far. At the moment, there are only 217 Tigrinya and
15,009 Amharic Wikipedia articles. We believe that Lesan will contribute
towards democratizing access to the Web through MT for millions of people.},
added-at = {2021-12-30T19:46:20.000+0100},
author = {Hadgu, Asmelash Teka and Aregawi, Abel and Beaudoin, Adam},
biburl = {https://www.bibsonomy.org/bibtex/2e2bfb75d26f841261ffa6d7b64bf7652/asmelash},
description = {Lesan -- Machine Translation for Low Resource Languages},
interhash = {2f0e00e4ab0f718f94abbe59e8c26558},
intrahash = {e2bfb75d26f841261ffa6d7b64bf7652},
keywords = {amharic lesan machinetranslation mt myown tigrinya},
note = {cite arxiv:2112.08191Comment: 4 pages, 2 figures, 35th Conference on Neural Information Processing Systems (NeurIPS 2021) demonstrations track},
timestamp = {2021-12-30T19:46:20.000+0100},
title = {Lesan -- Machine Translation for Low Resource Languages},
url = {http://arxiv.org/abs/2112.08191},
year = 2021
}