Sentiment Analysis is a popular text classification task in natural language processing. It involves
developing algorithms or machine learning models to determine the sentiment or opinion expressed in
a piece of text. The results of this task can be used by business owners and product developers to
understand their consumers’ perceptions of their products. Asides from customer feedback and
product/service analysis, this task can be useful for social media monitoring (Martin et al., 2021).
One of the popular applications of sentiment analysis is for classifying and detecting the positive
and negative sentiments on movie reviews. Movie reviews enable movie producers to monitor the
performances of their movies (Abhishek et al., 2020) and enhance the decision of movie viewers to
know whether a movie is good enough and worth investing time to watch (Lakshmi Devi et al.,
2020). However, the task has been under-explored for African languages compared to their western
counterparts, ”high resource languages”, that are privileged to have received enormous attention
due to the large amount of available textual data. African languages fall under the category of the low
resource languages which are on the disadvantaged end because of the limited availability of data
that gives them a poor representation (Nasim & Ghani, 2020). Recently, sentiment analysis has
received attention on African languages in the Twitter domain for Nigerian (Muhammad et al., 2022)
and Amharic (Yimam et al., 2020) languages. However, there is no available corpus in the movie
domain. We decided to tackle the problem of unavailability of Yoru`ba´ data for movie sentiment
analysis by creating the first Yoru`ba´ sentiment corpus for Nollywood movie reviews. Also, we
develop sentiment classification models using state-of-the-art pre-trained language models like
mBERT (Devlin et al., 2019) and AfriBERTa (Ogueji et al., 2021).
more »
« less
This content will become publicly available on August 22, 2024
NollySenti: Leveraging Transfer Learning and Machine Translation for Nigerian Movie Sentiment Classification
Africa has over 2000 indigenous languages but
they are under-represented in NLP research due
to lack of datasets. In recent years, there have
been progress in developing labelled corpora
for African languages. However, they are often
available in a single domain and may not generalize
to other domains. In this paper, we focus on the task of sentiment classification for cross-domain
adaptation. We create a new dataset, NollySenti—based on the Nollywood movie
reviews for five languages widely spoken in Nigeria (English, Hausa, Igbo, Nigerian-Pidgin,
and Yorùbá). We provide an extensive empirical evaluation using classical machine learning
methods and pre-trained language models. Leveraging transfer learning, we compare the
performance of cross-domain adaptation from Twitter domain, and cross-lingual adaptation
from English language. Our evaluation shows that transfer from English in the same target
domain leads to more than 5% improvement in accuracy compared to transfer from Twitter
in the same language. To further mitigate the domain difference, we leverage machine translation
(MT) from English to other Nigerian languages, which leads to a further improvement
of 7% over cross-lingual evaluation. While MT to low-resource languages are often of low quality,
through human evaluation, we show that most of the translated sentences preserve the sentiment of the original English reviews.
more »
« less
- Award ID(s):
- 1704113
- NSF-PAR ID:
- 10470679
- Publisher / Repository:
- 61st Annual Meeting of the Association for Computational Linguistics (ACL’23)
- Date Published:
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Sentiment classification typically relies on a large amount of labeled data. In practice, the availability of labels is highly imbalanced among different languages, e.g., more English texts are labeled than texts in any other languages, which creates a considerable inequality in the quality of related information services received by users speaking different languages. To tackle this problem, cross-lingual sentiment classification approaches aim to transfer knowledge learned from one language that has abundant labeled examples (i.e., the source language, usually English) to another language with fewer labels (i.e., the target language). The source and the target languages are usually bridged through off-the-shelf machine translation tools. Through such a channel, cross-language sentiment patterns can be successfully learned from English and transferred into the target languages. This approach, however, often fails to capture sentiment knowledge specific to the target language, and thus compromises the accuracy of the downstream classification task. In this paper, we employ emojis, which are widely available in many languages, as a new channel to learn both the cross-language and the language-specific sentiment patterns. We propose a novel representation learning method that uses emoji prediction as an instrument to learn respective sentiment-aware representations for each language. The learned representations are then integrated to facilitate cross-lingual sentiment classification. The proposed method demonstrates state-of-the-art performance on benchmark datasets, which is sustained even when sentiment labels are scarce.more » « less
-
The regularity of devastating cyber-attacks has made cybersecurity a grand societal challenge. Many cybersecurity professionals are closely examining the international Dark Web to proactively pinpoint potential cyber threats. Despite its potential, the Dark Web contains hundreds of thousands of non-English posts. While machine translation is the prevailing approach to process non-English text, applying MT on hacker forum text results in mistranslations. In this study, we draw upon Long-Short Term Memory (LSTM), Cross-Lingual Knowledge Transfer (CLKT), and Generative Adversarial Networks (GANs) principles to design a novel Adversarial CLKT (A-CLKT) approach. A-CLKT operates on untranslated text to retain the original semantics of the language and leverages the collective knowledge about cyber threats across languages to create a language invariant representation without any manual feature engineering or external resources. Three experiments demonstrate how A-CLKT outperforms state-of-the-art machine learning, deep learning, and CLKT algorithms in identifying cyber-threats in French and Russian forums.more » « less
-
Communication tools such as email facilitate communication and collaboration between speakers of different languages, who use two primary strategies—English as a common language and machine translation (MT) tools—to help them overcome language barriers. However, each of these communication strategies creates its own challenges for cross-lingual communication. In this paper, we compare how people’s interpretations of an email sender’s social intention, and their evaluation of the email and the senders, differ when using a common language versus MT in email communication. We conducted an online experiment in which monolingual native English speakers read and rated request emails written by native English speakers, emails written by bilingual Chinese speakers in English, and emails written in Chinese then machine-translated into English. We found that participants interpreted the social intentions of the email sender less accurately for machine-translated emails than for emails written by non-native speakers in English. Participants also rated the senders and emails less positively overall for machine-translated emails compared to emails written by non-native speakers in English. Based on these findings, we suggest design possibilities that could better aid multilingual communication.more » « less
-
Language documentation encompasses translation, typically into the dominant high-resource language in the region where the target language is spoken. To make data accessible to a broader audience, additional translation into other high-resource languages might be needed. Working within a project documenting Kotiria, we explore the extent to which state-of-the-art machine translation (MT) systems can support this second translation – in our case from Portuguese to English. This translation task is challenging for multiple reasons: (1) the data is out-of-domain with respect to the MT system’s training data, (2) much of the data is conversational, (3) existing translations include non-standard and uncommon expressions, often reflecting properties of the documented language, and (4) the data includes borrowings from other regional languages. Despite these challenges, existing MT systems perform at a usable level, though there is still room for improvement. We then conduct a qualitative analysis and suggest ways to improve MT between high-resource languages in a language documentation setting.more » « less
