Introducing ABENA: BERT Natural Language Processing for Twi

Fig. 1: We named our main model ABENA — A BERT Now in Akan

Introduction

Fig. 2: Our fastText (subword word2vec) Twi embedding model screenshot from a previous article

Motivation

Fig. 3: Illustrating the key idea behind transfer learning — instead of learning things for scratch, prior knowledge and experience should be shared and used to make the current task easier. Here, we see that learning to play the drum is easier if one already played the piano. Image from “Transfer Learning for NLP” [https://www.manning.com/books/transfer-learning-for-natural-language-processing]

ABENA Twi BERT Models

Fig.4: Convergence info for ABENA models. All models were trained on a single Tesla K80 GPU on an NC6 Azure VM instance.
Fig.5: Convergence info for DistilABENA models. All models were trained on a single Tesla K80 GPU on an NC6 Azure VM instance.

BAKO Twi BERT Model

Fig. 6: (BAKO) We also investigate training BERT models from scratch, yielding BAKO — BERT with Akan Knowledge Only. The Twi word “Bako” or “Baako” means “One”.
Fig.7: Convergence info for RoBAKO models trained from scratch. All models were trained on a single Tesla K80 GPU on an NC6 Azure VM instance.

Simple Sentiment Analysis/Classification Example

Fig.8: Description of all the models we trained and shared in this work.
Fig. 9: Simple Sentiment Analysis Example Dataset

Limitations and Ongoing/Future Work

Join Us?

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Paul Azunre

Paul Azunre

Paul Azunre holds a PhD in Computer Science from MIT and has served as a Principal Investigator on several DARPA programs. He founded Algorine & Ghana NLP