TechRxiv
UQuAD1_1.0.pdf (2.14 MB)
Download file

UQuAD1.0: Development of an Urdu Question Answering Training Data for Machine Reading Comprehension

Download (2.14 MB)
preprint
posted on 2021-11-05, 16:46 authored by Samreen AhmedSamreen Ahmed, shakeel khoja

In recent years, low-resource Machine Reading Comprehension (MRC) has made significant progress, with models getting remarkable performance on various language datasets. However, none of these models have been customized for the Urdu language. This work explores the semi-automated creation of the Urdu Question Answering Dataset (UQuAD1.0) by combining machine-translated SQuAD with human-generated samples derived from Wikipedia articles and Urdu RC worksheets from Cambridge O-level books. UQuAD1.0 is a large-scale Urdu dataset intended for extractive machine reading comprehension tasks consisting of 49k question Answers pairs in question, passage, and answer format. In UQuAD1.0, 45000 pairs of QA were generated by machine translation of the original SQuAD1.0 and approximately 4000 pairs via crowdsourcing. In this study, we used two types of MRC models: rule-based baseline and advanced Transformer-based models. However, we have discovered that the latter outperforms the others; thus, we have decided to concentrate solely on Transformer-based architectures. Using XLMRoBERTa and multi-lingual BERT, we acquire an F1 score of 0.66 and 0.63, respectively.

History

Email Address of Submitting Author

sakazi@iba.edu.pk

ORCID of Submitting Author

0000-0002-5394-8298

Submitting Author's Institution

institute of business administration

Submitting Author's Country

  • Pakistan