Pipeline-diacritizer

Multi-components system for automatic Arabic diacritics restoration Warning: This project has been deprecated due to its bad design and problems. This repository is only left because it has been referenced from the corresponding paper. For a better and newer project, check Multilevel Diacritizer . About This tool is a command-line application written in Python 3 that automatically add diacritics to raw undiacritized Arabic text. To accomplish this task, it uses several techniques: Deep Learning, rule-based and statistical corrections. The deep learning part was implemented using Tensorflow. It was released as a support for the research paper: "Multi-components system for automatic Arabic diacritization" which was presented in the ECIR2020 conference. Installation This tool is available as a Python 3 package pipeline-diacritizer installable through pip. For installation instructions check the Download and installation wiki page. Functions This tool has 4 main functions: preprocessing of the data, training on the processed data, testing, and restoring the diacritics of an undiacritized text. In addition, it can calculates some statistics on a given dataset and