Investigating the Applicability and Limitations of Deep Learning for Book-Guided Speech Synthesis

This thesis introduces a novel computational approach towards the automated generation of multi-cast audiobooks using deep learning. Given a book's text as input, it synthesizes speech in unique voices for each of the book's characters. A cascaded approach comprising two stages is devised: a textual analysis stage and a speech synthesis stage. In the former, character models are determined and sequences of direct speech attributed to them. An attempt is made to analyze and classify the emotional sentiment of direct speech segments. Guided by this, in the second stage, realistic speech audio is synthesized for each character, taking into account the respective character models. Towards this, neural speech synthesis models are trained and combined with a natural language processing pipeline. By implementing the approach in a web-based demonstrator application, a proof-of-concept for the automatic generation of synthetic audiobooks is provided.