Lyrics generation constrained by tone, melody and imagery

Abstract

In recent years, lyrics generation has caught the eye of industry because of the commercial potential of producing trendy content quickly at low cost. At the same time, lyrics generation poses intellectual challenges as well. This thesis presents a computational method for the automatic generation of lyrics with consideration of linguistic, melodic and cognitive constraints. The method is based on the well-known concept of scansion from poetry analysis, but is novel in applying it to lyrics generation. As the lyrics I aim to create are in Mandarin, the method has to achieve melody--tone matching. Different representations of Mandarin tones are possible, depending on how many different tone heights are defined, from 3 to 6 heights. Experimentally, I find that the lyrics generator that is trained on 3-height scansion performs best. The cognition constraint I consider here is imagery, which I manipulate by calculating the degree of basic-level category-ness (BLCness) of every possible concept. I use bilingual synthesized semantic features in order to predict BLCness of concepts at a large scale, based on Rosch's original definition. For the generation of synthesised features, a pre-trained machine translator is finetuned using human semantic norms. In the BLC chapter of this thesis, I can experimentally confirm Rosch's idea of cue validity for the first time. I present results in English, where my method outperforms the best currently known method, and in Mandarin, where it is the first such method. To realise the final system, after determining the optimal setting for several parameters such as kind of scansion representation and note length threshold, I use statistical translation via mBART, and a specially created parallel corpus of lyrics and pseudo-melodies. My scansion-based lyrics generator outputs lyrics for any given melody without requiring a parallel melody-lyrics dataset, while avoiding several problems of current lyrics generators with sentence length and rhyme schemes. My way of training the lyrics generator is by combining BLCness into the loss function. This thesis presents both novel automatic and several human-based evaluations. To my surprise, the best system uses fewer, rather than more high-BLC concepts. I hypothesise that this has to do with a dislike of cliche in song lyrics.