Audio samples from "Semi-Supervised Generative Modeling for Controllable Speech Synthesis"

This page contains a set of audio samples in support of the paper. All the utterances are unseen during training. Sections 1-3 demo controlling speaking rate, pitch variation (continuous labels), and affect (discrete labels), via semi-supervision of these factors, at multiple supervision levels. Sections 4-6 demo the transfer of controllability to unlabeled speakers, for whom we do not observe labels of the factors above (i.e., domain transfer).

Contents

In this section, we demonstrate the degree to which we are able to control the variation in speaking rate at different levels of supervision. Speaking rate is normalized to match the standard normal prior (z_s~N(μ=0, σ=1)).

In this section, we demonstrate the degree to which we are able to control the variation in F0 (fundamental frequency, a proxy for arousal or excitement) at different levels of supervision. F0 variation is normalized to match the standard normal prior (z_s~N(μ=0, σ=1)).

In this section, we demonstrate the degree to which we are able to control the variation in affect at different levels of supervision. Higher levels of supervision is required for controlling affect, compared to speaking rate and pitch variations. In general the variations are subtle, so as the upper bound we have also presented the fully supervised case (i.e., 100% supervision).

In this section, we demonstrate the degree to which speaking rate control at 10% supervision is able to generalize to speakers without speaking rate labels in training, which are colored blue.

In this section, we demonstrate the degree to which pitch variation control at 10% supervision is able to generalize to speakers without pitch variation labels in training, which are colored blue.

In this section, we demonstrate the degree to which affect control at 10% supervision is able to generalize to speakers without affect labels in training, which are colored blue.