SP Module 6 Speech Synthesis – Waveform Generation and Connected Speech

Diphone

Phones are not a suitable unit for waveform concatenation, so we used diphones, which capture co-articulation.

Diphone starts at the middle of one phone and ends at the middle of the other.

Coarticulation is the overlapping of adjacent articulations or the influence of the target phoneme on surrounding phonemes. Middles of phones are more stable in their spectral properties than the edges, because of coarticulation. So, concatenating diphones should lead to smoother joins

Waveform concatenation

Concatenation of waveforms is a simple way of making synthetic speech, but we need to take care about how we do it.

1. discontinuity cause pops
2. periodicity alignment cause glitches

Overlap-add

Cross-fading between two waveforms is an effective way to avoid some of the artefacts of concatenation.

Pitch period

This fundamental building block of speech waveforms offers a route to source-filter separation in the time domain.

Overlap of pitch period or impulse signal is observed.

extract pitch period (with taper window) for each pitch mark, and we make the time for each pitch period twice the T0T_0T0​.

overlap to get the reconstruction signal similar to the original one. the whole process is called copy sentences.

TD-PSOLA

Applying overlap-add techniques to pitch period waveforms allows the modification of F0 and duration without changing the phone identity.

Time-domain pitch-synchronous overlap-and-add

Pitch period closer to each other

Pitch period far apart from each other

make a copy of one pitch period and insert to the sequence.

delete one pitch period

Diphone synthesis:

• One recording of every diphone (small database)
• Use signal processing methods to change F0, duration, and smooth joins to match linguistic specification
• e.g. TD-PSOLA

Unit Selection

Unit selection:

• Record a large naturalistic database
• Select diphone units based on closeness to the linguistic specification
• If the database has enough variation, don’t worry (too much) about signal processing!

Choice of units to concatenate depends on:

• Target cost: how well the unit matches the linguistic specification
• Join cost: how well edges of the units match

Convolution

A non-mathematical illustration of the equivalence of convolution (in the time domain), multiplication of magnitude spectra, and addition of log magnitude spectra.

Summary