Direct Synthesis

Overview

When it comes to time-sensitive content delivery, using streaming synthesis is the fastest way to achieve the lowest time-to-first-sound. For documentation on streaming synthesis, see "Stream a clip".

For some applications, streaming is not an option. In these cases, the fastest time-to-first-sound can be achieved by sending synchronous requests directly to our synthesis servers. We call this direct synthesis. Summarized below are the steps required to use this API:

• Make a request to the direct synthesis endpoint. (Note: this is not the usual https://app.resemble.ai/api/... URL.)
• Decode the base64 “audio_content” attribute sent back in the response.
• Use the audio data.

HTTP Request

1 2 3 4 5 6 7 8 9 10 11 12 curl --request POST "YOUR_SYNTH_ENDPOINT" -H "Authorization: Bearer YOUR_API_TOKEN" -H "Content-Type: application/json" -H "Accept-Encoding: gzip, deflate, br" --data '{ "voice_uuid": <Voice to synthesize in>, "project_uuid": <Project to save to>, "title": <Title of the clip>, "data": <Text to synthesize>, "precision": "MULAW|PCM_16|PCM_24|PCM_32 (default)" "output_format": "mp3|wav (default)" }'

Request Headers

Header	Value	Description
Authorization	Bearer YOUR_API_TOKEN	API token can be obtained by logging into the Resemble web application and navigating to the API section.
Accept-Encoding	gzip, deflate, br	Either one of gzip, deflate, or br depending on the decompression algorithms your application supports. Omitting the Accept-Encoding header will disable compression.

Request Body

Attribute	Type	Description
voice_uuid	string	The voice to synthesize the text in.
project_uuid	string	The project to save the data to.
title	string	The title of the clip. This is optional, default is to name the clip Direct Synthesis {some-uuid}
data	string	The text or SSML to synthesize.
precision	string	The bit-depth of the generated wav file (if using wav as the response type). Either MULAW, PCM_16, PCM_24, or PCM_32 (default).
output_format	string	The output format of the produced audio. Either wav, or mp3.
sample_rate	integer	The sample rate of the produced audio. Either 8000, 16000, 22050, 32000, or 44100

HTTP Response

{
  "audio_content": <base64 encoded string of the raw audio bytes>,
  "phoneme_timestamps": {
    "phonemes": <string of phonemes>,
    "end_times": float[],
    "phoneme_chars": char[]
  },
  issues: string[],
  success: boolean
}

Response Body

Attribute	Type	Description
audio_content	string	Base64 encoded string. When decoded it will contain the byte array containing the audio.
phoneme_timestamps	object	Object containing phoneme_timestamp information. See section below for further information.
Issues	string[]	Any issues pertaining to the synthesis response.
Success	boolean	True if the response was successful, false otherwise.

Audio Timestamps Object

Attribute	Type	Description
graph_times	string	A string containing all the phonemes pertaining to the synthesized audio.
phon_times	float[]	An array of floats mapping 1 to 1 with the phoneme_chars. Each index represents the end time in the audio of the phoneme character at the same index in the phoneme_chars array.
phon_chars	char[]	An array of characters mapping 1 to 1 with the end_times array.
graph_chars	char[]	An array of characters mapping 1 to 1 with the end_times array.