Low Latency Synthesis (Recommended)

Overview

When it comes to time-sensitive content delivery, using streaming synthesis is the fastest way to achieve the lowest time-to-first-sound. For documentation on streaming synthesis, see "Stream a clip".

For some applications, streaming is not an option. In these cases, the fastest time-to-first-sound can be achieved by sending synchronous requests directly to our synthesis servers. We call this low-latency synthesis (LLS) or direct synthesis. Summarized below are the steps required to use this API:

• Make a request to the low latency synthesis endpoint. (Note: this is not the usual https://app.resemble.ai/api/... URL.)
• Decode the base64 “audio_content” attribute sent back in the response.
• Use the audio data.

HTTP Request

1 2 3 4 5 6 7 8 9 10 11 12 curl --request POST "YOUR_SYNTH_ENDPOINT" -H "Authorization: Bearer YOUR_API_TOKEN" -H "Content-Type: application/json" -H "Accept-Encoding: gzip, deflate, br" --data '{ "voice_uuid": <Voice to synthesize in>, "project_uuid": <Project to save to>, "title": <Title of the clip>, "data": <Text to synthesize>, "precision": "MULAW|PCM_16|PCM_24|PCM_32 (default)" "output_format": "mp3|wav (default)" }'

Request Headers

Header	Value	Description
Authorization	Bearer YOUR_API_TOKEN	API token can be obtained by logging into the Resemble web application and navigating to the API section.
Accept-Encoding	gzip, deflate, br	Either one of gzip, deflate, or br depending on the decompression algorithms your application supports. Omitting the Accept-Encoding header will disable compression.

Request Body

Attribute	Type	Description
voice_uuid	string	The voice to synthesize the text in.
project_uuid	string	The project to save the data to.
title	string	The title of the clip. This is optional, default is to name the clip Low Latency Synthesis {some-uuid}
data	string	The text or SSML to synthesize.
precision	string	The bit-depth of the generated wav file (if using wav as the response type). Either MULAW, PCM_16, PCM_24, or PCM_32 (default).
output_format	string	The output format of the produced audio. Either wav, or mp3.
sample_rate	integer	The sample rate of the produced audio. Either 8000, 16000, 22050, 32000, or 44100

HTTP Response

{
  "audio_content": <base64 encoded string of the raw audio bytes>,
  "audio_timestamps": {
    "graph_chars": string[],
    "graph_times": float[][],
    "phon_chars": string[],
    "phon-times": float[][],
  },
  "duration": float,
  "issues": string[],
  "output_format": string,
  "sample_rate": float,
  "success": boolean,
  "synth_duration": float,
  "title": string|null
}

Response Body

Attribute	Type	Description
audio_content	string	Base64 encoded string. When decoded it will contain the byte array containing the audio.
audio_timestamps	object	Object containing phoneme_timestamp information. See section below for further information.
duration	float	The duration of the produced audio file. Resemble does not bill on this value.
issues	string[]	Any issues pertaining to the synthesis response.
output_format	string	The output format of the produced audio. Either 'wav', or 'mp3'.
sample_rate	integer	The sample rate of the produced audio. Either 8000, 16000, 22050, 32000, or 44100.
success	boolean	True if the response was successful, false otherwise.
synth_duration	float	The duration of the raw audio file produced, before any post processing affects are applied (e.g. the 'prosody' tag which may increase or decrease the duration of the final audio file). Resemble bills on this value.
title	string	The title of the clip. If no title is provided in the request body, then the value will be null

Audio Timestamps Object

Attribute	Type	Description
graph_times	string	A string containing all the phonemes pertaining to the synthesized audio.
phon_times	float[]	An array of floats mapping 1 to 1 with the phoneme_chars. Each index represents the end time in the audio of the phoneme character at the same index in the phoneme_chars array.
phon_chars	char[]	An array of characters mapping 1 to 1 with the end_times array.
graph_chars	char[]	An array of characters mapping 1 to 1 with the end_times array.