A research speech synthesis model. It can use the text and audio from earlier turns to generate the next line.
Speech informed by an earlier turn
CSM takes the text for an upcoming line and can use earlier parts of the conversation. The developer's public example supplies the text and audio of previous turns as context. This makes it possible to compare how the same words sound in different situations. The released model synthesizes speech. You write the response for the experiment.
One reply in two situations
For the training scene, come up with an English phrase that the door is open. In the first case, there is a calm question. In the second, the character rushes in and clarifies whether it is possible to enter now. Write the context in your own voice with the permission of the participants. Generate the same phrase in both cases and compare duration, stress and transition to the beginning of speech. The intonation should fit the scene, and the words remain legible.
To understand the contribution of context, leave one option without the previous line and listen to it next to the rest.
Limits of the research release
Sesame indicates that other languages may work worse than English. Before integration, check the example on your device and estimate the amount of memory. To demonstrate, collect an artificial script and use only allowed records. Do not transfer to the model the properties of the entire Sesame voice product. The repository describes an independent model and the conditions for its permissible application. Keep the context separate for each attempt so that the comparison can be repeated.
Suitable tasks
- Check how an earlier line affects intonation
- Prepare the audio of a short learning dialogue
Strengths
- Earlier lines can be supplied as context
- Published weights and launch example
Limitations
- Main language English
- You can use the voice of a real person only with the consent
Selection and conditions
- Free access
- Yes
- Trial period
- Not confirmed
- No sign-up required
- Not confirmed
- Russian interface
- Not confirmed
- Local installation
- Yes
- API
- Not confirmed
“Not confirmed” means there is not enough information. The feature may still be available; check the developer’s terms.
- Input formats
- Text, Audio
- Result
- Audio
Terms
For self-starting, use published weights and a developer example. Requirements for the environment are given in the repository.From task to first result
Practical example
- Task
- Check how an earlier line affects intonation
- What to prepare
- A short fictional dialogue in English with two clearly identified participants.
Task brief or prompt
Speaker one asks when the repair shop opens. Speaker two answers that it opens at nine. Keep both lines short and preserve the pause.
Expected result
Black audio of two lines for listening.
How to check the result
- Distinguish the voices and check the change of the speaker with the text.
- Check pronunciation, duration of pauses and the right to voice examples.
An editorial teaching example of a task brief, not the result of hands-on testing of the service. Check that the required mode is available, that you have rights to the source material and that the intended use of the result is permitted.
Check details
Official sources
This description is based on the developer’s material. Features, prices and availability may change; check the primary source before use.
