One contract, many TTS models.
rhapsode is a speech server you run yourself. Install the engines you want, and every client talks to one HTTP API that says honestly what each engine can do. Ollama, but for speech.
MIT licensed. One Docker image for amd64 and arm64, with or without a GPU. A rhapsode was a performer who recited written verse aloud, which is the job description.
A client writes the line once
[clear throat] Right, that was The Verve Pipe. [laugh] Nobody warned me about that intro.
Its model is handed
[clear throat] Right, that was The Verve Pipe. [laugh] Nobody warned me about that intro.
This build performs both cues, so each becomes its own tag.
Every engine says what it can do, for the build it has loaded
A client asks GET /engines/{engine}/capabilities and gets back which cues, deliveries and dials the engine performs, which languages, and how it clones. Everything a client offers comes from that document, so a new engine works in it with no change.
It depends on the build. Chatterbox’s turbo performs cues and has no dials, and original is the other way round. So the document says what is loaded now, under current, and what every build could do, under variants. A flat list would let a client send dials that vanish without a word.
The licence names code and weights separately. Apache-2.0 code over research-only weights is common, and the weights are what decide whether you may ship. The catalog shows both before you install anything.
GET /engines/chatterbox/capabilities
{
"contract": 1,
"license": {
"code": "MIT",
"weights": "MIT",
"weightsCommercialUse": true
},
"current": {
"variant": "turbo",
"cues": ["laugh", "chuckle", "sigh", "gasp",
"cough", "clear throat", "sniff", "groan"],
"deliveries": [],
"dials": {},
"cloning": { "supported": true }
},
"variants": {
"turbo": {
"cues": ["laugh", "..."],
"dials": {}
},
"original": {
"cues": [],
"dials": { "exaggeration": {}, "cfgWeight": {} }
}
}
}
Why another speech server
rhapsode started as the speech layer for deadair.radio, a radio station whose hosts are synthesized. A station asks of speech what any product built on it does, only all at once.
- Many voices, on more than one engine
- A host, a newsreader and every character, each on whichever engine suits it. One API speaks them all, and each voice names its engine.
- A presenter who laughs, and a caller who clears their throat
- Cues ride in the text. The station offers only the cues the loaded engine reports, and the core strips the rest, so nobody reads “laugh” out loud on air.
- One graphics card, shared with a language model
- The core decides what is on the card, gives the memory back when a keep-alive runs out, and unloads an engine now when asked.
- Never silence
- Every engine runs in its own process, so a crash takes down that worker and not the server, and a failure is reported instead of arriving as a short, silent file.
Looking for a server that did all four turned up three kinds of project, and none of them was it.
- Single-model wrappers
- A web UI on one model family. You end up using somebody else’s UI backend as infrastructure.
- UI-first suites
- Twenty engines, and the API is a tab that starts a second server. Every engine it reaches shares one Python environment.
- API-first servers
- Well built, and they stop at the easy engines: no voice cloning, and no GPU to share between models.
- rhapsode
- Headless, multi-engine and contract-first, with every engine in its own process and a residency manager that knows a GPU holds one model at a time.
How it works
A worker speaks the engine-scoped part of the public API. There is no second protocol, so an engine author tests a worker with curl and never needs the core.
The engines
Each is a Python package with its own licences, installed through the API into a virtualenv of its own. Text to speech only: speech to text is a different problem, and deliberately not here.
| Engine | Runs on | Default weights | Cues | Voices | Licence: code / weights |
|---|---|---|---|---|---|
| KokoroAn ONNX engine with no torch: the one to install first, on whatever machine you have. | CPU | fp16, 205 MB | 0 of 8 | Blends, and style vectors | GPL-3.0-or-later / Apache-2.0 |
| ChatterboxClones a voice from a short clip, and performs cues or honours dials depending on the build. | GPU | turbo, 3.8 GB | 8 of 8 | Clones from a clip | MIT / MIT |
| OrpheusA Llama that speaks in audio codes, and streams while it is still generating. | GPU | q8, 3.5 GB, or full on Linux | 7 of 8 | Its own eight | Apache-2.0 / Apache-2.0 |
| DiaTwo speakers in one take, cloned from a clip and the words spoken in it. | GPU | 1.6b, 6.7 GB | 8 of 8 | Clones from a clip and its transcript | Apache-2.0 / Apache-2.0 |
| ToneNo weights and no speech: a sine wave that proves the protocol in CI on every commit. | Anywhere | None | 2 of 8 | Clones, to test the path | MIT / MIT |
Run it
With Docker there is nothing to clone. The image serves the API on port 8080 and a web page for installing engines on 8081, and keeps engines and their weights in a volume.
Install Kokoro from the page and it speaks on the CPU, from a download of about 200 MB. With an NVIDIA card, add compose.gpu.yaml and try Chatterbox.
The quick start has the rest, and running one has what the server keeps where, GPUs and upgrades.
curl -fsSLO https://raw.githubusercontent.com/MaroonedSoftware/rhapsode/main/compose.yaml
docker compose up -d
curl -X POST localhost:8080/speak \
-H 'content-type: application/json' \
-d '{"engine":"kokoro","text":"Right, that was The Verve Pipe.","format":"wav"}' \
--output line.wav
Build on it
Add an engine
Subclass one Python class. The SDK handles the socket, the handshake, the errors, encoding and long text, and
rhapsode-conformtells you whether your worker is one.Call the API
Every route, generated from the same contracts as the server, with a typed TypeScript client that depends on nothing.
Point an OpenAI client at it
/v1/audio/speechanswers with the model naming an engine. What an engine cannot do is refused, never quietly dropped.