voice_audio_multi/examples/server

Ready-to-edit server configs

Pick one config below. Each starts one server on http://127.0.0.1:8080. The files are normal JSON. _comment fields explain what to edit; the server ignores them. Do not add // comments or put _comment inside an option map.

Config	What it demonstrates
qwen3-tts.json	Base voice cloning, CustomVoice built-in speakers, VoiceDesign instructions; separate GGUFs
breeze-tts.json	One GGUF for instruction-only speech or voice cloning
kokoro.json	One GGUF with built-in voices; no cloning reference
pocket-tts.json	A language-specific GGUF, accompanying voice embeddings, and optional voice cloning
yue2.json	A model directory with separate main and VAE GGUFs plus sidecars
minimax-music3.json	A model directory with five component GGUFs, configs, and tokenizer
parakeet-asr.json	Audio transcription and word timestamps

1. Put your files somewhere permanent

Download the package described in the model's documentation. Keep its files and folders together. These configs do not download weights for you.

Copy your chosen config to a writable location, for example server.json. Replace every /PATH/TO/... value with a real absolute path on your computer. Do not copy a path from somebody else's machine. On Windows, use forward slashes, for example C:/models/kokoro-82m-q8_0.gguf.

You can install the packages with the repository's model manager. Run only the commands for your chosen model, from the repository root:

# Qwen3: choose the variant(s) you need.
python3 tools/model_manager_v2.py install qwen3_tts_1_7b_base_q8_0 --models-root models
python3 tools/model_manager_v2.py install qwen3_tts_1_7b_customvoice_q8_0 --models-root models
python3 tools/model_manager_v2.py install qwen3_tts_1_7b_voicedesign_q8_0 --models-root models
python3 tools/model_manager_v2.py install breeze_tts_2_q8_0 --models-root models
python3 tools/model_manager_v2.py install kokoro_82m_q8_0 --models-root models
python3 tools/model_manager_v2.py install pocket_tts_english_q8_0 --models-root models
# YuE2 needs BOTH packages in the same model root.
python3 tools/model_manager_v2.py install yue2_main_q8_0 --models-root models
python3 tools/model_manager_v2.py install yue2_vae_f16 --models-root models
python3 tools/model_manager_v2.py install minimax_music3_q4_0 --models-root models
python3 tools/model_manager_v2.py install parakeet_tdt_q8_0 --models-root models

A path ending in .gguf names a file. A directory path names the folder containing the complete package; do not replace it with just one component file. Qwen3's three entries require three different checkpoints. Delete entries for variants you did not download.

These examples select CUDA. Set backend to cpu for CPU execution, or use another backend supported by your build/model. CPU is especially practical for Kokoro, PocketTTS, and the ASR example. The large music models need substantially more resources. max_loaded_models: 1 unloads an idle model before loading another; it does not reduce the memory required by one model.

A voice library is a folder of your reference recordings

For Qwen3 Base or cloning with Breeze/Pocket, create this folder yourself:

voice-library/
  narrator.wav
  prompt_text

Put a short, clean recording of one person speaking into narrator.wav. Create a plain-text file named exactly prompt_text (no .txt extension). Its contents should be:

narrator|The exact words spoken in narrator.wav go here.

Replace the sentence after | with what the recording actually says. The word before | must match the WAV's filename without .wav. For another voice, add another WAV and another line. Set the config's voice_dir to this folder.

Send "voice": "narrator" to select the recording. The server reads its WAV and, when supported by the model, its transcript. This is different from a built-in voice, which is already supplied with the model. Configured voice_presets take priority over same-named files in the library, so use distinct names. PocketTTS uses reference audio without a reference transcript.

You can also send "voice_ref": "/absolute/path/reference.wav" and "reference_text": "The words in that recording." directly. These paths refer to the server's computer, not a remote client's computer. The speech endpoint also accepts a base64 reference object; see server API documentation.

2. Start the server

From the repository root, using your existing build:

build/debug/bin/audiocpp_server --config /absolute/path/server.json --log --log-file server.log

Open http://127.0.0.1:8080 for the UI, or use the requests below. The configs use lazy loading, so the first generation request loads the weights and takes longer. /health confirms the server is running; it does not prove that a model has loaded. /v1/models lists the configured IDs.

curl --fail-with-body http://127.0.0.1:8080/health
curl --fail-with-body http://127.0.0.1:8080/v1/models

The examples below use a POSIX shell. On Windows use curl.exe with a JSON request file and --data-binary @request.json to avoid shell quoting problems. If a command fails, inspect the response/error and server.log before trying to play the output file.

3. Make a request

Qwen3-TTS

Base: select your reference recording from the voice library.

curl --fail-with-body http://127.0.0.1:8080/v1/audio/speech -H 'Content-Type: application/json' \
  -d '{"model":"qwen3-clone","input":"Hello from my reference voice.","voice":"narrator"}' -o qwen-clone.wav

CustomVoice: the config defaults to Vivian; ryan selects the configured Ryan preset. VoiceDesign: instructions describes the new voice.

curl --fail-with-body http://127.0.0.1:8080/v1/audio/speech -H 'Content-Type: application/json' \
  -d '{"model":"qwen3-custom","input":"Hello from Ryan.","voice":"ryan"}' -o qwen-custom.wav
curl --fail-with-body http://127.0.0.1:8080/v1/audio/speech -H 'Content-Type: application/json' \
  -d '{"model":"qwen3-design","input":"Welcome to our story.","instructions":"A calm adult narrator with a warm voice."}' -o qwen-design.wav

BreezeTTS

Omit voice for instruction-only generation. Add voice for cloning.

curl --fail-with-body http://127.0.0.1:8080/v1/audio/speech -H 'Content-Type: application/json' \
  -d '{"model":"breeze","input":"Welcome to this demonstration.","options":{"instruct":"A warm female narrator, speaking clearly."}}' -o breeze.wav
curl --fail-with-body http://127.0.0.1:8080/v1/audio/speech -H 'Content-Type: application/json' \
  -d '{"model":"breeze","input":"This uses my reference recording.","voice":"narrator"}' -o breeze-clone.wav

Kokoro

heart and michael are configured aliases for built-in voices. No WAV reference is needed. Use a current published GGUF whose embedded spec declares speed.

curl --fail-with-body http://127.0.0.1:8080/v1/audio/speech -H 'Content-Type: application/json' \
  -d '{"model":"kokoro","input":"Hello from Kokoro.","voice":"heart","speed":1.0}' -o kokoro.wav

PocketTTS

Keep embeddings/alba.safetensors beside the English GGUF. The config defaults to Alba. To clone your own voice, select the library name instead.

curl --fail-with-body http://127.0.0.1:8080/v1/audio/speech -H 'Content-Type: application/json' \
  -d '{"model":"pocket","input":"Hello from Alba."}' -o pocket.wav
curl --fail-with-body http://127.0.0.1:8080/v1/audio/speech -H 'Content-Type: application/json' \
  -d '{"model":"pocket","input":"Hello in my reference voice.","voice":"narrator"}' -o pocket-clone.wav

YuE2: main model plus VAE

Point path at the folder containing:

Yue2-3B-GGUF/
  yue2-3b-q8_0.gguf
  yue2-vae-f16.gguf
  sidecars/
    yue2-model-config.json
    yue2-generation-config.json
    yue2-qwen.tiktoken
    yue2-vae-config.json

The yue2.model_gguf and yue2.vae_gguf session options choose filenames inside this folder. Change these filenames if you downloaded another supported precision. Do not pass absolute paths for these two options. Lyrics and style are different inputs: put lyrics in request.text and style in request.options.style.

curl --fail-with-body http://127.0.0.1:8080/v1/tasks/run -H 'Content-Type: application/json' \
  -d '{"model":"yue2","request":{"text":"[Verse]\nMorning sunlight fills the sky.\n[Chorus]\nSing together, you and I.","options":{"style":"English, acoustic pop, warm vocals","cot":"off"}}}' -o yue2-result.json

This returns JSON with base64 WAV audio; see the decoding command below. Optional AR/NAR LoRAs belong in session_options, using yue2.ar_lora or yue2.nar_lora plus the corresponding _scale. They are not needed for this example. See YuE2 before adding adapters.

MiniMax Music3: five components

Point path at the package folder. Keep its layout intact:

MiniMax-Music3-GGUF/
  config.json
  config/
    language_model.json
    rvq_depth_decoder.json
    condition_encoder.json
    transformer.json
    vocoder.json
  tokenizer/
    tokenizer.json
    tokenizer_config.json
  language_model_q4_0.gguf
  rvq_depth_decoder_q8_0.gguf
  condition_encoder.gguf
  transformer_q4_0.gguf
  vocoder.gguf

Here request.text is the musical description, and request.options.lyrics contains the required lyrics. duration_sec is a generation budget, not a promise of an exact output duration. Keep mem_saver enabled for this example.

curl --fail-with-body http://127.0.0.1:8080/v1/tasks/run -H 'Content-Type: application/json' \
  -d '{"model":"music3","request":{"text":"Warm acoustic pop with clear vocals and gentle drums.","options":{"lyrics":"[verse] Morning sunlight fills the sky. [chorus] Sing together, you and I.","duration_sec":20}}}' -o music3-result.json

Decode either music model's JSON output using Python (change the two filenames):

python -c 'import base64,json,pathlib; r=json.loads(pathlib.Path("music3-result.json").read_text()); pathlib.Path("music3.wav").write_bytes(base64.b64decode(r["audio"]))'

Parakeet ASR: upload audio from your client

Replace /PATH/TO/input.wav with a WAV on the computer running curl. The plain endpoint returns text; the details endpoint also preserves word positions and the sample rate needed to interpret them.

curl --fail-with-body http://127.0.0.1:8080/v1/audio/transcriptions \
  -F model=parakeet -F file=@/PATH/TO/input.wav
curl --fail-with-body http://127.0.0.1:8080/v1/audio/transcriptions/details \
  -F model=parakeet -F file=@/PATH/TO/input.wav

Streaming and option placement

The configs deliberately start in offline mode. For speech streaming, change that model entry to "mode": "streaming", restart the server, and add "stream_format": "sse" to its speech request. SSE carries base64 PCM chunks, not a single WAV file; use a streaming client instead of saving SSE as .wav. YuE2 and MiniMax Music3 support offline generation only.

Check the package's actual capabilities before enabling streaming:

build/debug/bin/audiocpp_cli --family breeze_tts --model /PATH/TO/breeze-tts-2-q8_0.gguf --help

Older GGUFs can embed older specs, so help and server model discovery may omit new modes/options even with a new binary. Update the package. If you deliberately use an older GGUF with a current source checkout, you can explicitly add "model_spec_override": "/PATH/TO/audio.cpp/model_specs/breeze_tts.json" inside that model entry (use the matching family file). This changes which contract the loader uses; it does not convert weights. The equivalent CLI flag is --model-spec-override /PATH/TO/audio.cpp/model_specs/breeze_tts.json. The offline examples above do not require this override.

• load_options: package selection needed when loading weights (Pocket language).
• session_options: component selection and session setup; restart/reload after changes.
• default_request_options: generation defaults. Request options override them.
• voice_presets: named built-in voices or fixed reference recordings.
• voice_dir: your reusable reference WAV library, shared by configured models.

Do not copy CLI flags literally into JSON. For example, --session-option yue2.model_gguf=... becomes "session_options": {"yue2.model_gguf": "..."}. See the server documentation for the complete API.