voice_audio_multi/docs/build/webgpu.md

WebGPU

WebGPU is disabled by default. Native builds use Dawn; browser builds use Emscripten's Emdawnwebgpu port and run the C++ CLI in a Web Worker. Browser inference requires no native inference server.

Run the following Bash commands from the repository root. Replace /path/to/... placeholders with your own installations or files.

Native build

Requirements

Install CMake, a C/C++ toolchain, and Dawn with its CMake package. Only the WebGPU backend requires C++20. Dawn_DIR must point to the directory containing DawnConfig.cmake.

Configure and compile

cmake -S . -B build/debug \
  -DCMAKE_BUILD_TYPE=Debug \
  -DAUDIOCPP_MODEL_SET=custom -DAUDIOCPP_MODELS=pocket_tts \
  -DENGINE_ENABLE_WEBGPU=ON \
  -DENGINE_ENABLE_CUDA=OFF -DENGINE_ENABLE_HIP=OFF \
  -DENGINE_ENABLE_VULKAN=OFF -DENGINE_ENABLE_METAL=OFF \
  -DDawn_DIR=/path/to/dawn/lib/cmake/Dawn
cmake --build build/debug --target audiocpp_cli -j 8

Change AUDIOCPP_MODELS to the model families you need. The Linux helper is an alternative:

CMAKE_PREFIX_PATH=/path/to/dawn bash scripts/build_linux.sh \
  --backend webgpu --model-set custom --models pocket_tts \
  --build-type Debug --build-dir build/debug --target audiocpp_cli -j 8

Use --backend webgpu; if using individual backend switches instead, pass --cuda off --webgpu on so automatic CUDA detection cannot enable both.

Verify and run

build/debug/bin/audiocpp_cli --version --log
build/debug/bin/audiocpp_cli --list-devices --log
build/debug/bin/audiocpp_cli \
  --task tts --family pocket_tts --model /path/to/model.gguf \
  --backend webgpu --threads 1 \
  --request-option voice_embedding_file=/path/to/voice.safetensors \
  --text "Hello from WebGPU." --out output.wav --log > run.log 2>&1

Use the options and sidecars required by your model. The server, C API backend string, and shared exec_plan parser also accept webgpu. best retains GGML's automatic selection and may select WebGPU when built in. GPU free/total memory is reported as unknown.

Browser build

Requirements and SDK setup

Install Emscripten and activate its SDK environment. Emscripten 6.0.13 and Chrome 150 were used for validation. The browser must support WebGPU, shader-f16 and JavaScript Promise Integration (JSPI). Pthread builds also require cross-origin isolation.

source /path/to/emsdk/emsdk_env.sh

Common browser configuration

These flags select PocketTTS for the example, disable native GPU backends, and enable pthreads for models that prefetch work asynchronously. Keep this Bash array in the same shell for the following configure commands.

browser_options=(
  -DCMAKE_BUILD_TYPE=Debug
  -DAUDIOCPP_MODEL_SET=custom -DAUDIOCPP_MODELS=pocket_tts
  -DAUDIOCPP_DEPLOYMENT_BUILD=ON
  -DENGINE_ENABLE_WEBGPU=ON -DGGML_WEBGPU_JSPI=ON
  -DENGINE_ENABLE_CUDA=OFF -DENGINE_ENABLE_HIP=OFF
  -DENGINE_ENABLE_VULKAN=OFF -DENGINE_ENABLE_METAL=OFF
  -DENGINE_ENABLE_OPENMP=OFF -DGGML_OPENMP=OFF
  -DENGINE_ENABLE_NATIVE_CPU=OFF -DENGINE_ENABLE_LLAMAFILE=OFF
  -DAUDIOCPP_BUILD_NATIVE_MODEL_MANAGER=OFF
  -DAUDIOCPP_BUILD_SERVER_FRONTENDS=OFF -DAUDIOCPP_BUILD_C_API=OFF
  -DENGINE_BUILD_TESTS=OFF
  '-DCMAKE_CXX_FLAGS_DEBUG=-O3 -g'
  '-DCMAKE_C_FLAGS_DEBUG=-O3 -g'
  '-DCMAKE_EXE_LINKER_FLAGS=-pthread -sPTHREAD_POOL_SIZE=navigator.hardwareConcurrency'
)

WASM32

emcmake cmake -S . -B build/debug/browser "${browser_options[@]}" \
  '-DCMAKE_CXX_FLAGS=-fwasm-exceptions -msimd128 -pthread' \
  '-DCMAKE_C_FLAGS=-msimd128 -pthread'
cmake --build build/debug/browser --target audiocpp_cli -j 8

WASM32 has a growable 4 GiB address-space ceiling. Weights, graph arenas and file copies must fit within it; this is not a 4 GiB GPU-memory allowance.

WASM64 for larger models

Use a separate build directory. Select the required model families through AUDIOCPP_MODELS.

emcmake cmake -S . -B build/debug/browser64 "${browser_options[@]}" \
  '-DCMAKE_CXX_FLAGS=-fwasm-exceptions -msimd128 -pthread -sMEMORY64=1' \
  '-DCMAKE_C_FLAGS=-msimd128 -pthread -sMEMORY64=1'
cmake --build build/debug/browser64 --target audiocpp_cli -j 8

This build permits growth up to a 16 GiB WASM address space, without allocating it all at startup. Files larger than 2 GiB use read-only WORKERFS mounts and require WASM64 in this example. Smaller files are copied into MEMFS in bounded chunks. Browser storage-buffer binding limits still apply independently.

Serve and open the example

Emscripten's server supplies the isolation headers required by pthreads:

emrun --no-browser --no-emrun-detect --serve-after-close \
  --hostname 127.0.0.1 --port 8081 build/debug/browser/bin/index.html

Open http://localhost:8081/index.html. For WASM64, serve build/debug/browser64/bin/index.html instead. Use localhost or HTTPS.

Select your model GGUF, its required sidecars, and an input WAV when applicable. The GGUF is mounted as /model.gguf, the WAV as /input.wav, and sidecars as /filename. The resource-directory input preserves the selected directory's relative paths. Adapt the page's CLI arguments to those paths and your model. The PocketTTS example requires a compatible voice embedding sidecar and its matching voice_embedding_file path.

Run displays logs and offers generated WAVs/text for download. --out-dir collects separation outputs. Stop terminates the worker; each run starts a fresh session. External-process workflows report an error because browsers cannot spawn native programs. --threads 1 limits model CPU threading; asynchronous prefetch may use additional workers.

Linux Chrome adapter configuration

The tested Linux/NVIDIA setup needed these flags to expose WebGPU and shader-f16. Use a separate browser profile:

google-chrome --user-data-dir=/path/to/dedicated-browser-profile \
  --enable-gpu --use-angle=vulkan --enable-features=Vulkan \
  --enable-unsafe-webgpu \
  --enable-dawn-features=vulkan_enable_f16_on_nvidia \
  http://localhost:8081/index.html

Adapter support varies by browser and driver. A missing required feature produces an error; the example does not silently switch inference to CPU.

Kokoro and eSpeak data

Generate data with a native build

The browser cannot run the native eSpeak data-generation tools. Generate their package first:

cmake -S . -B build/debug \
  -DCMAKE_BUILD_TYPE=Debug \
  -DAUDIOCPP_MODEL_SET=custom -DAUDIOCPP_MODELS=kokoro_tts \
  -DAUDIOCPP_STATIC_ESPEAK=ON
cmake --build build/debug --target audiocpp_espeak_package -j 8

The package is written to build/debug/_deps/espeak-static/espeak-ng-data.bin.

Configure the browser phonemizer

After setting up the SDK and browser_options above:

emcmake cmake -S . -B build/debug/browser "${browser_options[@]}" \
  -DAUDIOCPP_MODELS=kokoro_tts \
  -DAUDIOCPP_STATIC_ESPEAK=ON \
  -DAUDIOCPP_ESPEAK_DATA_PACKAGE="$PWD/build/debug/_deps/espeak-static/espeak-ng-data.bin" \
  '-DCMAKE_CXX_FLAGS=-fwasm-exceptions -msimd128 -pthread' \
  '-DCMAKE_C_FLAGS=-msimd128 -pthread'
cmake --build build/debug/browser --target audiocpp_cli -j 8

For WASM64, use build/debug/browser64 and include -sMEMORY64=1 in both compiler flag values, as in the WASM64 configuration above. The eSpeak child build inherits the same compilation and Debug flags.

Select espeak-ng-data.bin as a sidecar. Set the page's environment JSON to:

{"AUDIOCPP_ESPEAK_DATA":"/espeak-ng-data.bin"}

Run Kokoro within WASM32 memory limits

Kokoro's default host arenas can exhaust the WASM32 address space, including the default 384 MiB duration-graph arena. The successful WASM32 browser check used the explicit settings below; use this CLI argument array in the page after selecting a Kokoro GGUF and the eSpeak data sidecar:

[
  "--task", "tts", "--family", "kokoro_tts",
  "--model", "/model.gguf", "--backend", "webgpu", "--threads", "1",
  "--voice-id", "af_heart",
  "--session-option", "kokoro_tts.weight_context_mb=32",
  "--session-option", "kokoro_tts.predictor_duration_graph_arena_mb=128",
  "--session-option", "kokoro_tts.predictor_text_graph_arena_mb=128",
  "--session-option", "kokoro_tts.predictor_tail_graph_arena_mb=256",
  "--text", "Hello from the browser WebGPU backend.",
  "--request-option", "seed=1234",
  "--out", "/output.wav", "--log"
]

These are explicit host-memory settings for the short browser example, not model-default changes or a guarantee that longer requests fit. Use WASM64 when the model and workload exceed WASM32's address space. The eSpeak library is cross-compiled; its native-generated data is reused without changing phonemization algorithms.

Developer checks

Enable ENGINE_BUILD_TESTS=ON in the native WebGPU build, then run:

cmake -S . -B build/debug -DENGINE_BUILD_TESTS=ON
cmake --build build/debug \
  --target test-backend-ops backend_device_resolution_test lstm_sequence_test -j 8
build/debug/bin/test-backend-ops test -b WebGPU > backend-ops.log 2>&1
build/debug/bin/backend_device_resolution_test --log > backend-devices.log 2>&1
build/debug/bin/lstm_sequence_test --backend webgpu --log > lstm.log 2>&1

The GGML harness writes its own output and has no --log option. The optional nemotron_asr_webgpu_rnnt_test target requires both WebGPU and Nemotron in the selected native models; invoke it with --model /path/to/nemotron.gguf --log. It is a fixed-input model test, not automatic CTest coverage.

Implementation and validation

The backend is based on GGML commit facdc44a67da17df4433adcc468b424f48e9ded2, adapted to the vendored GGML API. Model-specific integration reuses Irodori's padding/cropping path, permits WebGPU in VibeVoice, and protects Nemotron output-view backing storage. Other backend kernels and shared model math are unchanged.

Performance results, output checks and validation limits are in the browser validation report.