voice_audio_multi/docs/models/crisperwhisper.md

CrisperWhisper 2.0

CrisperWhisper 2.0 Large is a Whisper encoder-decoder model for verbatim and intended transcription. The model weights and generated outputs use the upstream non-commercial research license. See the checkpoint's license before using or redistributing them.

Field	Value
Family	crisperwhisper
Tasks	asr, align
Modes	ASR: offline, streaming; alignment: offline
Output	Transcript and optional word timestamps
Long-form audio	Overlapping continuation windows, up to 30 seconds each
Languages	Explicit Whisper language codes; English by default

The BF16 and Q8_0 GGUF packages contain the tokenizer and configuration. Use crisperwhisper-2-large-q8_0.gguf for the smaller package; no external tokenizer files are required.

CLI

audiocpp_cli --task asr --family crisperwhisper \
  --model models/CrisperWhisper2.0-GGUF/crisperwhisper-2-large-bf16.gguf \
  --backend cuda --audio speech.wav \
  --request-option language=en \
  --text-out transcript.txt --words-out words.json --log

Use transcription_mode=intended for cleaned transcription, including normalized numbers. The default verbatim mode preserves disfluencies and vocal events. Language is supplied explicitly; it is not automatically detected.

To restore disfluencies in a known clean transcript, use --request-option transcription_mode=verbatimize --text "Your clean transcript". This mode requires audio no longer than 30 seconds.

To align an existing transcript, use --task align --text "The words spoken" with --audio and --words-out. Alignment first transcribes the recording, then maps the supplied words onto those timestamps. Unmatched words are interpolated between neighboring anchors; they are not independently detected in the audio. Alignment supports long recordings in offline mode.

Long recordings are processed in overlapping windows. Each window receives the last confirmed words as context. --mode streaming emits newly committed text after each window. It requires the complete input recording; it is not live PCM or causal streaming. Concatenate text deltas to obtain the final transcript.

Server

Register separate model IDs for transcription and alignment in the server config:

{
  "host": "127.0.0.1",
  "port": 8080,
  "backend": "cuda",
  "lazy_load": true,
  "models": [
    {
      "id": "crisperwhisper",
      "family": "crisperwhisper",
      "path": "/path/to/crisperwhisper-2-large-q8_0.gguf",
      "task": "asr",
      "mode": "streaming"
    },
    {
      "id": "crisperwhisper-align",
      "family": "crisperwhisper",
      "path": "/path/to/crisperwhisper-2-large-q8_0.gguf",
      "task": "align",
      "mode": "offline"
    }
  ]
}

audiocpp_server --config server.json --log

curl http://127.0.0.1:8080/v1/audio/transcriptions \
  -F model=crisperwhisper -F file=@speech.wav -F language=en -F stream=true

curl http://127.0.0.1:8080/v1/audio/alignments \
  -F model=crisperwhisper-align -F file=@speech.wav \
  -F language=en -F 'text=The words spoken in the recording.'

Omit stream=true for a final transcription response. Streaming emits ordinary transcript.text.delta events after each continuation window; it does not accept live microphone chunks. See usage for server options.

Common Options

Option	Default	Meaning
--audio	Required	Input recording.
--text-out	None	Save the transcript.
--words-out	None	Save word timestamps as JSON; also enables timestamp output.
--mode	offline	offline or window-based streaming.

Request Options (use with --request-option)

Option	Default	Meaning
language	en	Whisper language code, such as de, fr, es, or ja.
transcription_mode	verbatim	verbatim, intended, or verbatimize with a clean transcript supplied through --text.
return_timestamps	false	Return word timestamps.
max_tokens	256	Maximum generated tokens per window, up to 439 and limited by remaining decoder context.
audio_chunk_mode	auto	auto and fixed use continuation windows; none requires at most 30 seconds.
audio_chunk_duration_sec	30	Encoder window length, at most 30 seconds.
audio_chunk_overlap_sec	4	Overlap between windows; must be less than window length.

Session Options (use with --session-option)

Option	Default	Meaning
crisperwhisper.weight_type	native	Matmul weight storage override.

Conversion

Convert the original checkpoint with the standard GGUF tool. Configuration, tokenizer, license, and model spec are embedded in the resulting file.

audiocpp_gguf --input /path/to/checkpoint/model.safetensors \
  --root /path/to/checkpoint --family crisperwhisper \
  --model-spec model_specs/crisperwhisper.json --type orig \
  --output crisperwhisper-2-large-bf16.gguf