# Parakeet-TDT 0.6B v3

Default model-manager downloads use the published GGUF package when available;
the original source/conversion instructions below remain valid for manual use.

FastConformer-TDT ASR port of NVIDIA's [`nvidia/parakeet-tdt-0.6b-v3`](https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3)
(0.6B params, 25 European languages with auto language detection). The default
model-manager package is the standalone Q8_0 GGUF. The repository also supports
a Transformers-compatible `model.safetensors` checkpoint rather than a `.nemo`
archive.

## Status

Offline full-context, bounded-window long-form, and buffered-streaming sessions
are implemented. The full-context path is verified end to end on CPU and CUDA
against the real NeMo reference model, both by final transcription and by
numerical comparison at selected intermediate and full-encoder boundaries; see
[Validation](#validation).
Buffered modes deliberately re-encode fixed bidirectional windows and are not
native cache-aware streaming.

| Mode | Context and memory | Result timing |
|---|---|---|
| Offline `full_context` | Whole utterance; quadratic attention grows with duration | One final result |
| Offline `long_form` | Bounded overlapping windows; center regions decoded once | One final result |
| Streaming | Same bounded windows with retained predictor state | Partial snapshots after right-context lookahead, then a final result |

## Architecture

[Underdog Bark 3.0 Pro](underdog_bark3.md) is another English-only Parakeet-TDT
variant. Its converter and model-local adapter preserve the publisher's weight
scaling and 250 ms input prefix; use that guide rather than loading its original
BF16 checkpoint as stock Parakeet.

The same runtime can load the [Phonon-2 English ASR variant](#phonon-2) after
exact reconstruction of its custom five-value container into a dense F32
package. The Phonon package supports English-only offline and streaming ASR; it does
not inherit this base checkpoint's 25-language claims.

```
Frontend: 16kHz -> 128 mel bins, preemphasis=0.97, NeMo per-feature normalization
Encoder:  3-stage Conv2D subsampling (8x) -> 24-layer FastConformer -> 1024-dim
Decoder:  2-layer LSTM predictor + joint network (TDT: 5 duration classes [0..4])
```

## Build and run

```bash
cmake --build build/<preset> --target parakeet_warm_bench

build/<preset>/bin/parakeet_warm_bench \
    --model models/parakeet-tdt-0.6b-v3 \
    --audio tests/parakeet_tdt/assets/2086-149220-0033.wav \
    --backend cpu   # or: --backend cuda
```

Equivalent CLI commands:

```bash
# Offline full-context
build/<preset>/bin/audiocpp_cli \
    --task asr --family parakeet_tdt \
    --model models/parakeet-tdt-0.6b-v3 --backend cpu \
    --audio tests/parakeet_tdt/assets/2086-149220-0033.wav

# Offline bounded-window long-form
build/<preset>/bin/audiocpp_cli \
    --task asr --family parakeet_tdt \
    --model models/parakeet-tdt-0.6b-v3 --backend cpu \
    --session-option parakeet_tdt.offline_mode=long_form \
    --audio tests/parakeet_tdt/assets/2086-149220-0033.wav

# Buffered streaming from raw mono 16 kHz PCM
ffmpeg -loglevel error -i input.wav -f f32le -ac 1 -ar 16000 - \
  | build/<preset>/bin/audiocpp_cli \
      --task asr --family parakeet_tdt \
      --model models/parakeet-tdt-0.6b-v3 --backend cpu \
      --mode streaming --audio - --input-format f32le \
      --input-rate 16000 --input-channels 1
```

Install the model with:

```bash
python3 tools/model_manager_v2.py install parakeet_tdt --models-root models
```

This installs the upstream F32 safetensors package. The repository does not
ship a Parakeet weight file; build `audiocpp_gguf` and convert the installed
package to create a standalone GGUF with the model spec, tokenizer, and
configs embedded:

```bash
cmake --build build/<preset> --target audiocpp_gguf

# Original F32 weights
build/<preset>/bin/audiocpp_gguf \
    --input models/parakeet-tdt-0.6b-v3/model.safetensors \
    --root models/parakeet-tdt-0.6b-v3 \
    --family parakeet_tdt \
    --model-spec model_specs/parakeet_tdt.json \
    --type orig --overwrite \
    --output models/parakeet-tdt-0.6b-v3-f32.gguf

# F16 weights (use `--type bf16` for BF16)
build/<preset>/bin/audiocpp_gguf \
    --input models/parakeet-tdt-0.6b-v3/model.safetensors \
    --root models/parakeet-tdt-0.6b-v3 \
    --family parakeet_tdt \
    --model-spec model_specs/parakeet_tdt.json \
    --type f16 --overwrite \
    --output models/parakeet-tdt-0.6b-v3-f16.gguf

# Q8_0 weights
build/<preset>/bin/audiocpp_gguf \
    --input models/parakeet-tdt-0.6b-v3/model.safetensors \
    --root models/parakeet-tdt-0.6b-v3 \
    --family parakeet_tdt \
    --model-spec model_specs/parakeet_tdt.json \
    --type q8_0 --overwrite \
    --output models/parakeet-tdt-0.6b-v3-q8_0.gguf
```

Pass any `.gguf` file directly to `--model`. The tested files are about
2.4 GB for original F32, 1.2 GB for F16 or BF16, and 874 MB for Q8_0. All four
storage variants reproduce the checked-in golden transcription in offline
full-context, bounded-window long-form, and buffered-streaming tests. Q8_0 has
the broader multilingual accuracy tradeoff described under
[Performance](#performance).

GGUF is the package container, not a different execution engine: both the
safetensors and GGUF paths execute through ggml. With a safetensors source,
`parakeet_tdt.matmul_weight_type` converts eligible weights when the session is
created. With a pre-converted GGUF, the default `native` setting uses the
types stored in that file. The performance study below used the safetensors
package plus session-time storage conversion; the GGUF checks validate
standalone loading and output, not a separate GGUF-vs-safetensors speed claim.

## Orukeet r3 checkpoint

[Orukeet r3](https://huggingface.co/oruk/orukeet) (Oruk AI) is a fine-tune of
`parakeet-tdt-0.6b-v3` that replaces half of the encoder's temporal depthwise
filters with fitted, frozen Gabor kernels and continues training on
multilingual and multi-accent data. It keeps the exact architecture,
tensor shapes, preprocessor, and tokenizer of the stock checkpoint — the
fitted kernels are materialized as ordinary convolution weights — so it runs
through this same `parakeet_tdt` loader with no engine changes. It is a
weight variant, not a new model family.

Orukeet publishes a `.nemo` checkpoint rather than a Transformers-compatible
safetensors package, so conversion goes through
`tools/community_models/convert_orukeet.py`, which maps the 651 NeMo tensors
to the audio.cpp layout, reuses the stock sidecars (verified against the
`.nemo` config; the r3 export audit certifies the tokenizer is unchanged),
and packs a standalone GGUF via `audiocpp_gguf`. The script never downloads
anything; point it at a local copy of the pinned r3 file:

```bash
uv run --python 3.12 --with torch --with pyyaml --with safetensors \
    --with sentencepiece --with numpy \
  tools/community_models/convert_orukeet.py \
    --checkpoint path/to/orukeet-v0.1.0.nemo \
    --converter build/<preset>/bin/audiocpp_gguf \
    --gguf-output models/Orukeet-GGUF/orukeet-q8_0.gguf \
    --type q8_0
```

The stock install (`models/parakeet-tdt-0.6b-v3`, from
`model_manager_v2.py install parakeet_tdt`) supplies the reference sidecars.
Source checkpoint SHA-256:
`031c8ddab4845aeced904a7cde8e8aa57993b2e344716cf83a545b079c473b56`.

Validated on CUDA (RTX 4070 Ti SUPER) against the golden clip
(`tests/parakeet_tdt/assets/2086-149220-0033.wav`): all four storage
variants reproduce the checked-in stock transcription byte-identically with
word timestamps — F32 (~210 ms, ~35x realtime), F16 (~141 ms, ~53x),
BF16 (~186 ms, ~40x), Q8_0 (~162 ms, ~46x); tested files are about 2.5 GB
for F32, 1.3 GB for F16 or BF16, and 916 MB for Q8_0. This checks port
correctness on one English clip; Orukeet's vendor-reported multilingual
gains (pooled FLEURS WER 9.85% vs 11.01%) are not independently validated
here, and its final adaptation trained on LibriSpeech test-other, so treat
the vendor's LibriSpeech numbers with caution.

License: Orukeet r3 weights and fitted kernels are CC BY-SA 4.0, retaining
NVIDIA's CC BY 4.0 foundation attribution; redistribution of converted GGUFs
carries the ShareAlike terms. See the upstream
[NOTICE](https://huggingface.co/oruk/orukeet/blob/main/NOTICE.md) and
[weight license](https://huggingface.co/oruk/orukeet/blob/main/LICENSE-WEIGHTS).
The converter records both in the staging `provenance.json`.

## Options

Request options are bare names. Session options use the
`parakeet_tdt.` prefix.

| Option | Scope | Values/default | Meaning |
|---|---|---|---|
| `max_tokens` | request | integer >= 0; `0` | TDT token limit; `0` uses the model-derived limit |
| `keep_language_tags` | request | boolean; `false` | Preserve language-tag tokens in decoded text |
| `return_timestamps` | request | boolean; optional | Accepted for CLI `--words-out` compatibility. Word timestamps remain present when false or omitted. |
| `audio_chunk_mode` | request | `auto`, `fixed`, `vad`, `none`; `auto` | Offline chunking mode; `vad` skips silent spans with Silero VAD |
| `audio_chunk_duration_sec` | request | see description | VAD maximum chunk duration; omitted inherits the session duration. Fixed/long_form windows use the session setting, not this request override. |
| `weight_type` | session | `native`, `f32`, `f16`, `bf16`, `q8_0`; `native` | Fallback storage type for matmul weights |
| `matmul_weight_type` | session | same values; inherits `weight_type` | Encoder and decoder matmul storage type |
| `conv_weight_type` | session | `native`, `f32`, `f16`; `native` | True convolution-weight storage type |
| `perf_mode` | session | `off`, `flash_attention`; `off` | Attention implementation; flash attention was slower on the tested hardware |
| `weight_context_mb` | session | integer >= 1; `3072` | Weight arena size in MiB |
| `encoder_graph_arena_mb` | session | integer >= 1; `1024` | Encoder graph arena size in MiB |
| `decoder_graph_arena_mb` | session | integer >= 1; `256` | Decoder graph arena size in MiB |
| `audio_chunk_duration_sec` | session | float >= 0.001; `2` | Center-window duration for streaming and offline long-form execution; also the default maximum VAD chunk duration. |
| `left_context_sec` | session | float >= 0; `10` | Past context included in each bounded window |
| `right_context_sec` | session | float >= 0; `2` | Future context/lookahead included in each bounded window |
| `streaming_attention_mode` | session | `full_context`; `full_context` | Bidirectional attention within each bounded streaming window |
| `offline_mode` | session | `full_context`, `long_form`, `auto`; `full_context` | Offline scheduling policy |
| `audio_chunk_threshold_sec` | session | float >= 0.001; `30` | `auto` threshold for switching to long-form execution |
| `vad_model_path` | session | path; `assets/framework/models/silero_vad` | Silero VAD model directory for `audio_chunk_mode=vad` |

Word timestamps are built from the decoder's actual nonblank token-emission
frames. SentencePiece fragments and following punctuation are merged into
human-readable words. Each completed word ends at the next word's emission
boundary; the final word ends at the final token's predicted duration boundary,
clamped to the encoded audio.

## Buffered streaming

`RunMode::Streaming` provides bounded-window buffered streaming for callers
that need incremental results. The default window uses a 2-second center
region, 10 seconds of left context, and 2 seconds of right context. Each fixed
window is re-encoded with full bidirectional attention, then only its center
frames are decoded while the TDT predictor state is retained. This is not
cache-aware streaming: right context adds lookahead latency, partial text can
differ from offline full-context output, and recent text remains provisional
until the next center boundary. Use `nemotron_asr` when native cache-aware,
lower-latency ASR is required.

The window is controlled with `parakeet_tdt.audio_chunk_duration_sec`,
`parakeet_tdt.left_context_sec`, and `parakeet_tdt.right_context_sec`.
Streaming currently requires contiguous mono 16 kHz chunks. `finalize()`
flushes a short tail; `reset()` retains the loaded weights and reusable graphs.

## Long-form audio

Offline mode remains full-context by default. Set
`parakeet_tdt.offline_mode=long_form` to process long audio with the same
bounded overlapping-window scheduler, or use `auto` to
switch after `parakeet_tdt.audio_chunk_threshold_sec` (30 seconds by
default). Long-form mode preserves global timestamps and predictor state while
decoding every center region exactly once. Because each region sees bounded
rather than utterance-wide context, its transcript can differ from the default
full-context result.

Offline long-form input is downmixed and resampled to mono 16 kHz before
windowing, using the same preprocessing as full-context mode. Streaming still
requires mono 16 kHz input. Request `audio_chunk_mode=fixed` selects this
bounded-window scheduler even when `offline_mode=full_context`. Set the session
option `parakeet_tdt.audio_chunk_duration_sec` for its center-window duration;
the request option `audio_chunk_duration_sec` only controls VAD chunk duration.

Set `--request-option audio_chunk_mode=vad` to use Silero VAD-planned speech
spans instead of fixed windows. VAD mode uses
`audio_chunk_duration_sec` as the maximum speech chunk duration, falling back to
`parakeet_tdt.audio_chunk_duration_sec` when omitted. `parakeet_tdt.vad_model_path`
locates the bundled Silero VAD model. `auto` keeps the existing Parakeet
`offline_mode` behavior.

## Performance

Reference clip (`2086-149220-0033.wav`, 7.435s) on an Intel i7-9750H (6C/12T)
and a GTX 1650 Max-Q, against the real NeMo model on the same machine:

| implementation | settings | CPU | CUDA |
|---|---|---|---|
| **NeMo / PyTorch** (the Python reference) | as shipped | 857.7 ms (8.7x real-time) | **OOM** — won't load on this 4GB card |
| **audio.cpp** (this port) | untuned: 8 threads, f32 | ~1245 ms (6.0x real-time) | ~169 ms (44.0x real-time) |
| **audio.cpp** (this port) | tuned: 12 threads, `q8_0` | **~715 ms** (**10.4x** real-time) | **~131 ms** (56.8x real-time) |

Read down the CPU column: **untuned, this port loses to PyTorch** (~1245 ms vs
857.7 ms). Tuned, it wins by ~1.2x — and on this GPU the comparison doesn't
exist at all, because the PyTorch reference cannot load the model in 4 GB
while the native path runs it in 2642 MiB. "Untuned" here means the two
defaults, not a deliberately handicapped configuration: `--threads` defaults
to 8, and `parakeet_tdt.matmul_weight_type` defaults to `native`, which for
this checkpoint means f32 (`config.json` declares `dtype: float32`).

Re-measured on the current tree with the drift-cancelling harness (ABBA, 4
passes, median of 5 iterations per run): tuned is **1.72x** default on CPU and
**1.31x** on CUDA. Both defaults reproduce earlier figures closely; the CPU
tuned row previously read ~750 ms, which was measured before the optimization
pass and is now slightly conservative.

```bash
build/<preset>/bin/parakeet_warm_bench \
    --model models/parakeet-tdt-0.6b-v3 --audio tests/parakeet_tdt/assets/2086-149220-0033.wav \
    --backend cpu --threads 12 \
    --session-option parakeet_tdt.matmul_weight_type=q8_0
```

Two settings account for essentially all of the tunable gap:

- **Thread count.** 12 threads is fastest on this 6C/12T CPU — 3.1% over 6,
  1.1% over 8. Sweep it on your own hardware, and sweep it *interleaved*
  rather than as sequential before/after runs (this machine drifts 20-30%
  between cold and thermally saturated, which is enough to invert the answer).
- **Weight precision.** `parakeet_tdt.matmul_weight_type` accepts
  `native|f32|f16|bf16|q8_0` (default `native`; `parakeet_tdt.conv_weight_type`
  separately accepts `native|f32|f16`). Encoder graph compute is ~93-96% of
  wall time, so this is the setting that matters.

Measured over 120 FLEURS clips across 24 languages
([harness](../../tests/parakeet_tdt/multilingual/README.md),
[raw transcripts](../../tests/parakeet_tdt/multilingual/results/README.md)):

| weight type | encode speed | transcript identical to f32 | absolute WER |
|---|---|---|---|
| `native` (f32) | 1.00x | 100% | 0.1338 |
| `f16` | ~1.00x | 99.2% | 0.1336 |
| `bf16` | **1.14x** | 99.2% | 0.1334 |
| `q8_0` | **1.79x** | 91.7% | 0.1331 |

- **`bf16`** is close to free: 1.14x, 99.2% of transcripts byte-identical.
- **`q8_0`** is the real speedup at 1.79x. It changes ~8% of transcripts, but
  absolute WER does not move — the churn is mostly formatting (`T Rex` →
  `T-Rex`), and where words change it goes both directions. It costs
  reproducibility against an f32 baseline, not quality.
- **`f16`** is not a speedup here at all; don't assume fewer bits means faster.

`native` stays the default because it is the only setting that reproduces
exactly. Full methodology, profiling, and the optimizations that were tried
and rejected are in
[docs/reports/parakeet_tdt_performance.md](../reports/parakeet_tdt_performance.md).

## Validation

- **Golden-transcription regression test** (`ctest -R parakeet_golden_transcription_test`,
  wired into the normal `ENGINE_BUILD_TESTS` suite, skips cleanly when the model
  isn't downloaded): runs full-context and bounded-window long-form decoding
  against a checked-in LibriSpeech test-clean clip
  (`tests/parakeet_tdt/assets/2086-149220-0033.wav`) and asserts the decoded
  text matches the real NeMo model's transcription for that clip exactly:
  *"Well, I don't wish to see it any more, observed Phoebe, turning away her eyes.
  It is certainly very like the old portrait."*
- **Buffered-streaming regression test**
  (`ctest -R parakeet_streaming_transcription_test`): checks partial output,
  finalization, reset/reuse, option validation, and the final transcript on the
  same fixture.
- **Standalone GGUF path tests:** original F32, F16, BF16, and Q8_0 GGUFs with
  embedded specs and sidecars pass the offline, long-form, timestamp, and
  buffered-streaming regression paths while using their stored tensor types.
- **Multilingual precision study:** 120 FLEURS clips across the 24 supported
  languages covered by FLEURS measure transcript and WER movement for
  `native`, F16, BF16, and Q8_0. This is a precision comparison, not a general
  benchmark of Parakeet's absolute WER.
- **Numerical parity harness** (`tests/parakeet_tdt/parity/`, manual/pre-release
  check, not a CI gate — needs a NeMo install): compares mel features, a single
  isolated encoder layer, and the full encoder output directly against real NeMo
  activations via forward hooks. See `tests/parakeet_tdt/parity/README.md` for
  exact setup and run commands. This is what actually caught the encoder
  conv-module bug below — the golden-transcription test alone did not, since
  greedy decoding happened to be robust enough to it on that one clip.
  Currently passing on both backends: `mel_features` and `layer_0` at cosine
  1.000000, `enc_out` at 0.972258 on CPU and CUDA alike.
- Backends tested: CPU and CUDA (GTX 1650 Max-Q), both producing the exact same
  transcription.

Two real bugs were found and fixed getting to this state, worth knowing about if
you're touching this code:

1. The frontend was missing NeMo's per-feature mel normalization (subtract the
   per-mel-bin mean, divide by the per-mel-bin unbiased std over valid frames).
   Without it the encoder saw wildly out-of-distribution input and produced
   garbage (~500x too large in magnitude by the end of the conv subsampling
   stack).
2. The FastConformer encoder layer's conv module used causal (left-only)
   padding, and separately built the depthwise conv with `use_bias=false`,
   silently dropping the folded batch-norm bias term. The padding mode is
   wrong for this model's full-context (non-streaming) configuration — NeMo's
   `CausalConv1D` is only actually causal when constructed with `padding=None`;
   this model constructs it with an explicit symmetric `padding=(kernel-1)//2`.
   The dropped bias corrupted every layer's output by a per-channel constant
   offset; because greedy/argmax decoding is somewhat robust to numerical
   drift, this second bug did not visibly break transcription on the one test
   clip and was only caught by the numerical parity harness comparing actual
   activations against NeMo, not just final decoded text.

## Known limitations

- **Buffered streaming is not native cache-aware streaming.**
  `nvidia/parakeet-tdt-0.6b-v3` was trained and exported with
  `att_context_style="regular"` and `att_context_size=[-1, -1]` — unlimited,
  fully bidirectional attention context, not NeMo's `"chunked_limited"` style
  that cache-aware streaming depends on. Calling NeMo's own
  `encoder.setup_streaming_params()` on this checkpoint does not error, but
  produces a degenerate configuration (~5.8 second chunks, a ~10000-frame
  attention cache) that provides essentially none of the latency or bounded-
  memory benefit real streaming is for. NVIDIA's own maintainers, asked
  directly about streaming with this model on its Hugging Face discussion
  page, pointed users at a different, dedicated cache-aware streaming
  architecture rather than confirming this checkpoint for the purpose.
  The implemented `RunMode::Streaming` therefore uses bounded, overlapping
  full-attention windows and retains only the TDT predictor state between
  center regions. It bounds graph and live-buffer size and produces partial
  results, but re-encodes context, incurs configured right-lookahead latency,
  and can differ from utterance-wide offline output. Genuine causal streaming
  would need a purpose-trained `att_context_style="chunked_limited"`
  checkpoint.
  **If you need streaming ASR, this framework already has it**: the
  `nemotron_asr` family (`nvidia/nemotron-3.5-asr-streaming-0.6b`) is a
  same-size-class NVIDIA checkpoint actually trained cache-aware, with
  configurable chunk sizes down to 80ms, and already implements
  `IStreamingVoiceTaskSession` in this codebase.
- Direct layer-by-layer numerical parity against NeMo is limited to one
  English fixture. The end-to-end multilingual study is broader (120 clips,
  24 languages) but shallow at five clips per language and is intended to
  compare storage precisions, not certify production WER for every domain.
- The frontend does not implement NeMo's preprocessor `dither` (small random
  waveform noise, standard to disable at inference time in most ASR
  pipelines) — no observed effect on transcription correctness.

## Phonon-2

> [!IMPORTANT]
> Routing now follows the registered family: `parakeet_tdt`, `phonon2_asr`, or
> `underdog_bark3`. The `variant` field no longer selects Phonon or Bark behavior.
> Regenerate or refresh older Phonon/Bark GGUF specs with the matching family;
> renaming the file or changing `--family` alone does not update embedded metadata.

English-only Phonon-2 uses the `phonon2_asr` family and its own model contract,
sharing Parakeet’s encoder, predictor, joint and backend arithmetic. There is
no packed five-value runtime.

The model-local additions are optional hotwords, rolling live transcription,
and `word_timestamp_mode: "token_duration"`. Other Parakeet packages retain
their defaults. Shared transport support is limited to an optional revisable
transcript snapshot; no scheduler or multislot admission changes are included.

### Convert

Download the official Fermion Research artifact at revision
`7e153e4054c0a10db6d47d6fff5c062d4c77c154`, including the archive, NOTICE and
license files. Parakeet config/tokenizer sidecars use revision
`541d1f99c6b0c3cd0b11a95167540bb8edefd82b`. Install `numpy`, `safetensors` and
`zstandard`; conversion does not require PyTorch or NeMo.

```bash
python tools/community_models/convert_phonon2.py \
  --source-dir models/Phonon-2-source \
  --reference-dir models/Phonon-2-source/parakeet-reference \
  --output-dir models/Phonon-2-Parakeet-F32 \
  --converter build/bin/audiocpp_gguf \
  --gguf-type mixed-f16 \
  --gguf-output models/Phonon-2-Parakeet-GGUF/phonon-2-mixed-f16.gguf
```

The example selects the lossless mixed-F16 package described below. For F32,
BF16 or Q8 derivatives, choose `--gguf-type orig`, `bf16` or `q8_0` and distinct
output paths. F32 expands the published trained checkpoint; it is not a pre-quantization
FP32 checkpoint. The converter verifies the container, retains attribution,
expands 699 inference tensors and omits 24 training-only counters. It compensates
the projection by 1/32 to cancel the normal loader's input-scale fold and adjusts
BatchNorm gamma in the two channels affected by its variance floor. These are
documented conversion transforms, not byte-identical original tensor claims.
The F32 audit checks every transformed tensor and the resulting encoder output.

BF16/Q8 are rounded derivatives. Projection tensors remain F32 to preserve their
scale compensation; Q8 also keeps the duration-sensitive joint head in F32.
The dense packages are approximately 2.51/1.26/0.94 GB, not the original 164 MB
packed package. No public download/package-manager URL is advertised here.

The `phonon2_asr` family registration selects hotwords and rolling streaming.
`variant` is descriptive metadata and does not select an implementation. `word_timestamp_mode` controls timestamp
formatting only. Regenerate unpublished development packages with this converter;
there is no compatibility fallback for earlier Phonon metadata or option names.

`--gguf-type mixed-f16` offers lossless storage of the staged values: tensors
use F16 only when conversion back to F32 is exact. Other tensors, normalization
and compensated projections stay F32. The package is about 1.30 GB. It does
not override backend precision policy; different storage types can select
different native kernels, so exact weights do not guarantee identical duration
decisions. To reproduce the F32 package's runtime weight types,
use existing session options `phonon2_asr.matmul_weight_type=f32` and
`phonon2_asr.conv_weight_type=f32`; this restores its working weight memory.

`--gguf-type mixed-q8` creates `phonon-2-mixed-q8.gguf` (about 1.18 GB).
It keeps feed-forward and pointwise matrices in exact F16, normalization and
non-F16-representable values in F32, and quantizes the remaining matrices to
Q8_0. It is a mixed-precision, rounded derivative, not lossless whole-model
storage. Across the 21 local clips on each of CPU, CUDA and Vulkan, all 63
transcripts match the dense reference, with 60/63 word timestamp matches
versus 51/63 for the previous Q8. One CPU clip acquires a 240 ms end shift;
the sampled improvement does not establish full timestamp parity or corpus
WER equivalence. See the [mixed-storage validation](../reports/phonon2_validation.md#mixed-gguf-storage-follow-up).
The original `q8_0` conversion remains available with its previous policy.

TDT timestamps advance on an 80 ms grid. Small score differences can change
blank/token selection or the predicted duration, producing whole-frame shifts
even when the final transcript matches. See the
[duration investigation](../reports/phonon2_validation.md#duration-diagnostics) for
decoder replays, reference-mask checks and conversion tradeoffs.

Earlier development packages used unscaled projections and special runtime
precision controls. Reconvert them; the loader rejects their obsolete metadata
with an actionable error rather than silently applying incompatible scaling.

### CLI and hotwords

```bash
audiocpp_cli --task asr --family phonon2_asr \
  --model models/Phonon-2-Parakeet-GGUF/phonon-2-mixed-f16.gguf \
  --backend cuda --device 0 --threads 8 --audio recording.wav \
  --request-option 'hotwords=["Ada Lovelace","CUDA"]' \
  --request-option hotwords_score=2
```

Use `cpu`, or `vulkan --device 1` on the tested RTX 3090 setup. The device index
depends on the machine. For Windows executable paths add `.exe`.

Hotwords can be a JSON string array, `{ "word": "...", "spoken": ["..."] }`
entries, or comma/newline/semicolon-separated terms. Without separators, whitespace
separates terms. The model uses recognition-context text as a vocabulary fallback;
explicit hotwords, including an empty list, take precedence.
The policy accepts at most 25 distinct terms, strength 0–100 (default 2), 16 KiB
input, 256-byte terms and at most eight spoken alternatives. It does not bias
blank/duration logits or guarantee correction of every named entity. Request
state is cleared on reset/finalization. Stock Parakeet does not opt into it.

### Streaming

Use the same CLI command with `--mode streaming`, or configure a legacy server
model with `family: "parakeet_tdt"`, the new GGUF path, `task: "asr"` and
`mode: "streaming"`. Transcription uploads with `stream=true` and
`POST /v1/audio/transcriptions/live` (mono 16 kHz PCM) use existing transports.
JSON requests and batch multipart uploads can pass `hotwords` and
`hotwords_score` through the existing generic `options` object. Single-file
multipart uploads and live requests use the existing `prompt` field or query
parameter for vocabulary hints. JSON requests use `text` for recognition context.
There are no dedicated top-level hotword fields or live hotword query parameters.

The Phonon policy processes 50 ms energy blocks, allows a first provisional
decode after 350 ms of a voiced phrase, then every 500 ms of new audio, and
commits after 700 ms silence, 30 seconds, or input completion. It re-encodes the
growing phrase; this is not a cache-aware encoder or an offline speedup.

`partial_text_snapshot` and SSE `transcript.text.partial` contain the complete
current hypothesis and may revise previous words. Replace the provisional
display. `partial_text` and `transcript.text.delta` contain committed segments;
their concatenation equals final text. Committed timestamps have absolute
sample offsets. The original Phonon WebSocket protocol is not implemented.

### Validation and limits

See the [current validation report](../reports/phonon2_validation.md) and
[portable evidence](../../tests/parakeet_tdt/phonon2_validation.json) for exact inputs,
backend/storage results, numerical checks, route tests and build commands.
Backend/storage differences are measured rather than hidden by mandatory F32
kernel controls. Exact reference agreement is diagnostic; quantized storage,
provisional text and segmentation boundaries can differ. The sampled comparisons
do not establish corpus-level WER equivalence. RTX 5090, Metal, HIP and MUSA
runtime behavior was not tested locally.

Vulkan state/graph allocation optimizations are reviewed separately and are not
part of this minimal model PR. Raw previous and current evidence remains in
`outputs/phonon2-review-20261010/` and the earlier local Phonon artifact folders.