# Moonshine Streaming ASR

Build with `-DENGINE_ENABLE_NEON=ON` (default OFF) to enable the native path.
On ARM CPU, `--session-option 'moonshine_asr.exec_plan={"decoder":{"impl":"neon"}}'` selects the complete
NEON decoder, including embedding lookup, cross-KV preparation and cached
autoregressive steps. The encoder and sampling policy stay unchanged. Omission,
an empty plan, or `decoder.impl=default/ggml` preserves the graph path. Explicit
NEON requires an AArch64 CPU session. No artifacts or params are accepted.

[Moonshine](https://huggingface.co/moonshine-ai) is an English streaming ASR
family. audio.cpp supports the upstream tiny, small, and medium streaming
checkpoints through the `moonshine_asr` family.

## Install

The recommended packages are standalone Q8_0 GGUFs in the aggregate audio.cpp
GGUF repo:

```bash
python3 tools/model_manager_v2.py install moonshine_streaming_tiny_q8_0
python3 tools/model_manager_v2.py install moonshine_streaming_small_q8_0
python3 tools/model_manager_v2.py install moonshine_streaming_medium_q8_0
```

The GGUFs embed the model spec, configuration, and tokenizer sidecars, so each
file can be loaded directly.

## Quick Start

Offline transcription:

```bash
audiocpp_cli --task asr \
  --family moonshine_asr \
  --model models/Moonshine-Streaming-GGUF/moonshine-streaming-tiny-q8_0.gguf \
  --backend cuda \
  --audio assets/resources/sample_16k.wav \
  --text-out transcript.txt \
  --log
```

Streaming session with file input:

```bash
audiocpp_cli --task asr --mode streaming \
  --family moonshine_asr \
  --model models/Moonshine-Streaming-GGUF/moonshine-streaming-tiny-q8_0.gguf \
  --backend cuda \
  --audio assets/resources/sample_16k.wav \
  --text-out transcript.txt \
  --log
```

## Model

| Field | Value |
|---|---|
| Family | `moonshine_asr` |
| Task | `asr` |
| Modes | `offline`, `streaming` |
| Language | English |
| Recommended package | `moonshine_streaming_tiny_q8_0` |
| Output | Transcript text |
| Timestamps | Not exposed |

## Common Options (use directly)

| Option | Values | Default | Meaning |
|---|---|---:|---|
| `--audio` | WAV path | required | Speech input. |
| `--max-tokens` | integer | audio-duration derived | Maximum generated transcript tokens. Defaults to audio-duration derived limit. Nonpositive values select the automatic budget. |
| `--request-option audio_chunk_mode=...` | `auto`, `fixed`, `vad`, `none` | `auto` | Long-form audio chunking policy. |
| `--request-option audio_chunk_duration_sec=...` | seconds | `60` | Maximum chunk duration for `auto`, `fixed`, and `vad`. |
| `--text-out` | TXT path | not set | Transcript output. |

## Session Options (use with `--session-option`)

| Option | Values | Default | Meaning |
|---|---|---:|---|
| `moonshine_asr.weight_type` | `native`, `f32`, `f16` | backend/config dependent | Shared matmul storage override. When omitted, the CPU encoder with hidden size 620, 10 layers and decoder hidden size 512 uses f16; other encoders use native. |
| `moonshine_asr.conv_weight_type` | `native`, `f32`, `f16` | `native` | Frontend convolution weight storage override. |
| `moonshine_asr.decoder_weight_type` | `native`, `f32`, `f16` | explicit shared setting, otherwise `native` | Decoder matmul storage override. |
| `moonshine_asr.encoder_gelu` | `erf`, `exact`, `tanh`, `quick` | `quick` | Encoder GELU lowering. |
| `moonshine_asr.cpu_blas_scheduler` | bool | `true` | Use BLAS/Accelerate for supported CPU encoder matmuls. |
| `moonshine_asr.weight_context_mb` | integer MiB >= 1 | `256` | Weight context arena size. |
| `moonshine_asr.graph_arena_mb` | integer MiB >= 1 | `512` | Graph arena size. |
| `moonshine_asr.vad_model_path` | path | `assets/framework/models/silero_vad` | Silero VAD model path used by `audio_chunk_mode=vad`. |

## Conversion

The helper converts the upstream safetensors checkpoints into standalone GGUFs:

```bash
python3 tools/community_models/convert_moonshine_asr.py \
  --checkpoint models/moonshine-streaming-tiny \
  --converter build/debug/bin/audiocpp_gguf \
  --output models/Moonshine-Streaming-GGUF/moonshine-streaming-tiny-q8_0.gguf \
  --type q8_0 \
  --overwrite
```

Use the same command shape for `moonshine-streaming-small` and
`moonshine-streaming-medium`.

## Sources

- Tiny: <https://huggingface.co/moonshine-ai/moonshine-streaming-tiny>
- Small: <https://huggingface.co/moonshine-ai/moonshine-streaming-small>
- Medium: <https://huggingface.co/moonshine-ai/moonshine-streaming-medium>