Fabio
c36510c21e
Some checks failed
Linux Build / Linux cpu (push) Has been cancelled
Linux Build / Linux cpu-clang (push) Has been cancelled
Linux Build / Linux vulkan (push) Has been cancelled
mac-build / macOS CPU (push) Has been cancelled
Nix Build / Nix (cpu) (push) Has been cancelled
Nix Build / Nix (vulkan) (push) Has been cancelled
windows-build / Windows CPU (push) Has been cancelled
109 lines
4.4 KiB
Markdown
109 lines
4.4 KiB
Markdown
# Moonshine Streaming ASR
|
|
|
|
Build with `-DENGINE_ENABLE_NEON=ON` (default OFF) to enable the native path.
|
|
On ARM CPU, `--session-option 'moonshine_asr.exec_plan={"decoder":{"impl":"neon"}}'` selects the complete
|
|
NEON decoder, including embedding lookup, cross-KV preparation and cached
|
|
autoregressive steps. The encoder and sampling policy stay unchanged. Omission,
|
|
an empty plan, or `decoder.impl=default/ggml` preserves the graph path. Explicit
|
|
NEON requires an AArch64 CPU session. No artifacts or params are accepted.
|
|
|
|
[Moonshine](https://huggingface.co/moonshine-ai) is an English streaming ASR
|
|
family. audio.cpp supports the upstream tiny, small, and medium streaming
|
|
checkpoints through the `moonshine_asr` family.
|
|
|
|
## Install
|
|
|
|
The recommended packages are standalone Q8_0 GGUFs in the aggregate audio.cpp
|
|
GGUF repo:
|
|
|
|
```bash
|
|
python3 tools/model_manager_v2.py install moonshine_streaming_tiny_q8_0
|
|
python3 tools/model_manager_v2.py install moonshine_streaming_small_q8_0
|
|
python3 tools/model_manager_v2.py install moonshine_streaming_medium_q8_0
|
|
```
|
|
|
|
The GGUFs embed the model spec, configuration, and tokenizer sidecars, so each
|
|
file can be loaded directly.
|
|
|
|
## Quick Start
|
|
|
|
Offline transcription:
|
|
|
|
```bash
|
|
audiocpp_cli --task asr \
|
|
--family moonshine_asr \
|
|
--model models/Moonshine-Streaming-GGUF/moonshine-streaming-tiny-q8_0.gguf \
|
|
--backend cuda \
|
|
--audio assets/resources/sample_16k.wav \
|
|
--text-out transcript.txt \
|
|
--log
|
|
```
|
|
|
|
Streaming session with file input:
|
|
|
|
```bash
|
|
audiocpp_cli --task asr --mode streaming \
|
|
--family moonshine_asr \
|
|
--model models/Moonshine-Streaming-GGUF/moonshine-streaming-tiny-q8_0.gguf \
|
|
--backend cuda \
|
|
--audio assets/resources/sample_16k.wav \
|
|
--text-out transcript.txt \
|
|
--log
|
|
```
|
|
|
|
## Model
|
|
|
|
| Field | Value |
|
|
|---|---|
|
|
| Family | `moonshine_asr` |
|
|
| Task | `asr` |
|
|
| Modes | `offline`, `streaming` |
|
|
| Language | English |
|
|
| Recommended package | `moonshine_streaming_tiny_q8_0` |
|
|
| Output | Transcript text |
|
|
| Timestamps | Not exposed |
|
|
|
|
## Common Options (use directly)
|
|
|
|
| Option | Values | Default | Meaning |
|
|
|---|---|---:|---|
|
|
| `--audio` | WAV path | required | Speech input. |
|
|
| `--max-tokens` | integer | audio-duration derived | Maximum generated transcript tokens. Defaults to audio-duration derived limit. Nonpositive values select the automatic budget. |
|
|
| `--request-option audio_chunk_mode=...` | `auto`, `fixed`, `vad`, `none` | `auto` | Long-form audio chunking policy. |
|
|
| `--request-option audio_chunk_duration_sec=...` | seconds | `60` | Maximum chunk duration for `auto`, `fixed`, and `vad`. |
|
|
| `--text-out` | TXT path | not set | Transcript output. |
|
|
|
|
## Session Options (use with `--session-option`)
|
|
|
|
| Option | Values | Default | Meaning |
|
|
|---|---|---:|---|
|
|
| `moonshine_asr.weight_type` | `native`, `f32`, `f16` | backend/config dependent | Shared matmul storage override. When omitted, the CPU encoder with hidden size 620, 10 layers and decoder hidden size 512 uses f16; other encoders use native. |
|
|
| `moonshine_asr.conv_weight_type` | `native`, `f32`, `f16` | `native` | Frontend convolution weight storage override. |
|
|
| `moonshine_asr.decoder_weight_type` | `native`, `f32`, `f16` | explicit shared setting, otherwise `native` | Decoder matmul storage override. |
|
|
| `moonshine_asr.encoder_gelu` | `erf`, `exact`, `tanh`, `quick` | `quick` | Encoder GELU lowering. |
|
|
| `moonshine_asr.cpu_blas_scheduler` | bool | `true` | Use BLAS/Accelerate for supported CPU encoder matmuls. |
|
|
| `moonshine_asr.weight_context_mb` | integer MiB >= 1 | `256` | Weight context arena size. |
|
|
| `moonshine_asr.graph_arena_mb` | integer MiB >= 1 | `512` | Graph arena size. |
|
|
| `moonshine_asr.vad_model_path` | path | `assets/framework/models/silero_vad` | Silero VAD model path used by `audio_chunk_mode=vad`. |
|
|
|
|
## Conversion
|
|
|
|
The helper converts the upstream safetensors checkpoints into standalone GGUFs:
|
|
|
|
```bash
|
|
python3 tools/community_models/convert_moonshine_asr.py \
|
|
--checkpoint models/moonshine-streaming-tiny \
|
|
--converter build/debug/bin/audiocpp_gguf \
|
|
--output models/Moonshine-Streaming-GGUF/moonshine-streaming-tiny-q8_0.gguf \
|
|
--type q8_0 \
|
|
--overwrite
|
|
```
|
|
|
|
Use the same command shape for `moonshine-streaming-small` and
|
|
`moonshine-streaming-medium`.
|
|
|
|
## Sources
|
|
|
|
- Tiny: <https://huggingface.co/moonshine-ai/moonshine-streaming-tiny>
|
|
- Small: <https://huggingface.co/moonshine-ai/moonshine-streaming-small>
|
|
- Medium: <https://huggingface.co/moonshine-ai/moonshine-streaming-medium>
|