Fabio
c36510c21e
Some checks failed
Linux Build / Linux cpu (push) Has been cancelled
Linux Build / Linux cpu-clang (push) Has been cancelled
Linux Build / Linux vulkan (push) Has been cancelled
mac-build / macOS CPU (push) Has been cancelled
Nix Build / Nix (cpu) (push) Has been cancelled
Nix Build / Nix (vulkan) (push) Has been cancelled
windows-build / Windows CPU (push) Has been cancelled
109 lines
6 KiB
Markdown
109 lines
6 KiB
Markdown
# Ming3 Audio Tests
|
|
|
|
Use the Debug binaries in `build/debug`, conda `qwen3-tts`, and saved `--log`
|
|
output. Run measurements sequentially. Test CUDA before Vulkan; no CPU inference
|
|
is required by this port.
|
|
|
|
## Official Fixtures
|
|
|
|
Clone [Ming-3.0-audio-flash](https://github.com/inclusionAI/Ming-3.0-audio-flash)
|
|
into `reference/Ming-3.0-audio-flash`. Download the pinned
|
|
[Tiny checkpoint](https://huggingface.co/inclusionAI/Ming-3.0-audio-tiny/tree/25b96e30be3739849bf76c0f48ac11db4a262fb3)
|
|
and make it available at `reference/Ming-3.0-audio-tiny` (a symlink is sufficient).
|
|
The reference voice and exact transcript come from that checkpoint's
|
|
`voice_presets` directory.
|
|
|
|
The CLI consumes WAV, so decode the two complete MP3 fixtures once for both
|
|
implementations. Do not trim, resample, or shorten them:
|
|
|
|
```bash
|
|
ffmpeg -i reference/Ming-3.0-audio-flash/examples/audio/meeting.mp3 \
|
|
-c:a pcm_f32le reference/Ming-3.0-audio-flash/examples/audio/meeting.wav
|
|
ffmpeg -i reference/Ming-3.0-audio-flash/examples/audio/music.mp3 \
|
|
-c:a pcm_f32le reference/Ming-3.0-audio-flash/examples/audio/music.wav
|
|
```
|
|
|
|
| Upstream example | Case / Request |
|
|
| --- | --- |
|
|
| Short ASR | `official_asr`: `ant_group`, `ant_group_repeat` |
|
|
| Native Thinker + Talker, US_Stella | `official_spoken_qa`: `ip_us_stella` |
|
|
| Spoken QA with reference audio and transcript | `official_spoken_qa`: `reference_us_james` |
|
|
| Spoken QA with the checkpoint's Chinese reference preset | `official_spoken_qa`: `reference_lingkai` |
|
|
| Complete music understanding fixture | `official_understanding`: `music` |
|
|
| Complete meeting transcription and summary | `official_understanding`: `meeting` |
|
|
| Complete meeting, thinking enabled | `official_understanding`: `meeting_thinking` |
|
|
| Bundled tool-calling audio asset | `official_asr`: `tool_audio_transcription` (input/transcription coverage only, not a structured tool-call API) |
|
|
|
|
The short ASR case reproduces the literal prompt and 256-token limit. Upstream's
|
|
10-gram repetition filter cannot activate on its six-token result; this test
|
|
does not claim a native repetition-filter implementation. The vLLM/SGLang server
|
|
and streaming transports are separate from this offline model integration.
|
|
|
|
Q8_0 quantizes only Thinker expert banks. Q4_K also uses a Q8_0 text output
|
|
head, reducing GPU-resident weights during expert offload. Token embeddings,
|
|
attention and dense/shared MLP projections retain source precision.
|
|
Both keep routers F32 and
|
|
preserve the source precision of the audio encoder, Talker, VAE and speaker
|
|
encoder. Broader projection quantization caused repetition or prolonged
|
|
generation on the full meeting cases.
|
|
|
|
## CUDA Runs
|
|
|
|
Run the same cases independently for BF16, Q8_0, and Q4_K. Set `GGUF` to the
|
|
selected standalone file and use a distinct output directory for each:
|
|
|
|
```bash
|
|
conda run --no-capture-output -n qwen3-tts \
|
|
python tools/audiocpp_cli/run_audiocpp_cli_path_tests.py \
|
|
--cases tests/ming3_audio/cli_official_cases.json \
|
|
--audiocpp-cli-bin build/debug/bin/audiocpp_cli --model-path "$GGUF" \
|
|
--backend cuda --model-spec-override embedded --measure-resources \
|
|
--out-root "$OUTPUT" --log
|
|
```
|
|
|
|
`--only official_asr`, `--only official_spoken_qa`, or
|
|
`--only official_understanding` selects an individual group. For expert offload,
|
|
append `--session-option ming3_audio.disk_experts=true`. The selected GGUF must
|
|
embed the current spec before using newly added options without a spec override.
|
|
|
|
Also run `cli_asr_cases.json`, `cli_audio_lifecycle_cases.json`,
|
|
`cli_reference_cases.json`, and `cli_long_tts_cases.json` for longer ASR,
|
|
complete music/short-speech alternation, changed reference voices, and chunked
|
|
long-short-long generation. Compare repeated WAVs and transcribe generated audio;
|
|
a successful exit alone does not prove valid output or cache isolation.
|
|
For offload cache growth, run `cli_context_lifecycle_cases.json` with
|
|
`ming3_audio.disk_experts=true` and `--resource-sample-ms 20`. It repeats the
|
|
complete official meeting around a short request, retaining the 32,768-token
|
|
generation allowance. Check that the cache grows, both meeting responses match,
|
|
and the short transcript is unchanged after the meeting.
|
|
Review complete `stdout.log` responses for multiline text: the runner's
|
|
`text_outputs.json` currently records only their first line. The CLI's
|
|
`--text-out` also saves complete text when running cases directly.
|
|
|
|
## Official Python Boundary
|
|
|
|
`python_reference.py` uses the upstream example's message builder and processing
|
|
function. Pass a runner-produced `requests.json` to exercise exactly the same
|
|
prompts, audio, token limits, and thinking flag. `--capture` saves the backbone's
|
|
actual merged prefill input and final output, not separately recomputed
|
|
embeddings or per-step logits. Do not use captured runs as performance measurements.
|
|
|
|
`ming3_audio_thinker_parity` accepts `--capacity` to hold context allocation fixed,
|
|
`--prefill-tokens` to feed that same fixture in smaller pieces, and
|
|
`--disk-experts` to test staged experts. These are component diagnostics, not
|
|
standalone-GGUF acceptance tests. Compare completed generated-token sequences;
|
|
do not use internal logits or shortened generation limits as a quality pass.
|
|
|
|
`python_talker_reference.py` captures the Talker input and its final latents and
|
|
waveform. `ming3_audio_talker_parity` teacher-forces those final latents into the
|
|
native VAE and retains the strict waveform gate. Record any explicitly accepted
|
|
BF16 exception separately rather than changing that threshold.
|
|
|
|
For reference-conditioned fixtures, pass `--reference-voice`, `--reference-text`,
|
|
`--text`, and `--language` to the same parity test. Capture includes prompt IDs,
|
|
speaker features, reference latents, and the initial CUDA RNG offset. Optional
|
|
`--generated-output` runs normal generation from those reference features;
|
|
`--official-inputs` selects the captured rather than native-prepared prompt
|
|
embeddings for that control. These controls do not replace end-to-end tests with
|
|
native reference encoding. Input mismatch and the strict waveform gate remain
|
|
test failures even when diagnostic WAVs are written.
|