voice_audio_multi/tests/ming3_audio/README.md

Ming3 Audio Tests

Use the Debug binaries in build/debug, conda qwen3-tts, and saved --log output. Run measurements sequentially. Test CUDA before Vulkan; no CPU inference is required by this port.

Official Fixtures

Clone Ming-3.0-audio-flash into reference/Ming-3.0-audio-flash. Download the pinned Tiny checkpoint and make it available at reference/Ming-3.0-audio-tiny (a symlink is sufficient). The reference voice and exact transcript come from that checkpoint's voice_presets directory.

The CLI consumes WAV, so decode the two complete MP3 fixtures once for both implementations. Do not trim, resample, or shorten them:

ffmpeg -i reference/Ming-3.0-audio-flash/examples/audio/meeting.mp3 \
  -c:a pcm_f32le reference/Ming-3.0-audio-flash/examples/audio/meeting.wav
ffmpeg -i reference/Ming-3.0-audio-flash/examples/audio/music.mp3 \
  -c:a pcm_f32le reference/Ming-3.0-audio-flash/examples/audio/music.wav

Upstream example	Case / Request
Short ASR	official_asr: ant_group, ant_group_repeat
Native Thinker + Talker, US_Stella	official_spoken_qa: ip_us_stella
Spoken QA with reference audio and transcript	official_spoken_qa: reference_us_james
Spoken QA with the checkpoint's Chinese reference preset	official_spoken_qa: reference_lingkai
Complete music understanding fixture	official_understanding: music
Complete meeting transcription and summary	official_understanding: meeting
Complete meeting, thinking enabled	official_understanding: meeting_thinking
Bundled tool-calling audio asset	official_asr: tool_audio_transcription (input/transcription coverage only, not a structured tool-call API)

The short ASR case reproduces the literal prompt and 256-token limit. Upstream's 10-gram repetition filter cannot activate on its six-token result; this test does not claim a native repetition-filter implementation. The vLLM/SGLang server and streaming transports are separate from this offline model integration.

Q8_0 quantizes only Thinker expert banks. Q4_K also uses a Q8_0 text output head, reducing GPU-resident weights during expert offload. Token embeddings, attention and dense/shared MLP projections retain source precision. Both keep routers F32 and preserve the source precision of the audio encoder, Talker, VAE and speaker encoder. Broader projection quantization caused repetition or prolonged generation on the full meeting cases.

CUDA Runs

Run the same cases independently for BF16, Q8_0, and Q4_K. Set GGUF to the selected standalone file and use a distinct output directory for each:

conda run --no-capture-output -n qwen3-tts \
  python tools/audiocpp_cli/run_audiocpp_cli_path_tests.py \
  --cases tests/ming3_audio/cli_official_cases.json \
  --audiocpp-cli-bin build/debug/bin/audiocpp_cli --model-path "$GGUF" \
  --backend cuda --model-spec-override embedded --measure-resources \
  --out-root "$OUTPUT" --log

--only official_asr, --only official_spoken_qa, or --only official_understanding selects an individual group. For expert offload, append --session-option ming3_audio.disk_experts=true. The selected GGUF must embed the current spec before using newly added options without a spec override.

Also run cli_asr_cases.json, cli_audio_lifecycle_cases.json, cli_reference_cases.json, and cli_long_tts_cases.json for longer ASR, complete music/short-speech alternation, changed reference voices, and chunked long-short-long generation. Compare repeated WAVs and transcribe generated audio; a successful exit alone does not prove valid output or cache isolation. For offload cache growth, run cli_context_lifecycle_cases.json with ming3_audio.disk_experts=true and --resource-sample-ms 20. It repeats the complete official meeting around a short request, retaining the 32,768-token generation allowance. Check that the cache grows, both meeting responses match, and the short transcript is unchanged after the meeting. Review complete stdout.log responses for multiline text: the runner's text_outputs.json currently records only their first line. The CLI's --text-out also saves complete text when running cases directly.

Official Python Boundary

python_reference.py uses the upstream example's message builder and processing function. Pass a runner-produced requests.json to exercise exactly the same prompts, audio, token limits, and thinking flag. --capture saves the backbone's actual merged prefill input and final output, not separately recomputed embeddings or per-step logits. Do not use captured runs as performance measurements.

ming3_audio_thinker_parity accepts --capacity to hold context allocation fixed, --prefill-tokens to feed that same fixture in smaller pieces, and --disk-experts to test staged experts. These are component diagnostics, not standalone-GGUF acceptance tests. Compare completed generated-token sequences; do not use internal logits or shortened generation limits as a quality pass.

python_talker_reference.py captures the Talker input and its final latents and waveform. ming3_audio_talker_parity teacher-forces those final latents into the native VAE and retains the strict waveform gate. Record any explicitly accepted BF16 exception separately rather than changing that threshold.

For reference-conditioned fixtures, pass --reference-voice, --reference-text, --text, and --language to the same parity test. Capture includes prompt IDs, speaker features, reference latents, and the initial CUDA RNG offset. Optional --generated-output runs normal generation from those reference features; --official-inputs selects the captured rather than native-prepared prompt embeddings for that control. These controls do not replace end-to-end tests with native reference encoding. Input mismatch and the strict waveform gate remain test failures even when diagnostic WAVs are written.