voice_audio_multi/tests/realtime_reuse/README.md
Fabio c36510c21e
Some checks failed
Linux Build / Linux cpu (push) Has been cancelled
Linux Build / Linux cpu-clang (push) Has been cancelled
Linux Build / Linux vulkan (push) Has been cancelled
mac-build / macOS CPU (push) Has been cancelled
Nix Build / Nix (cpu) (push) Has been cancelled
Nix Build / Nix (vulkan) (push) Has been cancelled
windows-build / Windows CPU (push) Has been cancelled
Initial import: audio.cpp with Orange Pi documentation
2026-11-04 19:05:22 +01:00

2 KiB

Real-time RE-USE validation

Use the original checkpoint from nvidia/Real-time_RE-USE. The converter checks its SHA256 and verifies the complete tensor inventory, shapes, and retained F32 tensor bytes after packaging.

python tests/realtime_reuse/convert_gguf.py --source SOURCE \
  --output models/Real-time-RE-USE-GGUF/realtime-reuse-f32.gguf \
  --log conversion.log

run_python_reference.py --help describes reference generation with the original Python implementation. Use --no-tf32 for the strict F32 waveform comparison; leave that diagnostic switch off for ordinary Python performance measurements. --reference-file reference.safetensors exports the waveform, frontend features, and expected restored waveform together with the selected configuration.

build/debug/bin/test_realtime_reuse_reference \
  --model models/Real-time-RE-USE-GGUF/realtime-reuse-f32.gguf \
  --reference reference.safetensors --backend cuda --log --log-file parity.log

build/debug/bin/test_realtime_reuse_streaming \
  --model models/Real-time-RE-USE-GGUF/realtime-reuse-f32.gguf \
  --audio noisy.wav --backend cuda --sample-rate 8000 --all-configs \
  --log --log-file streaming.log

The reference test compares the final waveform after supplying the official frontend features to the C++ network (cosine greater than 0.99999). It also logs frontend differences separately. The streaming test covers all 30 exit/lookahead combinations, checks early emission, input-fragmentation invariance, exact sample counts, finite samples, long-short-long repeatability, and streaming/offline waveform similarity through the 1→2→4→8 frame batch transitions and the padded final batch. The first 40 ms are excluded from the last comparison because upstream online inference deliberately skips the first-frame sweep filter.

cli_cases.json supplies offline, streaming, request-reuse, and long-form cases for the existing CLI path runner. Run resource measurements sequentially.