Q-SPT is a 6.25 Hz dual-stream speech tokenizer for speech language models. Learnable queries compress the semantic and acoustic streams separately, and an autoregressive text loss keeps linguistic information in the semantic stream.
- 16 kHz speech ↔ 6.25 Hz tokens
- 8 token streams per frame: 1 semantic stream (FSQ, 32768 codes) + 7 acoustic streams (RVQ, 4096 codes each)
- The semantic stream is built on a frozen SenseVoiceSmall encoder
conda create -n qspt python=3.10 -y
conda activate qspt
git clone https://github.com/KU-AGI/Q-SPT.git
cd Q-SPT
pip install torch==2.7.1 torchaudio==2.7.1 --index-url https://download.pytorch.org/whl/cu118
pip install -e .python inference.py --input samples/1089-134686-0000.flac --output recon.wavOptions: --num_quantizers N (1–8, default 8), --save_codes codes.pt, --model DIR (local weights).
import torchaudio
from qspt import QSPT
codec = QSPT.from_pretrained("KU-AGI/Q-SPT").cuda()
wav, sr = torchaudio.load("samples/1089-134686-0000.flac")
codes = codec.encode(wav, sr) # (1, 8, N) tokens at 6.25 Hz; codes[:, 0] is the semantic stream
recon = codec.decode(codes) # (1, 1, ~N * 2560) waveform at 16 kHz
torchaudio.save("recon.wav", recon[0].cpu(), 16000)
semantic_only = codec.decode(codes[:, :1]) # reconstruction from the semantic stream only@article{yun2026qspt,
title={Q-SPT: Learnable Query-Based Compression for Low-Frame-Rate Speech Tokenization},
author={Yun, Jeeyoung and Yun, Seohwan and Kim, Sungwoong},
journal={arXiv preprint arXiv:2610.01492},
year={2026}
}We thank the authors of DualCodec, FlexiCodec, and SenseVoice for making their code and models available.
