Skip to content
KU-AGIPublic

About

Official implementation of "Q-SPT: Learnable Query-Based Compression for Low-Frame-Rate Speech Tokenization"

Resources

Stars

2 stars

Watchers

0 watching

Forks

Latest commit

 

History

1 Commit

Folders and files

Repository files navigation

Q-SPT: Learnable Query-Based Compression for Low-Frame-Rate Speech Tokenization

arXiv Hugging Face

Q-SPT is a 6.25 Hz dual-stream speech tokenizer for speech language models. Learnable queries compress the semantic and acoustic streams separately, and an autoregressive text loss keeps linguistic information in the semantic stream.

  • 16 kHz speech ↔ 6.25 Hz tokens
  • 8 token streams per frame: 1 semantic stream (FSQ, 32768 codes) + 7 acoustic streams (RVQ, 4096 codes each)
  • The semantic stream is built on a frozen SenseVoiceSmall encoder

Q-SPT

Installation

conda create -n qspt python=3.10 -y
conda activate qspt

git clone https://github.com/KU-AGI/Q-SPT.git
cd Q-SPT
pip install torch==2.7.1 torchaudio==2.7.1 --index-url https://download.pytorch.org/whl/cu118
pip install -e .

Inference

python inference.py --input samples/1089-134686-0000.flac --output recon.wav

Options: --num_quantizers N (1–8, default 8), --save_codes codes.pt, --model DIR (local weights).

import torchaudio
from qspt import QSPT

codec = QSPT.from_pretrained("KU-AGI/Q-SPT").cuda()

wav, sr = torchaudio.load("samples/1089-134686-0000.flac")
codes = codec.encode(wav, sr)   # (1, 8, N) tokens at 6.25 Hz; codes[:, 0] is the semantic stream
recon = codec.decode(codes)     # (1, 1, ~N * 2560) waveform at 16 kHz
torchaudio.save("recon.wav", recon[0].cpu(), 16000)

semantic_only = codec.decode(codes[:, :1])  # reconstruction from the semantic stream only

Citation

@article{yun2026qspt,
  title={Q-SPT: Learnable Query-Based Compression for Low-Frame-Rate Speech Tokenization},
  author={Yun, Jeeyoung and Yun, Seohwan and Kim, Sungwoong},
  journal={arXiv preprint arXiv:2610.01492},
  year={2026}
}

Acknowledgement

We thank the authors of DualCodec, FlexiCodec, and SenseVoice for making their code and models available.

About

Official implementation of "Q-SPT: Learnable Query-Based Compression for Low-Frame-Rate Speech Tokenization"

Resources

Stars

2 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages