From 7f2b650ff184caccd4acbafb3037130f2920da64 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E7=8E=8B=E7=BF=B0=E5=9D=A4?= Date: Mon, 26 Oct 2026 04:09:42 +0000 Subject: [PATCH 1/2] docs: update reference results --- README.md | 98 ++++++++++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 86 insertions(+), 12 deletions(-) diff --git a/README.md b/README.md index 6fb9d05..f64caeb 100644 --- a/README.md +++ b/README.md @@ -135,18 +135,92 @@ deployment parameters, and capability boundary beside the implementation: ## Reference results -These reference runs cover 2,081 cases. Operation success averages requested -operations; all-operations success requires every requested operation in a case -to succeed. AuK uses the official Prompt Enhancer. - -| Candidate | One-take operation success | Sequential operation success | One-take all operations | Sequential all operations | -|---|---:|---:|---:|---:| -| dots.tts.edit | **63.9%** | **62.7%** | **48.8%** | **48.2%** | -| MiMo-Audio-Instruct | 37.3% | 39.1% | 25.6% | 26.8% | -| Step-Audio-EditX | 36.0% | 31.1% | 24.9% | 20.6% | -| Ming-UniAudio | 18.7% | 18.8% | 8.7% | 10.5% | -| FireRedTTS3-Instruct | 16.0% | 20.5% | 8.3% | 11.6% | -| AuK | 23.6% | 33.3% | 12.3% | 21.9% | +Precise speech editing requires both successful execution of the requested +operations and preservation of the source speech. These reference results cover +all 2,081 cases across the five editing categories. Each value is the arithmetic +mean of the one-take and sequential results. Bold indicates the best result among +the listed systems. AuK uses the official Prompt Enhancer. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
CandidateInstruction FollowingLocal Preservation
Operation success (%) ↑All success (%) ↑SIM ↑WDTW-Dur (%) ↓WDTW-F0 (st) ↓
dots.tts.edit63.3048.490.7797.181.84
MiMo-Audio-Instruct38.2026.210.62315.254.63
Step-Audio-EditX33.5722.730.71113.663.41
AuK28.4317.110.7599.402.92
Ming-UniAudio18.739.610.51413.194.19
FireRedTTS3-Instruct18.269.970.7777.762.01
+ +Operation success measures the fraction of requested operations that succeed; +All success measures the fraction of cases in which every requested operation +succeeds. SIM measures speaker similarity between source and edited speech. +WDTW-Dur and WDTW-F0 measure duration and pitch changes over regions designated +for preservation by the instruction; st denotes semitones. + +Within each execution mode, operation success is pooled over requested operations, +while All success and SIM weight cases equally. WDTW metrics pool distances and +normalization terms before division. The two execution-mode scores are then averaged. + +On the 240 compositional editing cases, sequential execution increases +dots.tts.edit's operation success from 68.12% to 68.48% and All success from 37.92% +to 42.92%. One-take execution achieves higher speaker similarity and lower WDTW errors. The radars show instruction following, preservation, and quality. Compositional axes report Component and All Success ([evaluation protocol](docs/evaluation.md)). From b4e81901e17d504dc5c3fab8b89f01b0967570bc Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E7=8E=8B=E7=BF=B0=E5=9D=A4?= Date: Mon, 26 Oct 2026 04:13:03 +0000 Subject: [PATCH 2/2] docs: center reference result values --- README.md | 60 +++++++++++++++++++++++++++---------------------------- 1 file changed, 30 insertions(+), 30 deletions(-) diff --git a/README.md b/README.md index f64caeb..bc5b427 100644 --- a/README.md +++ b/README.md @@ -159,51 +159,51 @@ the listed systems. AuK uses the official Prompt Enhancer. dots.tts.edit - 63.30 - 48.49 - 0.779 - 7.18 - 1.84 + 63.30 + 48.49 + 0.779 + 7.18 + 1.84 MiMo-Audio-Instruct - 38.20 - 26.21 - 0.623 - 15.25 - 4.63 + 38.20 + 26.21 + 0.623 + 15.25 + 4.63 Step-Audio-EditX - 33.57 - 22.73 - 0.711 - 13.66 - 3.41 + 33.57 + 22.73 + 0.711 + 13.66 + 3.41 AuK - 28.43 - 17.11 - 0.759 - 9.40 - 2.92 + 28.43 + 17.11 + 0.759 + 9.40 + 2.92 Ming-UniAudio - 18.73 - 9.61 - 0.514 - 13.19 - 4.19 + 18.73 + 9.61 + 0.514 + 13.19 + 4.19 FireRedTTS3-Instruct - 18.26 - 9.97 - 0.777 - 7.76 - 2.01 + 18.26 + 9.97 + 0.777 + 7.76 + 2.01