Skip to content

feat(export): linear audio mixdown for full-timeline export (refs Phase5/#112)#117

Merged
appergb merged 1 commit into
mainfrom
feat-export-audio
Jun 23, 2026
Merged

feat(export): linear audio mixdown for full-timeline export (refs Phase5/#112)#117
appergb merged 1 commit into
mainfrom
feat-export-audio

Conversation

@Felix201209

Copy link
Copy Markdown
Collaborator

What

Extends the #112 full-timeline video export spine with a linear audio mixdown, turning the dead push_audio side-channel into a wired decode → mix → mux path. A timeline with audio now exports an mp4 with a real AAC track; a timeline without audio produces the same video-only file as before.

How

  • opentake-media — new pure encode::mix module: lays each clip's mono f32 PCM at its frame-derived sample offset, applies a per-sample gain envelope, sums overlapping clips, and hard-limits to [-1.0, 1.0]. mono_f32_to_s16le produces the mux wire format. No ffmpeg/domain deps — fully unit-tested.
  • opentake-media encoder: finish now mux's a supplied mixed buffer via a second ffmpeg pass (-c:v copy + AAC/LPCM, -shortest). On mux failure it restores the video-only file rather than losing the export. mux_args is pure and unit-tested; temp artifacts are cleaned up.
  • export.rs orchestrator: decodes every audio/video clip's source window via the reused extract_pcm (mix rate, mono f32), builds the volume_at gain envelope, skips muted tracks and audio-less sources, mixes, and pushes to the encoder.
  • export_integration.rs: ffmpeg+GPU-gated test builds a 440 Hz sine-audio fixture, exports, and asserts ffprobe reads back an AAC audio stream plus cleaned-up temps. The video-only test now also asserts no audio stream leaks in.

Scope

Linear mixdown skeleton only — no resampling curve, pan/stereo field, or dynamics (follow-ups). H.264/.mp4 path; H.265/ProRes still gated as before.

Verification

  • cargo fmt --all clean
  • cargo clippy -p opentake-media -p opentake-tauri --all-targets -- -D warnings clean
  • cargo test -p opentake-media (24 encode/mix unit tests) green
  • cargo test -p opentake-tauri export — 10 unit + 2 ffmpeg+GPU-gated integration tests green (audio test genuinely ran, not skipped)

Hard constraints honored: Tauri boundary returns Err(String); opentake-domain untouched (zero new deps); mixdown is pure logic in opentake-media; serde fields unchanged; unit tests under #[cfg(test)].

🤖 Generated with Claude Code

…se5/#112)

Extend the #112 video export spine with a linear audio mixdown, replacing
the dead `push_audio` side-channel with a wired decode → mix → mux path.

- opentake-media: new pure `encode::mix` module — lay each clip's mono f32
  PCM at its frame-derived sample offset, apply a per-sample gain envelope,
  sum overlapping clips, hard-limit to [-1,1]. `mono_f32_to_s16le` for the
  mux wire format. Fully unit-tested, no ffmpeg/domain deps.
- opentake-media encoder: `finish` now mux's a supplied mixed buffer via a
  second ffmpeg pass (`-c:v copy` + AAC/LPCM `-shortest`), restoring the
  video-only file if the mux fails. `mux_args` is pure and unit-tested.
- export.rs orchestrator: decode every audio/video clip's source window with
  the reused `extract_pcm` (mix rate, mono f32), build the `volume_at` gain
  envelope, skip muted tracks and audio-less sources, mix and push to the
  encoder. No-audio timelines still produce the same video-only output.
- export_integration.rs: ffmpeg+GPU-gated test builds a sine-audio fixture,
  exports, and asserts ffprobe reads back an AAC audio stream + cleaned temps;
  video-only test now asserts no audio stream leaks in.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@Felix201209 Felix201209 requested a review from appergb as a code owner June 23, 2026 14:44
@Felix201209 Felix201209 self-assigned this Jun 23, 2026

@appergb appergb left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

审核通过 ✅(自动审核流程 · 主控亲审,子 Agent 因 API 529 过载)

  • #112 导出 spine 的线性音频混音扩展,真接通原 dead push_audio
  • mix.rs 纯混音正确:帧→样本偏移铺放、逐样本 volume_at 增益包络、叠加、硬限幅 [-1,1]、mono f32→s16le;12 单测覆盖空/单/偏移/重叠/溢出双轨/增益/长度校验/输出长度/编码与 clamp。
  • encode finish() mux:第二趟 ffmpeg(-c:v copy + AAC/LPCM + -shortest);mux 失败回退视频-only 文件(不丢导出)、临时件 best-effort 清理;mux_args/sibling_temp 纯函数有测试。
  • export.rs 编排:解码音/视频 clip 源窗(extract_pcm,mono f32 @48k)→增益包络→混音→push;跳过静音轨/无音源/NoTrack→静音;视频路径未改(-18 仅文档/import),无音频仍产出与 #112 相同视频-only 文件。
  • 集成测试:加 440Hz 正弦 fixture 断言 AAC 轨 + 视频-only 断言不漏音轨 + 临时件清理。不碰 #91 区,CI 双绿。

follow-up(非阻塞):project_clip_audio ~60 行略超 50 规约;本切片仅线性混音(无重采样曲线/pan/立体声/动态),按设计后续。

@appergb appergb merged commit 5eb70c2 into main Jun 23, 2026
2 checks passed
@appergb appergb deleted the feat-export-audio branch June 23, 2026 14:57
appergb added a commit that referenced this pull request Jun 23, 2026
docs: 导出线性音频混音(#117)归档 + Phase 5 进度
H-Chris233 pushed a commit to H-Chris233/OpenTake that referenced this pull request Jun 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants