Skip to content

Add cascade STT/LLM/TTS user simulator - #214

Closed
tara-bogavelli wants to merge 2 commits into
mainfrom
feat/cascade-user-simulator
Closed

tara-bogavelli wants to merge 2 commits into
mainfrom
feat/cascade-user-simulator

Conversation

@tara-bogavelli

@tara-bogavelli tara-bogavelli commented Aug 21, 2026 •

Copy link
Copy Markdown
Collaborator

Adds a standalone cascade user simulator that combines streaming STT, a caller LLM, and Cartesia TTS. The caller uses the shared persona/goal prompts and end-call tool, waits for replies, and records transcripts, audio boundaries, clean caller audio, and per-tick diagnostics.

Includes both real-time streaming and tick-driven OpenAI Realtime adapters, pacing selection, background-noise/connection perturbations, transcript fallback, and the latest inactivity/provider-stall fixes. Also removes unused Backend/Role scaffolding and documents simulator setup.

This PR is the independently mergeable base of a two-PR stack. All listener decisions, interruptions, backchannels, speculative generation, localized phrase caching, and barge-in truncation are in dependent draft PR #228, which targets this branch. Those flags and execution paths are absent here.

Validation:

  • pytest tests -q: 2,192 passed, 52 skipped, 3 xfailed.
  • pre-commit run --all-files: passed.
  • Added full-loop regressions for reply waiting, partial transcripts, hang-up, contiguous inactivity, and provider-stall artifact preservation.
  • Bounded Pipecat debug run exchanged four caller turns before the 90-second conversation limit; the smoke configuration omitted mandatory judge deployments, so final validation did not pass.
  • OpenAI Realtime base smoke failed during the upstream connection handshake. The dependent branch subsequently exercised this adapter live, including out-of-turn speech, but does not substitute for successful base end-to-end validation.

Rebuilt on current main (4cd0028b). The original branch tip is preserved locally as backup/cascade-user-simulator-20260916 (8467d0a4).

@tara-bogavelli
tara-bogavelli marked this pull request as draft August 21, 2026 00:03
@tara-bogavelli
tara-bogavelli force-pushed the feat/cascade-user-simulator branch from 8467d0a to 1860af9 Compare September 16, 2026 20:19
@tara-bogavelli tara-bogavelli changed the title new user simulator Add cascade STT/LLM/TTS user simulator Sep 16, 2026
@JosephMarinier

Copy link
Copy Markdown
Collaborator

Closing in favor of #240

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants