Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions src-tauri/crates/berd-call/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -164,7 +164,7 @@ berd-call session --stt-backend openai --stt-url wss://proxy.example/v1/realtime

In an Expert–Spokesperson call, the Expert reaches you through the Spokesperson: `berd-call message-spokesperson --re CURSOR [--resolves HANDOFF_ID]... TEXT` has it tell you in its own voice. Like the directives of Berd's Expert, the message must acknowledge the latest input (the Spokesperson's own speech doesn't count), or it comes back `spoke:false` with what was missed, and a handoff it names settles once the Spokesperson has said it. `speak` is refused unless the call was started with `--allow-direct-agent-speech`. With it, the Expert can also speak for itself: berd-call plays each `speak` reply through the configured TTS backend, taking turns with the Spokesperson's audio, and your speech interrupts either of them. Afterwards the Spokesperson gets a silent note of what you heard, so it can follow up without repeating it. A reply has to acknowledge everything you've said, as `speak --re` always does. Such a call holds no handoff open: each still reaches the Expert as a `handoff` row, but nothing waits for an answer, reminds the Expert, or holds back renewing or recovering the Spokesperson's session, and there is nothing for `--resolves` to name. `--voice`, `--rate`, `status` and `settings --tts` describe and change the Expert's voice. The Spokesperson keeps its own voice, shown as `session.spokesperson`: choose it with `--spokesperson-voice NAME`, at `start` or live with `settings`, and the menu bar offers both voices. It speaks as close to the agents' `--rate` as it can (0.25–1.5×), so the menu bar's one Speech Rate sets both; `--spokesperson-rate` gives it a speed of its own. Saved settings keep the Spokesperson's voice for the next call whose replies have their own, but not `--allow-direct-agent-speech`, so pass that on each `start`.

The three URL flags take full endpoints and override only their matching service for that call. Expert–Spokesperson uses `--realtime-url` for both input and output; `--stt-url` applies only to conventional mode, and `--tts-url` also applies to the Expert's own voice. Without an override, each service uses its OpenAI endpoint (or its existing environment override). `OPENAI_API_KEY` supplies the CLI credential; the desktop app stores keys separately by endpoint URL in Keychain.
The three URL flags take full endpoints and override only their matching service for that call. Expert–Spokesperson uses `--realtime-url` for both input and output; `--stt-url` applies only to conventional mode, and `--tts-url` also applies to the Expert's own voice. Without an override, each service uses its OpenAI endpoint (or its existing environment override). Each service reads its own key, `OPENAI_REALTIME_API_KEY` (`--realtime-url`), `OPENAI_STT_API_KEY` (`--stt-url`) or `OPENAI_TTS_API_KEY` (`--tts-url`), and falls back to `OPENAI_API_KEY` when it is unset or blank; the desktop app stores keys separately by endpoint URL in Keychain.

The default Siri backend still requires an exact installed voice name and
language. Missing or unavailable Siri voice configuration and an unavailable
Expand All @@ -191,7 +191,7 @@ berd-call synthesize --tts-backend openai --model gpt-4o-mini-tts \

The command rejects an existing output before constructing the backend, writes to a same-directory temporary file, synchronizes the completed WAV, and publishes it without clobbering a target that appears concurrently. Failure leaves no partial target. Input text is nonempty and at most 16 KiB; output is bounded to ten minutes of finite source PCM in `[-1, 1]`. Backend cancellation, empty or invalid PCM, and a non-1x PCM playback specification fail rather than publishing a misleading file.

Pocket rendering supports only rate `1.0`: its other rates are a host playback time-stretch policy and are not encoded into synthesized PCM. Siri applies its rate during native synthesis. OpenAI requires an explicit model, voice, and `--allow-paid-openai`; one invocation makes at most one request and reads the credential only from `OPENAI_API_KEY` after output preflight.
Pocket rendering supports only rate `1.0`: its other rates are a host playback time-stretch policy and are not encoded into synthesized PCM. Siri applies its rate during native synthesis. OpenAI requires an explicit model, voice, and `--allow-paid-openai`; one invocation makes at most one request and reads the credential only from `OPENAI_TTS_API_KEY`, else `OPENAI_API_KEY`, after output preflight.

Success emits one schema-version-one JSON line with the public backend identity, requested rate, and WAV encoding, sample rate, channels, bit depth, source frames, duration, and byte count. It never serializes the prompt, credential, endpoint, Pocket bundle path, or temporary path. Operation failure emits one sanitized error line and exits 1; usage failure emits no JSON and exits 2. Stdout is reserved for this machine-readable terminal record, not audio data.

Expand Down Expand Up @@ -390,7 +390,7 @@ environment, but never includes the URL.
OpenAI benchmarking is disabled unless the command includes
`--allow-paid-openai`. The CLI preflights the full workload, including the warm
mode's extra request, and rejects more than 20 requests or 65,536 total prompt
bytes before constructing the backend. A missing `OPENAI_API_KEY` still fails as
bytes before constructing the backend. A missing `OPENAI_TTS_API_KEY` and `OPENAI_API_KEY` still fails as
a structured initialization error without making a request.

## STT benchmarks
Expand Down Expand Up @@ -437,7 +437,7 @@ punctuation to whitespace, and reports substitutions, deletions, and insertions
alongside the aggregate rate.

OpenAI STT benchmarking requires `--allow-paid-openai` and reads its key only
from `OPENAI_API_KEY`. Before resolving credentials or connecting, the CLI
from `OPENAI_STT_API_KEY`, else `OPENAI_API_KEY`. Before resolving credentials or connecting, the CLI
rejects a warmup-inclusive workload above 20 recognition commits or 120 seconds
of streamed PCM. Endpoint and model overrides use the same environment variables
as the session; reports record only which source supplied them and never include
Expand Down
23 changes: 12 additions & 11 deletions src-tauri/crates/berd-call/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6842,7 +6842,11 @@ fn stt_benchmark_target(config: &SttBackendConfig) -> Result<SttBenchmarkTarget,
vad_threshold: 0.5,
endpoint_source: Some(endpoint_source.into()),
model_source: Some(model_source.into()),
credential_source: Some("OPENAI_API_KEY environment variable".into()),
credential_source: Some(format!(
"{} environment variable",
berd_call::openai::openai_key_var("OPENAI_STT_API_KEY")
.unwrap_or("OPENAI_STT_API_KEY")
)),
})
}
}
Expand Down Expand Up @@ -6887,10 +6891,8 @@ fn create_openai_tts_configuration(
voice: String,
endpoint: Option<&str>,
) -> Result<TtsConfiguration, String> {
let api_key = std::env::var("OPENAI_API_KEY")
.ok()
.filter(|key| !key.trim().is_empty())
.ok_or_else(|| "OPENAI_API_KEY is required".to_string())?;
let api_key = berd_call::openai::openai_key("OPENAI_TTS_API_KEY")
.ok_or_else(|| "OPENAI_TTS_API_KEY or OPENAI_API_KEY is required".to_string())?;
let base =
std::env::var("OPENAI_BASE_URL").unwrap_or_else(|_| "https://api.openai.com/v1".into());
Ok(TtsConfiguration::openai(
Expand Down Expand Up @@ -7148,10 +7150,9 @@ fn create_input_runtime(
}
}
SttBackendConfig::OpenAi => {
let api_key = std::env::var("OPENAI_API_KEY")
.ok()
.filter(|key| !key.trim().is_empty())
.ok_or_else(|| "OPENAI_API_KEY is required for OpenAI STT".to_string())?;
let api_key = berd_call::openai::openai_key("OPENAI_STT_API_KEY").ok_or_else(|| {
"OPENAI_STT_API_KEY or OPENAI_API_KEY is required for OpenAI STT".to_string()
})?;
let endpoint = endpoint_override
.map(str::to_string)
.or_else(|| {
Expand Down Expand Up @@ -7293,7 +7294,7 @@ fn public_tts_rejection_message(kind: TtsConfigurationRejectionKind) -> &'static
fn public_tts_startup_error(config: &TtsBackendConfig) -> String {
match config {
TtsBackendConfig::OpenAi { .. } => {
"OpenAI TTS could not initialize; verify OPENAI_API_KEY and the selected model and voice"
"OpenAI TTS could not initialize; verify OPENAI_TTS_API_KEY or OPENAI_API_KEY and the selected model and voice"
.into()
}
TtsBackendConfig::Siri { .. } =>
Expand All @@ -7315,7 +7316,7 @@ fn public_stt_startup_error(config: &SttBackendConfig) -> String {
.into()
}
SttBackendConfig::OpenAi => {
"OpenAI speech recognition could not initialize; verify OPENAI_API_KEY and the selected transcription model"
"OpenAI speech recognition could not initialize; verify OPENAI_STT_API_KEY or OPENAI_API_KEY and the selected transcription model"
.into()
}
}
Expand Down
60 changes: 59 additions & 1 deletion src-tauri/crates/berd-call/src/openai.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,29 @@ use futures_util::StreamExt;
use reqwest::header::{HeaderMap, HeaderValue, AUTHORIZATION, CONTENT_TYPE};
use serde_json::json;

/// The variable and key for one OpenAI service: its own variable (for example
/// `OPENAI_TTS_API_KEY`), else `OPENAI_API_KEY`. Blank values count as unset.
fn credential(
kind_var: &'static str,
get: impl Fn(&str) -> Option<String>,
) -> Option<(&'static str, String)> {
[kind_var, "OPENAI_API_KEY"].into_iter().find_map(|name| {
get(name)
.filter(|key| !key.trim().is_empty())
.map(|key| (name, key))
})
}

/// The environment variable that supplies `kind_var`'s key, if any.
pub fn openai_key_var(kind_var: &'static str) -> Option<&'static str> {
credential(kind_var, |name| std::env::var(name).ok()).map(|(name, _)| name)
}

/// The OpenAI key for one service, falling back to `OPENAI_API_KEY`.
pub fn openai_key(kind_var: &'static str) -> Option<String> {
credential(kind_var, |name| std::env::var(name).ok()).map(|(_, key)| key)
}

#[derive(Clone, Debug)]
pub struct OpenAiSpeechConfig {
pub endpoint: String,
Expand Down Expand Up @@ -130,7 +153,42 @@ fn pcm16le_to_f32(bytes: &[u8]) -> Vec<f32> {

#[cfg(test)]
mod tests {
use super::pcm16le_to_f32;
use super::{credential, pcm16le_to_f32};

fn lookup<'a>(vars: &'a [(&'a str, &'a str)]) -> impl Fn(&str) -> Option<String> + 'a {
move |name| {
vars.iter()
.find(|(n, _)| *n == name)
.map(|(_, v)| v.to_string())
}
}

#[test]
fn service_key_wins_over_the_shared_key() {
let vars = [("OPENAI_API_KEY", "shared"), ("OPENAI_TTS_API_KEY", "tts")];
assert_eq!(
credential("OPENAI_TTS_API_KEY", lookup(&vars)),
Some(("OPENAI_TTS_API_KEY", "tts".into()))
);
assert_eq!(
credential("OPENAI_STT_API_KEY", lookup(&vars)),
Some(("OPENAI_API_KEY", "shared".into()))
);
}

#[test]
fn blank_or_unset_service_key_falls_back_to_the_shared_key() {
let vars = [
("OPENAI_API_KEY", "shared"),
("OPENAI_REALTIME_API_KEY", " "),
];
assert_eq!(
credential("OPENAI_REALTIME_API_KEY", lookup(&vars)),
Some(("OPENAI_API_KEY", "shared".into()))
);
let blank = [("OPENAI_API_KEY", " "), ("OPENAI_STT_API_KEY", "")];
assert_eq!(credential("OPENAI_STT_API_KEY", lookup(&blank)), None);
}

#[test]
fn decodes_little_endian_pcm_without_changing_frame_units() {
Expand Down
10 changes: 5 additions & 5 deletions src-tauri/crates/berd-call/src/openai_spokesperson.rs
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,7 @@ const GRACEFUL_SHUTDOWN_TIMEOUT: Duration = Duration::from_secs(5);

/// The OpenAI key the Spokesperson needs, when the environment has one.
pub fn api_key() -> Option<String> {
std::env::var("OPENAI_API_KEY")
.ok()
.filter(|value| !value.trim().is_empty())
crate::openai::openai_key("OPENAI_REALTIME_API_KEY")
}

/// Connection settings for the live Spokesperson. This deliberately does not
Expand Down Expand Up @@ -58,8 +56,10 @@ impl OpenAiSpokespersonConfig {
}

pub fn from_environment() -> Result<Self, String> {
let api_key = api_key()
.ok_or_else(|| "OPENAI_API_KEY is required for Expert-Spokesperson mode".to_string())?;
let api_key = api_key().ok_or_else(|| {
"OPENAI_REALTIME_API_KEY or OPENAI_API_KEY is required for Expert-Spokesperson mode"
.to_string()
})?;
let mut config = Self::new(
api_key,
RealtimeSpokespersonSessionOptions {
Expand Down
Loading