Skip to content

Commit 4c1936d

Browse files
committed
Document experimental status and benchmark recommendation
1 parent 468567a commit 4c1936d

1 file changed

Lines changed: 20 additions & 3 deletions

File tree

‎README.md‎

Lines changed: 20 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,18 @@
11
# Switchloom
22

3+
> **Experimental. I currently recommend against adopting this routing workflow.**
4+
>
5+
> I built Switchloom to see whether mixing models could get me the same quality
6+
> for less money. My benchmarks did not deliver that result. In the second
7+
> Pokédex run, the team with Jev cost almost as much as Astra alone and hit the
8+
> time limit before final acceptance. Astra finished and passed all 14 independent
9+
> functional checks.
10+
>
11+
> I am keeping the code and results public, but I would not add this layer to my
12+
> everyday workflow based on these results. This is evidence from two pilots on
13+
> one application, not proof that all model routing fails.
14+
> [Read the benchmark and its limitations](https://switchloom.ai/benchmarks/pokedex-round-2).
15+
316
**One workflow prompt for persistent Codex tasks, with optional TypeSafe/Jev routing.**
417

518
Assign capabilities to models, then copy the prompt into Codex Desktop with
@@ -28,9 +41,13 @@ Jev-versus-vanilla measurements remain open in [TASKS.md](TASKS.md).
2841

2942
## Setup
3043

31-
Both modes use the same handoff. Vanilla sets an explicit capability and does
32-
not call TypeSafe, so it needs no API key. Jev mode, enabled by default, needs
33-
the CLI, the skill and a local key:
44+
The copied desktop workflow currently requires the Switchloom CLI and skill in
45+
both modes. The CLI prepares routing decisions and handoff arguments. Codex
46+
creates the tasks, sends the messages and runs the work. The public Playground
47+
does not require a local CLI installation.
48+
49+
Vanilla sets an explicit capability and does not call TypeSafe, so it needs no
50+
API key. Jev mode, enabled by default, additionally needs a local TypeSafe key:
3451

3552
```sh
3653
cargo install --path . --locked

0 commit comments

Comments
 (0)