Repository navigation
docs: README for the harness, a guide for adding a suite, GitHub-only scenarios - #37
Merged
Merged
Conversation
artogahr
added this pull request to stack #38
October 16, 2026 12:41
RobertCrupa
requested changes
Oct 16, 2026
artogahr
added a commit
that referenced
this pull request
Oct 19, 2026
…cess, where the report comes from Review feedback on #37: Langfuse section explains org access, one project per thing under test and keys on every run instead of listing projects; the Store default is gone; reports are per suite and the guide says who writes them, where, and when.
…Hub-only scenarios The README described the legacy markdown runner. It now explains what the harness is, with store-actors as the first suite. docs/adding-a-suite.md walks a team through profile, scenarios, Langfuse project, task and results. The runner README no longer tells people to edit scenarios in the Langfuse UI; it points at the repo, the evals-scenario skill, Langfuse access and the alias models. Claude-Session: https://claude.ai/code/session_01Gb2b7DMXcfsDdz6ALUWA3e
…ed to use-apify-evals Claude-Session: https://claude.ai/code/session_01Gb2b7DMXcfsDdz6ALUWA3e
Co-authored-by: Robert-George Crupa <robertcrupa@outlook.com>
…cess, where the report comes from Review feedback on #37: Langfuse section explains org access, one project per thing under test and keys on every run instead of listing projects; the Store default is gone; reports are per suite and the guide says who writes them, where, and when.
artogahr
force-pushed
the
docs/suite-guide-and-skill
branch
from
October 19, 2026 11:50
1a19172 to
e252da2
Compare
RobertCrupa
approved these changes
Oct 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #31, because the docs describe the alias models that PR adds.
README.md: describes the harness as it is now (the old one described the legacy markdown runner) with a "you want to / read" table.docs/adding-a-suite.md: how a team adds a suite: profile, scenarios, Langfuse project (Generic Evals for one-off runs, own project for anything kept, Store project for the Store suite), first run, models, results, what needs the AI team.docs/.The scenario-writing skill lives in
apify/agent-skills-internalasuse-apify-evals(apify/agent-skills-internal#138); the docs point at it.https://claude.ai/code/session_01Gb2b7DMXcfsDdz6ALUWA3e