Repository navigation
ollama: five tools over the local daemon, a benchmark that measures, and three skills - #7
Merged
Merged
Conversation
…and three skills The pack could call a model; it could not say what that model did here. This row reads the fields every Ollama response already carries, and refuses to print a number without its conditions. - ollama_status: models on disk, what is loaded, and 'ollama serve' when the daemon is down. It starts nothing. - ollama_run: ONE bounded inference (num_predict is always sent), streamed so the first token is timed, with the daemon's token counts and nanosecond durations. - ollama_bench: 3+ runs only; temperature 0 and seed 42 by default; an unload before run 1 so 'cold' means cold; median, p95, min/max and spread; the cold/warm claim stated at the FIRST TOKEN because eval_duration excludes the load. A/B and num_ctx sweeps. - ollama_models: list/show, and pull/delete behind a confirmation that repeats the tag. No implicit download. - ollama_embed: /api/embed with a /api/embeddings fallback, dimension and cosine similarity; a completion model refused by capability. - three skills (REST API, metric arithmetic, Python client) whose curl/Python/JS examples the package check parses and compiles. - the loopback fence: an OLLAMA_HOST on another machine is refused before any request; DSH_OLLAMA_ALLOW_REMOTE=1 allows one explicitly. The package check serves a fake daemon with fixed nanosecond durations, so 100 tokens in 2 s is exactly 50.0 tok/s. The live half was run against this machine's own Ollama 0.40.3: one generation, a 3-run bench, and a cold bench that measured a 2.19 s load and a 22 ms warm first token. Verified: packages/dsh-ollama 'npm run check' (345 pass with the daemon stopped, live skipped loudly; 352 with it running), check-node-routes, check-no-secrets, check-scripts, check-skill-examples, check-media-examples, check-client-bundles.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
dsh-ollama: a host-only row of fiveollama_*tools over the machine's own daemon, plus three bundled skills. The pack could already call a model; this row reports what the model actually did here.ollama_status— daemon version, models on disk (family/params/quant/bytes/digest), what is loaded now (size_vramas offload %, context, expiry), the CLI version. A daemon that is down is an answer namingollama serve; it starts nothing.ollama_run— ONE bounded inference (num_predictis always sent, default 512, ceiling 8192), streamed so the first token is timed; token counts, tok/s fromeval_count / eval_duration, load, daemon total, wall clock,done_reason. Alengthfinish says the answer was cut.ollama_bench— 3+ runs only (2 is refused by name), temperature 0 and seed 42 by default,keep_alive: 0before run 1 so "cold" means cold, median/p95/min/max/spread, A/B andnum_ctxsweeps. The cold/warm claim is stated at the first token, becauseeval_durationexcludes the load and a cold load never lowers tok/s.ollama_models— list/show (parameters, template, license, capabilities; hidden size vs embedding dimension by capability), andpull/deletebehind a confirmation that repeats the tag.ollama_embed—/api/embedwith a/api/embeddingsfallback, dimension, norms, cosine similarity; a completion model refused by capability.Two rules the code is engineered against: a refusal is an answer (missing model lists the tags + the exact pull; a typo'd option is refused rather than silently ignored by the daemon), and loopback only by default — an
OLLAMA_HOSTon another machine is refused before any request, withDSH_OLLAMA_ALLOW_REMOTE=1as the explicit opt-in.Skills
ollama-api(REST surface, streaming NDJSON, options, keep_alive, errors,/v1differences, env vars),ollama-inference-metrics(what each duration covers, the arithmetic, what changes the numbers, the reporting rules),ollama-python(the client, streaming, tools, structured output, async, and a project shape whose arithmetic is unit-tested against a fake transport).Verification
packages/dsh-ollama:npm run check— 345 pass with the daemon stopped (live half skipped loudly), 352 pass with it running. A fake daemon on127.0.0.1:0with fixed nanosecond durations makes tok/s and TTFT assertions (100 tokens in 2 s = exactly 50.0 tok/s).llama3.2:3b: one generation, a 3-run bench, and a cold bench that measured a 2.19 s load and a 22 ms warm first token.check-node-routes(row, manifest, skill registration, fence),check-no-secrets,check-scripts,check-skill-examples,check-media-examples,check-client-bundles.ast, JS blocks bynode --check, and every endpoint the row calls must appear in the docs.docs/PLAN-ollama.mdis retired — its scope is shipped, and its design rationale now lives inpackages/dsh-ollama/README.md.Not in this PR
No UI: alpha.1 is host-only, like
dsh-wslanddsh-ocr. A results tab is the obvious alpha.2.