Local KB Agent Starter

This is a first implementation slice for a local, knowledge-base-first research assistant. It is intentionally a typed Python core, not a framework-first agent.

First milestone

The first useful version should do four things reliably:

1. Route a request into a typed plan.
2. Retrieve or ingest local knowledge with provenance.
3. Produce an answer package with a verification preamble.
4. Propose profile/note updates for human approval.

Local model roles

Recommended starting roles:

• planner_fast: qwen3:4b or qwen3:4b-thinking
• synthesizer_code: qwen2.5-coder:7b
• utility_tiny: optional small model only after benchmarks prove it helps

Install

python -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"

Optional extras:

pip install -e ".[ingest,api,observability,dev]"

For local dense vectors, install the optional embedded vector-store dependency:

pip install -e ".[dense,dev]"
ollama pull nomic-embed-text

Try the router

kb-agent plan "Explain RAG file format caveats, but assume I understand embeddings."

Build and inspect a local text corpus

Index a supported text file or directory. A collection label lets you keep different projects separate during retrieval:

kb-agent ingest ./notes --collection personal_notes
kb-agent ingest ./project-docs --collection project_docs

Inspect lexical retrieval without starting a model or writing a run record:

kb-agent search "timeout handling" --collection project_docs --limit 5

Restrict an answer to one or more collections by repeating --collection:

kb-agent ask "How does this configuration work?" \
  --collection project_docs --collection personal_notes

Without --collection, local retrieval searches every indexed collection.

Build the local dense index

Dense vectors are an additional local index derived from the same canonical chunks as SQLite FTS. Create them deliberately during ingestion:

kb-agent ingest ./project-docs --collection project_docs --embed \
  --embedding-model nomic-embed-text

This uses local Ollama embeddings and stores vectors under data/vector_index/. Once that index exists, normal ask automatically combines lexical FTS and dense candidates with reciprocal-rank fusion before context packing:

kb-agent ask "How is certificate validation configured?" \
  --collection project_docs --embedding-model nomic-embed-text

ask --dry-run stays model-free and therefore shows lexical retrieval only. No hosted model or API key is involved.

Optional local reranking

Hybrid retrieval aims to recall useful chunks; an optional cross-encoder can then compare the request with each fused candidate and order that small set more precisely before context packing. It is disabled unless selected, so the current default behavior and dependency set remain unchanged.

pip install -e ".[dense,rerank,dev]"
kb-agent ask "How is certificate validation configured?" \
  --collection project_docs \
  --reranker-model cross-encoder/ms-marco-MiniLM-L6-v2

The first use of a model identifier may download it into the Sentence Transformers cache. To guarantee --offline makes no network request, pass a previously cached local model directory; --offline enforces local-files-only loading for the reranker as well as direct local Ollama synthesis. ask --dry-run always skips reranking, so it remains model-free.

Optional: use Open WebUI as a model gateway

This project can call Open WebUI as a single gateway for models you configure there. The project still owns its profile, planning, retrieval, verification, and run-record logic; Open WebUI only forwards the final model request to the selected local or hosted model.

No API key is needed for direct local Ollama use. If you later choose to use Open WebUI with hosted models, keep the provider key in Open WebUI and give this project only an Open WebUI API key:

1. Run Open WebUI and sign in to the account that will manage its connections.

2. In Settings → Admin → Connections, add the provider connection you want to use. Enter the provider API key there; do not add it to this repository or this project's environment.

3. Enable Open WebUI API keys if your instance requires it, then create a dedicated non-admin API key for this project. Its permissions should be limited to the model connections you intend to use.

4. Store only that Open WebUI key in your local shell or uncommitted .env:

   export OPENWEBUI_BASE_URL=http://localhost:3000
   export OPENWEBUI_API_KEY=your-openwebui-key

5. Select the Open WebUI gateway and one of its configured model IDs:

   kb-agent ask "Explain RAG" \
     --synthesis-provider openwebui --model your-openwebui-model-id

Use direct Ollama when a request must remain offline:

kb-agent ask "Explain RAG" \
  --offline --synthesis-provider ollama --model qwen3:4b

An Open WebUI gateway can route to either local or hosted models, so it is not used for the hard offline mode. Do not commit .env files or any provider/Open WebUI key.

Design rule

Permanent memory is proposal-based. The assistant can draft note/profile updates, but human approval should be required before writing durable knowledge.

For now, a completed ask conservatively proposes profile updates only when you state knowledge directly (for example, “assume I understand embeddings” or “I’m new to reranking”). The proposal is saved with the run and printed as not applied; it does not modify data/profiles/.

Review then explicitly approve a saved proposal using its run and proposal IDs:

kb-agent proposal-show <run-id> <proposal-id>
kb-agent profile-proposal-apply <run-id> <proposal-id>

Applying is refused if the profile has changed since the run, so review a fresh proposal rather than applying an outdated view of your knowledge.