Skip to content

feat(embedserver): add local face identity vectors - #311

Open
rodboev wants to merge 23 commits into
mainfrom
feat/face-vectors
Open

rodboev wants to merge 23 commits into
mainfrom
feat/face-vectors

Conversation

@rodboev

@rodboev rodboev commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

Apps using Kit's local embeddings server can now turn each face box in a photo into an AuraFace-v1 identity vector without sending the photo anywhere.

On Teststrip's astronaut corpus, the largest same-person distance is 1.182335. With David Scott's face from the Gemini-8 photo, the smallest different-person distance is 1.196469. A threshold of 1.189402 separates every measured pair, and the obscured EVA-training face returns null.

  • Client.EmbedFaces posts a JPEG or PNG preview and normalized boxes to the new POST /v1/faces route.
  • Each box gets a 512-number unit vector, or null when YuNet finds no face in it.
  • Responses carry model, revision, and recipe, so stored vectors stay tied to the model and alignment that made them.
  • Setting InstallOptions.Model to embedlocal.FaceModel downloads only AuraFace-v1 and YuNet (260,926,740 bytes), plus the server bundle when it is missing.
  • A running server picks up newly installed models without a restart, and GET /v1/models lists them.
  • Direct kennembed launches require installation first and no longer download the text model at startup.
  • internal/embedassets.BundleTag pins the published v0.48.2-face.6 server bundle, which serves faces.

EmbedFaces sends up to 8 boxes per request. The server decodes one preview at a time and takes the shared inference gate separately for each face, so text requests can run between faces. It releases the gate before writing the response, so a slow reader cannot block text requests.

Refs kenn-io/docbank#612, slice 6b1 (face vectors).

Face matching needs stable vectors across photos. Sharing the existing server lets apps add it without another native distribution, while separate model consent keeps text installations usable offline.
Face setup should keep existing text installations usable and give transient failures the retry policy the caller selected. Model discovery must stay cheap enough for ordinary searches, while loading still verifies the downloaded bytes.

Include the detector pin in vector identity so a later detector update cannot mix incompatible face vectors.
A server can accept requests before either model is loaded. Missing model assets need an installation step, so repeating the same request cannot fix the failure.

Concurrent photos must share preview decoding as well as native inference to keep decoded images from accumulating in the request queue. A photo now holds the gate through its boxes, so text requests wait until that photo finishes.

Keep the BGR input recipe because it separates the measured corpus more widely than the upstream RGB recipe.
Both native test lanes need the same verified models and photos before the installed-only server starts. Wikimedia downloads also need an identifying user agent.

Rounded face boxes can end a fraction outside the preview. Accept that rounding while keeping image decoding and pixel limits on the server, and share the existing vector decoder.

Report the original astronaut manifest separately from the extra face annotation so its calibrated distance remains clear.
The release workflow rejects an action until its own lockfile entry includes that dependency. Adding fixture caching therefore needs coverage for the bundle workflow as well as the existing parity workflow.
The bundle run hit a rate limit during staging, while its checksum failure omitted the affected file. Reuse the workflow's bounded download retries and name each failed asset so the next failure can be traced to its source.

Staging retains verified files between attempts, so a retry resumes from the remaining downloads.
Ordinary test runs should catch changes to advertised model availability. Keep that coverage independent of downloaded models, while native acceptance focuses on face inference.

Pin the text model's URLs and cache layout to literal expectations so an incorrect path cannot agree with itself. Shared test cases retain the distinct discovery failures without repeating their setup.
The generated thumbnail repeatedly failed its checksum on the Linux runner. The original photograph matches the reference corpus's recorded size and digest, so use that source file for the pinned fixture.
A malformed face response should fail for the expected reason. Otherwise an earlier failure can hide a broken vector norm check.

Keep shared behavior covered once, and let model pins change without unrelated test failures.
Installing face models with the previous bundle left callers unable to request face vectors. All six archives and their checksums are now published, so fresh installations can use the server's face route and capability discovery.
Processing 128 faces in one HTTP request can exceed the default client timeout. Smaller batches let the full call finish under the caller's context and give text requests a chance to run between batches.

A slow response reader must also leave inference available to other callers. Keep preview decoding within the shared memory bound, and release the inference gate before response writes can block.
Model discovery alone cannot show that a packaged server can run face inference. Exercise both people in one photograph through the staged executable before accepting its bundle.

Keep decode rejection checks beside the blocked-response scenario that also verifies inference stays available. Explicit expected messages keep renamed test cases from silently losing their assertions.
A slow face response reader could keep text inference waiting on the shared gate. The published face bundle now releases that gate before writing responses, so fresh installations need its revised server.
Discovery deliberately checks file sizes before loading verifies digests. Name both responsibilities and explain the sparse fixtures so these tests do not suggest that discovery proves model integrity.

The client already reconstructs indexed face results and tests their ordering. Packaged acceptance can focus on returning actual face vectors.
@roborev-ci

roborev-ci Bot commented Nov 5, 2026

Copy link
Copy Markdown

roborev: Combined Review (fb9ff58)

Verdict: No findings at or above medium severity.


Reviewers: 3x codex, codex (security), codex (test-value) | Synthesis: codex, 10s | Total: 9m32s

Keep face installation and text bundle reuse while adopting the shared asset check's explicit content-verification option. Installation and loading still verify digests; discovery uses file metadata.

Bring in the current installation-status API and bounded Ollama probing without rewriting the face branch's commits.
The existing fallback cases already exercise old text servers, and the batch scenario covers ordered face vectors and empty results. Keep those checks together so request assertions also cover the batched path.

Model discovery can now run through one sequence of installed files, separate from the HTTP installation-error cases. This keeps each failure tied to the behavior it checks.
@roborev-ci

roborev-ci Bot commented Nov 5, 2026

Copy link
Copy Markdown

roborev: Combined Review (0ac807c)

Verdict: Changes require fixes for 1 finding.

Medium

  • embedlocal/status.go:68: Inspect ignores Options.Model and always checks text assets, incorrectly reporting face installation readiness and download requirements for text-only or face-only installations.

    Fix: Resolve Options.Model through SelectedModel and inspect the selected directory and assets. Add coverage for text-only and face-only installations.


Reviewers: 3x codex, codex (security), codex (test-value) | Synthesis: codex, 10s | Total: 10m29s

A caller checking face readiness could see zero download bytes on a text-only installation, or a text-model download estimate when all face files were present. Use the same model selection as installation and endpoint discovery so the estimate describes the requested feature.

Ollama provides text embeddings, so its availability must not make a face installation look ready or delay that inspection with a text-model probe.
Existing text users saw a 20 MB download estimate even though installation reused their complete cached server without downloading. Inspection and installation need the same bundle selection rule so the estimate remains accurate after an upgrade.

Text setup still needs discovery to report missing text models on a face-only server. Returning that server's address immediately would make callers skip installation and leave text inference unavailable.
The setup example recreated an app's decision to install after discovery. Keep its missing-model and ready-model checks at the endpoint boundary, with asset writes covered by the installer tests.

Share model-test setup and leave cached-bundle repair policy with the installer coverage. Preserve inspection cases that can catch a wrong model argument and face-specific cancellation coverage.
@roborev-ci

roborev-ci Bot commented Nov 5, 2026

Copy link
Copy Markdown

roborev: Combined Review (c4a6543)

Verdict: Changes require fixes for 1 finding.

Medium

  • embedserver/face_handler.go:77: A multi-box face request holds the shared inference gate through up to 128 face inferences, delaying concurrent text requests and potentially exceeding their deadlines. This violates the package requirement to take the gate per input.

    Fix: Bound decoded previews with separate admission control and acquire/release the inference gate per box. Add a concurrency test showing a waiting text request runs between face items.


Reviewers: 3x codex, codex (security), codex (test-value) | Synthesis: codex, 15s | Total: 11m45s

A face request held inference for every box in its photo, so a waiting text query could miss its deadline. Face items need the same opportunity to yield to another request as text batch items.

Keep one decoded preview admitted at a time while taking the inference gate for each box. Shutdown must drain that preview before closing models, since face processing can now be between native calls when cancellation arrives.
Checking admission after a response finished could hide a handler that held the decoded preview while a reader stalled. Check both gates at the blocked write so the test observes the promised release point.

Use valid boxes to isolate the box-count limit, separate discovery from installation errors, and keep retry and packaged acceptance focused on their own boundaries.
@roborev-ci

roborev-ci Bot commented Nov 5, 2026

Copy link
Copy Markdown

roborev: Combined Review (95728cc)

Verdict: No findings at or above medium severity.


Reviewers: 3x codex, codex (security), codex (test-value) | Synthesis: codex, 11s | Total: 8m42s

The previous published server held inference for an entire photo's boxes, delaying concurrent text queries. Install the revised bundle so text can run between face items while decoded previews remain bounded.
A truncated image reaches a different rejection path from an invalid header. Keep its status and message checks with HTTP validation, while blocked-response coverage checks preview admission and a concurrent text request.

Batch ordering should survive a tuning change to the request size. Derive its accounting from the configured batch size, and let packaged acceptance check for an actual face result after client validation.
@roborev-ci

roborev-ci Bot commented Nov 5, 2026

Copy link
Copy Markdown

roborev: Combined Review (0f7c208)

Verdict: No findings at or above medium severity.


Reviewers: 3x codex, codex (security), codex (test-value) | Synthesis: codex, 11s | Total: 7m33s

@rodboev
rodboev marked this pull request as ready for review November 5, 2026 20:33

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant