Skip to content

fix(models): report honest provider credential status - #6538

Merged
benjaminshafii merged 7 commits into
devfrom
fix/provider-key-status
Oct 27, 2026
Merged

benjaminshafii merged 7 commits into
devfrom
fix/provider-key-status

Conversation

@benjaminshafii

@benjaminshafii benjaminshafii commented Oct 27, 2026 •

Copy link
Copy Markdown
Member

What is this about?

AI Providers distinguishes saved, verified and rejected credentials instead of treating a saved key as proof it works.

What problem does it solve?

A provider can reject a chat with an authentication error while Settings still claims it is ready.

What was the situation before?

Saving an invalid key listed models and showed “Ready to use,” even after a chat returned 401.

How is this implemented?

  • Behind providerCredentialStatus (cloud and self-hosted, off). Turning it off restores the previous status UI without changing credentials or chats. Rollout docs.
  • Track workspace/provider outcomes from live and durable inference evidence, preserving native HTTP status without requiring a response body. Exclude policy, quota, transport and Gateway member-sign-in failures.
  • Show “Not checked yet” before success and “Key not accepted” after rejection, with Replace key for personal keys or contact-admin direction for organization providers.
  • Reset on replacement, organization configuration changes or observed success; ignore outcomes from requests predating replacement. Refresh cached engine availability after saving.

What else changed?

none

Release note

AI Providers can distinguish an untested or rejected key from a provider that has successfully answered a chat.

Evidence

evals/proofs/provider-credential-status.proof.test.ts — before: saved and rejected keys claimed readiness; after: saved keys are unchecked, a loopback-provider 401 shows Key not accepted, replacement clears it, and success earns Ready to use.

Design: P1, P5, C6, V2

@vercel

vercel Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
openwork-app Ready Ready Preview Oct 27, 2026 4:31am UTC
3 Skipped Deployments
Project Deployment Actions Updated
openwork-den Ignored Ignored Preview Oct 27, 2026 4:31am UTC
openwork-diagnostics Ignored Ignored Preview Oct 27, 2026 4:31am UTC
openwork-landing Ignored Ignored Preview Oct 27, 2026 4:31am UTC

Request Review

@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

OpenWork evidence

Status Commit Preview Updated
🟡 Updating evidence dee4d32 View run 2026-10-27 04:38:09 UTC

A preview for this commit is being prepared.

diff-warden[bot]
diff-warden Bot previously approved these changes Oct 27, 2026

@diff-warden diff-warden Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warden clearance. Complete security and confidentiality review of c0c3947e62d2db3c98cd80ef3fff7e27cad12309 found no high or medium security findings and no confidentiality findings. Any new push dismisses this approval. Analysis run

@diff-warden

diff-warden Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Warden: approved

No high or medium security findings and no confidentiality findings.

Reviewed dee4d32ab46c · Warden run · Updates after every Warden run.

diff-warden[bot]
diff-warden Bot previously approved these changes Oct 27, 2026

@diff-warden diff-warden Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warden clearance. Complete security and confidentiality review of c056af3a3db68133c5cca1056d39128353bd2512 found no high or medium security findings and no confidentiality findings. Any new push dismisses this approval. Analysis run

diff-warden[bot]
diff-warden Bot previously approved these changes Oct 27, 2026

@diff-warden diff-warden Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warden clearance. Complete security and confidentiality review of a866ddc4f12627c52903e54edbf92a101b49e63a found no high or medium security findings and no confidentiality findings. Any new push dismisses this approval. Analysis run

diff-warden[bot]
diff-warden Bot previously approved these changes Oct 27, 2026

@diff-warden diff-warden Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warden clearance. Complete security and confidentiality review of a10c8e3c3aecd2e6b039b068554ad62cdbe57003 found no high or medium security findings and no confidentiality findings. Any new push dismisses this approval. Analysis run

diff-warden[bot]
diff-warden Bot previously approved these changes Oct 27, 2026

@diff-warden diff-warden Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warden clearance. Complete security and confidentiality review of b7ef89e1e70061cddf3ce2dd9e5462ba1b6a4f38 found no high or medium security findings and no confidentiality findings. Any new push dismisses this approval. Analysis run

@diff-warden diff-warden Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warden clearance. Complete security and confidentiality review of dee4d32ab46ccd2695c59b2227daac12089d4e72 found no high or medium security findings and no confidentiality findings. Any new push dismisses this approval. Analysis run

@benjaminshafii
benjaminshafii marked this pull request as ready for review October 27, 2026 04:35
@benjaminshafii
benjaminshafii merged commit c73c8df into dev Oct 27, 2026
46 of 50 checks passed

This branch was successfully deployed

5 active (3 outdated) deployments
Preview – openwork-app — dee4d32a Deployed Oct 27, 2026 by vercel[bot]
Preview – openwork-diagnostics — c0c3947e Deployed Oct 27, 2026 by vercel[bot]
Preview – openwork-landing — c0c3947e Deployed Oct 27, 2026 by vercel[bot]
Preview – openwork-den — c0c3947e Deployed Oct 27, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant