fix: unstick the anti-pattern "Why?" explanation and use Copilot Auto

CHANGELOG.md

@@ -2,6 +2,12 @@
 
 ## Unreleased
 
+- Fix the anti-pattern **Why?** explanation getting stuck on "Asking AI for an explanation…" (#278):
+  every explanation state now renders through Preact, empty or failed responses show an error,
+  and timeouts, denied access, and quota errors fail fast instead of retrying for minutes
+- Use GitHub Copilot's Auto model for all AI features instead of hard-coded model families. In VS Code,
+  the Why? buttons, Skill Finder, and Level Up are hidden and the context review is disabled while
+  GitHub Copilot isn't available; they come back on sign-in without reopening the dashboard
 - Fix canvas mode intermittently loading with an empty content pane or a stuck
   loading screen (#269): the canvas bridge now subscribes to host events only
   after the dashboard signals it is listening, so the replayed `dataReady` is

README.md

@@ -205,7 +205,7 @@ image requests.
 - **Read-only** — the extension never modifies your session files
 - **Local analysis** — all parsing and analytics run entirely on your machine
 - **No proprietary telemetry** — the extension does not phone home or collect usage data
-- **Optional AI features** — some features (rule compiler, skill finder, context review) use the VS Code built-in Copilot language model API when explicitly invoked by the user
+- **Optional AI features** — some features (rule compiler, skill finder, Level Up learning content, context review, anti-pattern **Why?** explanations) use GitHub Copilot's Auto model through the VS Code language model API when explicitly invoked by the user. In VS Code, the Why? buttons, Skill Finder, and Level Up are hidden and the context review is disabled while GitHub Copilot isn't available (for example, when you're signed out); they come back as soon as you sign in. Rule generation falls back to a starter template
 
 ---
 

src/core/copilot-model.ts

@@ -0,0 +1,27 @@
+/*---------------------------------------------------------------------------------------------
+ *  Copyright (c) Microsoft Corporation. All rights reserved.
+ *  Licensed under the MIT License. See LICENSE in the project root for license information.
+ *--------------------------------------------------------------------------------------------*/
+
+/**
+ * The single language model source for every AI feature: GitHub Copilot's Auto
+ * router. Copilot routes each request to a model that is enabled for the user's
+ * plan and organization, so the extension never hard-codes model families that
+ * go stale or are selectable-but-disabled (#91, #278).
+ *
+ * Select by `id`, not `family`: Auto reports the family of the model it routes
+ * to, so `{ family: 'auto' }` never matches.
+ */
+export const COPILOT_AUTO_MODEL_SELECTOR = { vendor: 'copilot', id: 'auto' } as const;
+
+/** Shown by VS Code in the one-time consent dialog for language model access. */
+export const COPILOT_LM_JUSTIFICATION =
+  'AI Engineer Coach uses GitHub Copilot to explain anti-pattern detections, review context files, and generate skills, rules, and quizzes when you ask.';
+
+/**
+ * Upper bound for one Copilot-backed webview request on the extension host,
+ * including model selection and retries. The webview's timeout for these
+ * requests (`LLM_METHODS` in `src/webview/shared.ts`) must outlast it with room
+ * to deliver the reply, so the user always sees the answer or the error.
+ */
+export const COPILOT_REQUEST_BUDGET_MS = 120_000;

src/core/rule-compiler.ts

@@ -11,6 +11,7 @@
  * to a simple template-based heuristic.
  */
 
+import { COPILOT_AUTO_MODEL_SELECTOR, COPILOT_LM_JUSTIFICATION } from './copilot-model';
 import { FIELD_SCHEMA, FUNCTION_CATALOG } from './dsl/index';
 import { parseRule } from './rule-parser';
 import type { DetectionRule } from './types/rule-types';
@@ -82,7 +83,7 @@ async function compileLlm(
   const lm = vscode.lm;
   if (!lm) return null;
 
-  const models = await lm.selectChatModels({ family: 'gpt-4.1' });
+  const models = await lm.selectChatModels(COPILOT_AUTO_MODEL_SELECTOR);
   const model = models[0];
   if (!model) return null;
 
@@ -94,7 +95,7 @@ async function compileLlm(
     vscode.LanguageModelChatMessage.User(userPrompt),
   ];
 
-  const response = await model.sendRequest(messages, {});
+  const response = await model.sendRequest(messages, { justification: COPILOT_LM_JUSTIFICATION });
   let result = '';
   for await (const chunk of response.text) {
     result += chunk;

src/core/types/rpc-types.ts

@@ -142,9 +142,18 @@ export interface ExtensionMethodMap extends RpcMethodMap {
   saveModelBudgets: { params: { budgets: Record<string, number> }; result: { ok: boolean } };
   loadModelBudgets: { params: Record<string, unknown> | undefined; result: Record<string, number> };
   showOutput: { params: undefined; result: { ok: boolean } };
-  getCapabilities: { params: Record<string, unknown> | undefined; result: { host: 'vscode' | 'canvas'; llm: boolean } };
+  getCapabilities: { params: Record<string, unknown> | undefined; result: HostCapabilities };
 }
 
+/** What the dashboard host can do. `llm` is true when GitHub Copilot's Auto model is usable. */
+export interface HostCapabilities {
+  host: 'vscode' | 'canvas';
+  llm: boolean;
+}
+
+/** Pushed by the VS Code host when language model availability changes after load. */
+export type HostCapabilitiesMessage = { type: 'capabilities' } & HostCapabilities;
+
 export type ExtensionMethodName = keyof ExtensionMethodMap;
 export type WebviewRequestMessage<M extends ExtensionMethodName = ExtensionMethodName> = {
   type: 'request';

src/webview/capabilities.test.ts

@@ -0,0 +1,97 @@
+/*---------------------------------------------------------------------------------------------
+ *  Copyright (c) Microsoft Corporation. All rights reserved.
+ *  Licensed under the MIT License. See LICENSE in the project root for license information.
+ *--------------------------------------------------------------------------------------------*/
+
+/**
+ * @vitest-environment jsdom
+ */
+
+import { beforeEach, describe, expect, it, vi } from 'vitest';
+
+const mocks = vi.hoisted(() => ({ rpc: vi.fn() }));
+vi.mock('./shared', () => ({ rpc: mocks.rpc }));
+
+type CapabilitiesModule = typeof import('./capabilities');
+
+/** Fresh module per test: capabilities are module-level state. */
+async function loadModule(): Promise<CapabilitiesModule> {
+  vi.resetModules();
+  return import('./capabilities');
+}
+
+function pushFromHost(data: unknown): void {
+  window.dispatchEvent(new MessageEvent('message', { data }));
+}
+
+const aiControlsHidden = () => document.documentElement.classList.contains('llm-unavailable');
+
+beforeEach(() => {
+  mocks.rpc.mockReset();
+  document.documentElement.classList.remove('llm-unavailable');
+});
+
+describe('webview capabilities', () => {
+  it('hides AI controls when the VS Code host reports no Copilot model', async () => {
+    const caps = await loadModule();
+    mocks.rpc.mockResolvedValue({ host: 'vscode', llm: false });
+
+    await caps.loadCapabilities();
+
+    expect(mocks.rpc).toHaveBeenCalledWith('getCapabilities');
+    expect(caps.llmAvailable()).toBe(false);
+    expect(aiControlsHidden()).toBe(true);
+    expect(caps.llmUnavailableNote()).toBe('Sign in to GitHub Copilot in VS Code to use this.');
+  });
+
+  it('keeps AI controls visible when Copilot is available', async () => {
+    const caps = await loadModule();
+    mocks.rpc.mockResolvedValue({ host: 'vscode', llm: true });
+
+    await caps.loadCapabilities();
+
+    expect(caps.llmAvailable()).toBe(true);
+    expect(aiControlsHidden()).toBe(false);
+  });
+
+  it('keeps full capabilities when the probe RPC fails', async () => {
+    const caps = await loadModule();
+    mocks.rpc.mockRejectedValue(new Error('RPC timeout: getCapabilities'));
+
+    await caps.loadCapabilities();
+
+    expect(caps.capabilities()).toEqual({ host: 'vscode', llm: true });
+    expect(aiControlsHidden()).toBe(false);
+  });
+
+  it('points canvas users to VS Code', async () => {
+    const caps = await loadModule();
+    mocks.rpc.mockResolvedValue({ host: 'canvas', llm: false });
+
+    await caps.loadCapabilities();
+
+    expect(caps.capabilities()).toEqual({ host: 'canvas', llm: false });
+    expect(aiControlsHidden()).toBe(true);
+    expect(caps.llmUnavailableNote()).toBe('Open in VS Code with the local Copilot agent to use this.');
+  });
+});
+
+describe('capabilities pushed by the host', () => {
+  it('shows and hides AI controls in place and ignores unrelated messages', async () => {
+    const caps = await loadModule();
+    mocks.rpc.mockResolvedValue({ host: 'vscode', llm: true });
+    await caps.loadCapabilities();
+
+    pushFromHost({ type: 'capabilities', host: 'vscode', llm: false });
+    expect(caps.llmAvailable()).toBe(false);
+    expect(aiControlsHidden()).toBe(true);
+
+    pushFromHost({ type: 'response', id: '1', data: { llm: true } });
+    expect(caps.llmAvailable()).toBe(false);
+    expect(aiControlsHidden()).toBe(true);
+
+    pushFromHost({ type: 'capabilities', host: 'vscode', llm: true });
+    expect(caps.llmAvailable()).toBe(true);
+    expect(aiControlsHidden()).toBe(false);
+  });
+});

src/webview/capabilities.ts

@@ -7,23 +7,45 @@
  *
  * The dashboard runs in two hosts: the VS Code extension (full local agent, can
  * call the language model) and the Copilot app canvas (read-only data view, no
- * model access). Features that need the local agent are gated on `llm`. */
+ * model access). Features that need the language model are gated on `llm`. In
+ * VS Code, `llm` tracks whether GitHub Copilot's Auto model is usable and the
+ * host pushes updates when that changes (e.g. after signing in to Copilot).
+ *
+ * Pages always render their AI controls marked `llm-only` (or `llm-fallback`
+ * for the unavailable variant); the `llm-unavailable` class on the root element
+ * shows or hides them through CSS. A capability change therefore never rebuilds
+ * the page, so open editors, drafts, and loaded results survive it. */
 
+import type { HostCapabilities } from '../core/types/rpc-types';
 import { rpc } from './shared';
 
-export interface HostCapabilities {
-  host: 'vscode' | 'canvas';
-  llm: boolean;
-}
+export type { HostCapabilities };
 
 let caps: HostCapabilities = { host: 'vscode', llm: true };
+let pushListenerInstalled = false;
+
+function applyCapabilities(value: Partial<HostCapabilities>): void {
+  caps = { host: value.host === 'canvas' ? 'canvas' : 'vscode', llm: value.llm !== false };
+  document.documentElement.classList.toggle('llm-unavailable', !caps.llm);
+}
+
+function installPushListener(): void {
+  if (pushListenerInstalled) return;
+  pushListenerInstalled = true;
+  window.addEventListener('message', (ev: MessageEvent) => {
+    const data: unknown = ev.data;
+    if (typeof data !== 'object' || data === null) return;
+    const msg = data as Record<string, unknown>;
+    if (msg.type !== 'capabilities') return;
+    applyCapabilities({ host: msg.host === 'canvas' ? 'canvas' : 'vscode', llm: msg.llm !== false });
+  });
+}
 
 export async function loadCapabilities(): Promise<void> {
+  installPushListener();
   try {
     const result = await rpc<Partial<HostCapabilities>>('getCapabilities');
-    if (result && typeof result === 'object') {
-      caps = { host: result.host === 'canvas' ? 'canvas' : 'vscode', llm: result.llm !== false };
-    }
+    if (result && typeof result === 'object') applyCapabilities(result);
   } catch {
     /* keep the default full-capability profile */
   }
@@ -37,5 +59,9 @@ export function llmAvailable(): boolean {
   return caps.llm;
 }
 
-/** Short note shown next to greyed-out, agent-dependent features. */
-export const LLM_UNAVAILABLE_NOTE = 'Open in VS Code with the local Copilot agent to use this.';
+/** Short note shown next to greyed-out, language-model-dependent features. */
+export function llmUnavailableNote(): string {
+  return caps.host === 'canvas'
+    ? 'Open in VS Code with the local Copilot agent to use this.'
+    : 'Sign in to GitHub Copilot in VS Code to use this.';
+}

src/webview/dashboard-shell.ts

@@ -29,9 +29,9 @@ export function getDashboardShellHtml(opts?: { includeSkillFinder?: boolean; inc
       <li class="github-app-nav-item" hidden><a href="#" data-page="github-app-issue-credits"><span class="nav-icon"><svg width="16" height="16" viewBox="0 0 16 16" fill="none"><circle cx="8" cy="8" r="6" stroke="currentColor" stroke-width="1.3"/><path d="M9.8 5.4C9.4 4.9 8.8 4.6 8 4.6c-1.1 0-2 .6-2 1.5 0 2.3 4.2 1.1 4.2 3.5 0 1-.9 1.8-2.2 1.8-.9 0-1.7-.4-2.1-1.1M8 3.4v1.2M8 11.4v1.2" stroke="currentColor" stroke-width="1.2" stroke-linecap="round"/></svg></span> Issue credits<span class="nav-badge" id="badge-github-app-issue-credits" aria-live="polite" aria-atomic="true" aria-label="GitHub issues with credit estimates"></span></a></li>
       <li class="nav-group-header">Improve</li>
       <li><a href="#" data-page="anti-patterns"><span class="nav-icon">&#9888;</span> Anti-Patterns<span class="nav-badge" id="badge-antipatterns" aria-live="polite" aria-atomic="true" aria-label="Anti-patterns"></span></a></li>
-      ${includeSkillFinder ? '<li><a href="#" data-page="skills"><span class="nav-icon"><svg width="16" height="16" viewBox="0 0 16 16" fill="none"><circle cx="4" cy="4" r="2.5" stroke="currentColor" stroke-width="1.3" fill="none"/><circle cx="12" cy="4" r="2.5" stroke="currentColor" stroke-width="1.3" fill="none"/><circle cx="8" cy="13" r="2.5" stroke="currentColor" stroke-width="1.3" fill="none"/><line x1="4" y1="6.5" x2="8" y2="10.5" stroke="currentColor" stroke-width="1.2"/><line x1="12" y1="6.5" x2="8" y2="10.5" stroke="currentColor" stroke-width="1.2"/></svg></span> Skill Finder<span class="nav-badge" id="badge-skills" aria-live="polite" aria-atomic="true" aria-label="Skill Finder"></span></a></li>' : ''}
+      ${includeSkillFinder ? '<li class="llm-only"><a href="#" data-page="skills"><span class="nav-icon"><svg width="16" height="16" viewBox="0 0 16 16" fill="none"><circle cx="4" cy="4" r="2.5" stroke="currentColor" stroke-width="1.3" fill="none"/><circle cx="12" cy="4" r="2.5" stroke="currentColor" stroke-width="1.3" fill="none"/><circle cx="8" cy="13" r="2.5" stroke="currentColor" stroke-width="1.3" fill="none"/><line x1="4" y1="6.5" x2="8" y2="10.5" stroke="currentColor" stroke-width="1.2"/><line x1="12" y1="6.5" x2="8" y2="10.5" stroke="currentColor" stroke-width="1.2"/></svg></span> Skill Finder<span class="nav-badge" id="badge-skills" aria-live="polite" aria-atomic="true" aria-label="Skill Finder"></span></a></li>' : ''}
       <li><a href="#" data-page="config-health"><span class="nav-icon"><svg width="16" height="16" viewBox="0 0 16 16" fill="none"><path d="M8 1L1.5 4.5V11.5L8 15L14.5 11.5V4.5Z" stroke="currentColor" stroke-width="1.3" fill="none"/><path d="M8 5.5V10.5M5.5 8H10.5" stroke="currentColor" stroke-width="1.5" stroke-linecap="round"/></svg></span> Context Health</a></li>
-      ${includeLevelUp ? '<li><a href="#" data-page="level-up"><span class="nav-icon"><svg width="16" height="16" viewBox="0 0 16 16" fill="none"><path d="M8 1l2 5h5l-4 3.5 1.5 5L8 11.5 3.5 14.5 5 9.5 1 6h5z" stroke="currentColor" stroke-width="1.2" fill="none"/></svg></span> Level Up</a></li>' : ''}
+      ${includeLevelUp ? '<li class="llm-only"><a href="#" data-page="level-up"><span class="nav-icon"><svg width="16" height="16" viewBox="0 0 16 16" fill="none"><path d="M8 1l2 5h5l-4 3.5 1.5 5L8 11.5 3.5 14.5 5 9.5 1 6h5z" stroke="currentColor" stroke-width="1.2" fill="none"/></svg></span> Level Up</a></li>' : ''}
     </ul>
     <div class="sidebar-filters">
       <div class="sidebar-filter">