Skip to content

Copilot capture attribution is inconsistent #12

@codeprakhar25

Description

@codeprakhar25

agentdiff version

Observed on current development/release builds; exact version should be confirmed during reproduction.

Which agent were you using?

GitHub Copilot / VS Code extension path.

OS

Likely cross-platform; needs reproduction on Windows/WSL and macOS/Linux.

What happened?

Copilot attribution capture appears shaky and is not reported correctly every time. In practice this means Copilot-authored edits may be missed, attributed inconsistently, or not represented accurately in trace output.

This is especially risky because Copilot edits often happen inline inside a normal editor buffer, not through a clean command/event stream like some CLI agents.

Steps to reproduce

Need to turn this into a deterministic repro, but likely flow:

  1. Install/configure AgentDiff with Copilot support.
  2. In VS Code, accept Copilot inline completions and/or Copilot Chat edits.
  3. Commit the changed file.
  4. Run:
    agentdiff list
    agentdiff blame <changed-file>
    agentdiff report --format json --context
  5. Compare expected Copilot-authored ranges with recorded traces.

Things to investigate

  • Whether inline completions and Copilot Chat edits produce different VS Code events.
  • Whether accepted completions can be distinguished from human typing.
  • Whether rapid edits/debounce timing causes dropped ranges.
  • Whether buffer save events are too coarse for reliable line attribution.
  • Whether Copilot should be captured as context/statistics only unless confidence is high.

Expected behavior

Copilot edits should either be captured accurately with confidence/trust metadata, or AgentDiff should explicitly mark the capture as low-confidence/unsupported instead of implying precise attribution.

Acceptance criteria

  • A deterministic Copilot capture repro exists.
  • Known unsupported Copilot edit modes are documented.
  • Captured Copilot ranges are consistently correct in the supported mode.
  • Low-confidence capture is surfaced clearly in agentdiff list/report/blame.
  • Tests or extension fixtures cover the reliable path.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions