Skip to content

Feat: Multi-Release Historical Analysis and Regression Detection #33

@knabben

Description

@knabben

Description

The handbook requires flagging regressions as close to the source as possible (when offending code was merged). Currently, signalhound does not track test history or detect regressions.

Requirements

  • Track test history across multiple releases
  • Detect regressions (failures after code merge)
  • Identify PRs that introduced failures
  • Link failures to specific commits/PRs
  • Generate regression reports

Acceptance Criteria

  • Test history is tracked across releases
  • Regressions are detected (failures after merge)
  • PRs that introduced failures are identified
  • Failures are linked to commits/PRs
  • Regression reports are generated
  • Unit tests for regression detection
  • Integration tests with historical data

Implementation Notes

  • Query TestGrid for historical test data
  • Query GitHub API for PR/commit information
  • Match test failures to PR merge dates
  • Generate regression timeline
  • Support --detect-regressions command

Metadata

Metadata

Assignees

No one assigned

    Labels

    lifecycle/rottenDenotes an issue or PR that has aged beyond stale and will be auto-closed.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions