Skip to content

Commit bf72ffc

Browse files
imykhnoimykhno
authored
RHIDP-12121: Support "StatusGrouped" and "Average" types of aggregation (#2923)
* feat(scorecard): add `average` aggregation type Signed-off-by: Ihor Mykhno <imykhno@redhat.com> * fix(scorecard): average gauge out-of-range Signed-off-by: Ihor Mykhno <imykhno@redhat.com> * fix(scorecard): default aggregation threshold value usage Signed-off-by: Ihor Mykhno <imykhno@redhat.com> * fix(scorecard): sonarqube issues Signed-off-by: Ihor Mykhno <imykhno@redhat.com> * refactor(scorecard): update type definitions for tooltip and state management Signed-off-by: Ihor Mykhno <imykhno@redhat.com> * fix(scorecard): update tooltip translations and aggregation configuration Signed-off-by: Ihor Mykhno <imykhno@redhat.com> * refactor(scorecard): rename `aggregationResultThresholds` to `thresholds` in configuration Signed-off-by: Ihor Mykhno <imykhno@redhat.com> * refactor(scorecard): consolidate AggregationConfig imports across service files Signed-off-by: Ihor Mykhno <imykhno@redhat.com> * fix(scorecard): issues after merging main Signed-off-by: Ihor Mykhno <imykhno@redhat.com> * feat(scorecard): enhance aggregation KPI configuration with new types Signed-off-by: Ihor Mykhno <imykhno@redhat.com> * Revert "feat(scorecard): enhance aggregation KPI configuration with new types" This reverts commit b264e55. * refactor(scorecard): update README and aggregation configuration for average KPI support Signed-off-by: Ihor Mykhno <imykhno@redhat.com> * feat(scorecard): use the i18n for loading indicators and remove hardcoded labels Signed-off-by: Ihor Mykhno <imykhno@redhat.com> * refactor(scorecard): rename `aggregationKinds` to `aggregationTypes` Signed-off-by: Ihor Mykhno <imykhno@redhat.com> * feat(scorecard): add detailed documentation for threshold rules in configuration Signed-off-by: Ihor Mykhno <imykhno@redhat.com> * feat(scorecard): enhance threshold configuration with aggregation rules Signed-off-by: Ihor Mykhno <imykhno@redhat.com> * fix(scorecard): average aggregation card percentage value Signed-off-by: Ihor Mykhno <imykhno@redhat.com> * refactor(scorecard): rename `ThresholdRuleAggregationConfig` to `AggregationThresholdRule` Signed-off-by: Ihor Mykhno <imykhno@redhat.com> --------- Signed-off-by: Ihor Mykhno <imykhno@redhat.com>
1 parent f3aea26 commit bf72ffc

96 files changed

Lines changed: 4191 additions & 1026 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎workspaces/scorecard/.changeset/twenty-trains-fetch.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
---
2+
'@red-hat-developer-hub/backstage-plugin-scorecard-backend': minor
3+
'@red-hat-developer-hub/backstage-plugin-scorecard-common': minor
4+
'@red-hat-developer-hub/backstage-plugin-scorecard-node': minor
5+
'@red-hat-developer-hub/backstage-plugin-scorecard': minor
6+
---
7+
8+
Adds `**average**` as an aggregation KPI type alongside `**statusGrouped**`, with configurable `**options.statusScores**` and optional `**options.thresholds**` (same shape as metric thresholds) for homepage donut coloring against `**averageScore × 100**`, with built-in defaults when omitted.

‎workspaces/scorecard/README.md‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,3 +12,13 @@ yarn install
1212
- **app-legacy:** Run `yarn start:legacy` to start the legacy frontend with the backend. Use the Scorecard tab on entity pages or the scorecard homepage card.
1313

1414
> **Notice:** The guest user has admin permissions in this application for quick setup. For better control, specify more users and groups in `app-config.local.yaml` and define a separate admin/admins permission instead of using the guest user. Using the guest user as admin is not recommended for permission management.
15+
16+
## Documentation
17+
18+
| Topic | Location |
19+
| ----------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------- |
20+
| Aggregation KPIs (`statusGrouped`, `average`), API, ownership | [plugins/scorecard-backend/docs/aggregation.md](plugins/scorecard-backend/docs/aggregation.md) |
21+
| Backend installation and RBAC, **`scorecard.aggregationKPIs`** examples | [plugins/scorecard-backend/README.md](plugins/scorecard-backend/README.md) |
22+
| Drill-down (entity list for a metric) | [plugins/scorecard-backend/docs/drill-down.md](plugins/scorecard-backend/docs/drill-down.md) |
23+
| Metric thresholds, annotations, **average KPI result colors** | [plugins/scorecard-backend/docs/thresholds.md](plugins/scorecard-backend/docs/thresholds.md) |
24+
| Frontend (homepage cards, NFS) | [plugins/scorecard/README.md](plugins/scorecard/README.md) |

‎workspaces/scorecard/app-config.yaml‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -211,6 +211,27 @@ scorecard:
211211
type: statusGrouped
212212
description: This KPI is provide information about GitHub open PRs grouped by status.
213213
metricId: github.open_prs
214+
openPrsWeightedKpi:
215+
title: GitHub Open PRs (weighted health)
216+
type: average
217+
description: Weighted health average for open PRs by threshold status across your entities.
218+
metricId: github.open_prs
219+
options:
220+
statusScores:
221+
success: 100
222+
warning: 40
223+
error: 0
224+
thresholds:
225+
rules:
226+
- key: success
227+
expression: '>=80'
228+
color: '#6bb300' # green
229+
- key: warning
230+
expression: '30-79'
231+
color: 'rgb(224, 189, 108)' # light orange
232+
- key: error
233+
expression: '<30'
234+
color: '#be1ec7' # purple
214235
openIssuesKpi:
215236
title: Jira Open Issues KPI
216237
type: statusGrouped

‎workspaces/scorecard/packages/app-legacy/e2e-tests/constants/homepageWidgetTitles.ts‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ export const AGGREGATED_CARDS_METRIC_IDS = {
1919
withDefaultAggregation: 'github.open_prs',
2020
withGithubOpenPrs: 'openPrsKpi',
2121
withJiraOpenIssuesKpi: 'openIssuesKpi',
22+
withOpenPrsWeightedKpi: 'openPrsWeightedKpi',
2223
} as const;
2324

2425
export const AGGREGATED_CARDS_WIDGET_TITLES = {
@@ -27,4 +28,5 @@ export const AGGREGATED_CARDS_WIDGET_TITLES = {
2728
withDefaultAggregation: 'Scorecard: With default aggregation config (GitHub)',
2829
withGithubOpenPrs: 'Scorecard: GitHub open PRs',
2930
withJiraOpenIssuesKpi: 'Scorecard: Jira open blocking tickets',
31+
withOpenPrsWeightedKpi: 'Scorecard: GitHub open PRs (weighted health)',
3032
} as const;

‎workspaces/scorecard/packages/app-legacy/e2e-tests/constants/routes.ts‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,10 @@ export const ScorecardRoutes = {
2222
OPEN_ISSUES_KPI_METADATA_ROUTE:
2323
'**/api/scorecard/aggregations/openIssuesKpi/metadata',
2424
OPEN_PRS_KPI_AGGREGATION_ROUTE: '**/api/scorecard/aggregations/openPrsKpi',
25+
OPEN_PRS_WEIGHTED_KPI_METADATA_ROUTE:
26+
'**/api/scorecard/aggregations/openPrsWeightedKpi/metadata',
27+
OPEN_PRS_WEIGHTED_KPI_AGGREGATION_ROUTE:
28+
'**/api/scorecard/aggregations/openPrsWeightedKpi',
2529
OPEN_ISSUES_KPI_AGGREGATION_ROUTE:
2630
'**/api/scorecard/aggregations/openIssuesKpi',
2731
/** Default aggregation when aggregationId is the metric id (no KPI entry). */

‎workspaces/scorecard/packages/app-legacy/e2e-tests/pages/HomePage.ts‎

Lines changed: 22 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@
1515
*/
1616

1717
import { Locator, Page, expect } from '@playwright/test';
18+
import { AGGREGATED_CARDS_WIDGET_TITLES } from '../constants/homepageWidgetTitles';
1819
import {
1920
ScorecardMessages,
2021
getEntitiesLabel,
@@ -62,6 +63,11 @@ export class HomePage {
6263
cardPattern = /Scorecard:\s*GitHub open PRs|ScorecardGithubHomepage/i;
6364
} else if (cardName === 'Scorecard: Jira open blocking') {
6465
cardPattern = /Scorecard:\s*Jira open blocking|ScorecardJiraHomepage/i;
66+
} else if (
67+
cardName === AGGREGATED_CARDS_WIDGET_TITLES.withOpenPrsWeightedKpi
68+
) {
69+
cardPattern =
70+
/Scorecard:\s*GitHub open PRs \(weighted health\)|ScorecardOpenPrsWeightedKpi/i;
6571
} else {
6672
cardPattern = new RegExp(escapeRegex(cardName), 'i');
6773
}
@@ -125,6 +131,21 @@ export class HomePage {
125131
}
126132

127133
async clickDrillDownLink() {
128-
await this.page.getByText(getEntitiesLabel(this.translations)).click();
134+
// CardSubheader renders the count as a Link (e.g. "10 entities"). The card
135+
// description can also contain the word "entities" (see API metadata), so
136+
// getByText(entitiesLabel) is ambiguous. MUI Tooltip also sets the link’s
137+
// accessible name to the long tooltip, so getByRole('link', { name }) is
138+
// locale‑fragile. Match only links whose *visible* text is "{{count}} <label>".
139+
const entitiesLabel = getEntitiesLabel(this.translations);
140+
await this.page
141+
.getByRole('link')
142+
.filter({
143+
hasText: new RegExp(
144+
String.raw`^\d+\s*${escapeRegex(entitiesLabel)}$`,
145+
'i',
146+
),
147+
})
148+
.first()
149+
.click();
129150
}
130151
}

‎workspaces/scorecard/packages/app-legacy/e2e-tests/scorecard.test.ts‎

Lines changed: 181 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,10 @@ import {
3737
emptyGithubAggregatedResponse,
3838
emptyJiraAggregatedResponse,
3939
openPrsKpiMetadataResponse,
40+
openPrsWeightedAggregatedResponse,
41+
emptyOpenPrsWeightedAggregatedResponse,
42+
openPrsWeightedKpiMetadataResponse,
43+
openPrsWeightedUnsupportedAggregationResponse,
4044
notAllowedAggregationErrorBody,
4145
githubEntitiesDrillDownResponse,
4246
jiraEntitiesDrillDownResponse,
@@ -58,6 +62,11 @@ import {
5862
mockAllDefaultHomepageAggregationsSuccess,
5963
mockHomepageAggregationsPermissionDenied,
6064
} from './utils/mockHomepageAggregations';
65+
import {
66+
expectAverageCardCenterPercent,
67+
verifyAverageDonutCenterTooltip,
68+
verifyAverageLegendTooltipForStatus,
69+
} from './utils/averageCardAssertions';
6170
import { runAccessibilityTests } from './utils/accessibility';
6271
import { ScorecardRoutes } from './constants/routes';
6372
import {
@@ -83,6 +92,7 @@ async function addAggregatedScorecardWidgets(homePage: HomePage) {
8392
await homePage.addCard(AGGREGATED_CARDS_WIDGET_TITLES.withDefaultAggregation);
8493
await homePage.addCard(AGGREGATED_CARDS_WIDGET_TITLES.withGithubOpenPrs);
8594
await homePage.addCard(AGGREGATED_CARDS_WIDGET_TITLES.withJiraOpenIssuesKpi);
95+
await homePage.addCard(AGGREGATED_CARDS_WIDGET_TITLES.withOpenPrsWeightedKpi);
8696

8797
await homePage.saveChanges();
8898
}
@@ -965,5 +975,176 @@ test.describe('Scorecard Plugin Tests', () => {
965975
);
966976
});
967977
});
978+
979+
test.describe('Average aggregation KPI (openPrsWeightedKpi)', () => {
980+
test('Verify title and description from API metadata', async () => {
981+
await mockApiResponse(
982+
page,
983+
ScorecardRoutes.OPEN_PRS_WEIGHTED_KPI_AGGREGATION_ROUTE,
984+
openPrsWeightedAggregatedResponse,
985+
);
986+
987+
await addWidgets(
988+
homePage,
989+
AGGREGATED_CARDS_WIDGET_TITLES.withOpenPrsWeightedKpi,
990+
);
991+
await page.reload();
992+
993+
const card = homePage.getCard(
994+
AGGREGATED_CARDS_METRIC_IDS.withOpenPrsWeightedKpi,
995+
);
996+
await expect(card).toBeVisible();
997+
await expect(card).toContainText(
998+
openPrsWeightedKpiMetadataResponse.title,
999+
);
1000+
await expect(card).toContainText(
1001+
openPrsWeightedKpiMetadataResponse.description,
1002+
);
1003+
});
1004+
1005+
test('Verify center score and average tooltips', async () => {
1006+
await mockApiResponse(
1007+
page,
1008+
ScorecardRoutes.OPEN_PRS_WEIGHTED_KPI_AGGREGATION_ROUTE,
1009+
openPrsWeightedAggregatedResponse,
1010+
);
1011+
1012+
await addWidgets(
1013+
homePage,
1014+
AGGREGATED_CARDS_WIDGET_TITLES.withOpenPrsWeightedKpi,
1015+
);
1016+
await page.reload();
1017+
1018+
const card = homePage.getCard(
1019+
AGGREGATED_CARDS_METRIC_IDS.withOpenPrsWeightedKpi,
1020+
);
1021+
await expectAverageCardCenterPercent(card, '50%');
1022+
await verifyAverageDonutCenterTooltip(
1023+
page,
1024+
card,
1025+
translations,
1026+
500,
1027+
1000,
1028+
);
1029+
await verifyAverageLegendTooltipForStatus(
1030+
page,
1031+
card,
1032+
translations,
1033+
currentLocale,
1034+
'success',
1035+
);
1036+
});
1037+
1038+
test('Verify empty aggregated response shows no data', async () => {
1039+
await mockApiResponse(
1040+
page,
1041+
ScorecardRoutes.OPEN_PRS_WEIGHTED_KPI_AGGREGATION_ROUTE,
1042+
emptyOpenPrsWeightedAggregatedResponse,
1043+
);
1044+
1045+
await addWidgets(
1046+
homePage,
1047+
AGGREGATED_CARDS_WIDGET_TITLES.withOpenPrsWeightedKpi,
1048+
);
1049+
await page.reload();
1050+
1051+
await homePage.expectCardHasNoDataFound(
1052+
AGGREGATED_CARDS_METRIC_IDS.withOpenPrsWeightedKpi,
1053+
);
1054+
});
1055+
1056+
test('Accessibility on weighted average card', async ({
1057+
browser: _browser,
1058+
}, testInfo) => {
1059+
await mockApiResponse(
1060+
page,
1061+
ScorecardRoutes.OPEN_PRS_WEIGHTED_KPI_AGGREGATION_ROUTE,
1062+
openPrsWeightedAggregatedResponse,
1063+
);
1064+
1065+
await homePage.navigateToHome();
1066+
await homePage.enterEditMode();
1067+
await homePage.clearAllCards();
1068+
await homePage.addCard(
1069+
AGGREGATED_CARDS_WIDGET_TITLES.withOpenPrsWeightedKpi,
1070+
);
1071+
await homePage.saveChanges();
1072+
await page.reload();
1073+
1074+
await runAccessibilityTests(page, testInfo);
1075+
});
1076+
1077+
test('GitHub weighted KPI: drill-down, average card, and table', async () => {
1078+
await mockApiResponse(
1079+
page,
1080+
ScorecardRoutes.OPEN_PRS_WEIGHTED_KPI_AGGREGATION_ROUTE,
1081+
openPrsWeightedAggregatedResponse,
1082+
);
1083+
await mockScorecardEntitiesDrillDownWithSort(
1084+
page,
1085+
githubEntitiesDrillDownResponse,
1086+
'github.open_prs',
1087+
);
1088+
1089+
await homePage.navigateToHome();
1090+
await page.reload();
1091+
await homePage.enterEditMode();
1092+
await homePage.clearAllCards();
1093+
await homePage.addCard(
1094+
AGGREGATED_CARDS_WIDGET_TITLES.withOpenPrsWeightedKpi,
1095+
);
1096+
await homePage.saveChanges();
1097+
1098+
await homePage.clickDrillDownLink();
1099+
await scorecardDrillDownPage.expectOnPage('github.open_prs', {
1100+
aggregationId: AGGREGATED_CARDS_METRIC_IDS.withOpenPrsWeightedKpi,
1101+
});
1102+
await scorecardDrillDownPage.expectPageTitle(
1103+
'github.open_prs',
1104+
openPrsWeightedKpiMetadataResponse.title,
1105+
);
1106+
1107+
const drillCard = scorecardDrillDownPage.getDrillDownCard(
1108+
'github.open_prs',
1109+
{
1110+
aggregationId: AGGREGATED_CARDS_METRIC_IDS.withOpenPrsWeightedKpi,
1111+
},
1112+
);
1113+
await expectAverageCardCenterPercent(drillCard, '50%');
1114+
await scorecardDrillDownPage.verifySomeEntitiesNotReportingTooltip();
1115+
await scorecardDrillDownPage.expectTableHeadersVisible();
1116+
await scorecardDrillDownPage.expectEntityNamesVisible([
1117+
'all-scorecards-service',
1118+
'Red Hat Developer Hub',
1119+
'github-scorecard-only-service',
1120+
'all-scorecards-service-different-owner',
1121+
'backend-api',
1122+
]);
1123+
});
1124+
});
1125+
1126+
test.describe('Unsupported aggregation type', () => {
1127+
test('Shows unsupported message when aggregationType is unknown', async () => {
1128+
await mockApiResponse(
1129+
page,
1130+
ScorecardRoutes.OPEN_PRS_WEIGHTED_KPI_AGGREGATION_ROUTE,
1131+
openPrsWeightedUnsupportedAggregationResponse,
1132+
);
1133+
1134+
await addWidgets(
1135+
homePage,
1136+
AGGREGATED_CARDS_WIDGET_TITLES.withOpenPrsWeightedKpi,
1137+
);
1138+
await page.reload();
1139+
1140+
const card = homePage.getCard(
1141+
AGGREGATED_CARDS_METRIC_IDS.withOpenPrsWeightedKpi,
1142+
);
1143+
await expect(card).toContainText(
1144+
translations.errors.unsupportedAggregationType,
1145+
);
1146+
await expect(card).toContainText('customUnknownAggregationKind');
1147+
});
1148+
});
9681149
});
9691150
});

‎workspaces/scorecard/packages/app-legacy/e2e-tests/utils/apiUtils.ts‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,6 @@
1616
import { Page, expect } from '@playwright/test';
1717
import { ScorecardRoutes } from '../constants/routes';
1818

19-
/** Metric-id aggregation URLs (drill-down uses `github.open_prs` / `jira.open_issues`). */
2019
const GITHUB_AGGREGATION_ROUTE =
2120
ScorecardRoutes.GITHUB_OPEN_PRS_METRIC_AGGREGATION_ROUTE;
2221
const JIRA_AGGREGATION_ROUTE =

0 commit comments

Comments
 (0)