Skip to content

Commit 061a265

Browse files
rrosattirrosatti
authored
[Konflux] improve backend performance with targeted catalog lookups and response caching (#2630)
* fix(konflux): replace unfiltered catalog scan with targeted entity lookup getRelatedEntities was calling catalog.getEntities() with no filter, fetching the entire catalog on every request. Now uses the parent entity's hasPart relations with catalog.getEntitiesByRefs() to fetch only the needed subcomponents. * perf(konflux): cache API clients and catalog lookups, strip managedFields - Cache CustomObjectsApi instances per cluster instead of creating new KubeConfig + client on every request. Auth headers are injected per-request via middleware, so cached clients are safe to share. - Add a 30s TTL cache for catalog entity/config/combination lookups to avoid duplicate calls across parallel resource requests. - Strip metadata.managedFields from K8s and Kubearchive responses to reduce payload size (~50% reduction). * perf(konflux): increase staleTime and disable refetchOnWindowFocus Increase react-query staleTime from 30s to 5min and disable refetchOnWindowFocus to reduce unnecessary re-fetches when switching browser tabs. * chore(konflux): add changesets Add changesets with the changes made for both konflux and konflux-backend plugins. * fixup! perf(konflux): cache API clients and catalog lookups, strip managedFields * fixup! perf(konflux): cache API clients and catalog lookups, strip managedFields
1 parent a86326d commit 061a265

12 files changed

Lines changed: 270 additions & 91 deletions

File tree

‎workspaces/konflux/.changeset/better-melons-shave.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
---
2+
'@red-hat-developer-hub/backstage-plugin-konflux-backend': patch
3+
---
4+
5+
- Fix: replace unfiltered catalog scan in getRelatedEntities with targeted lookup using hasPart relations and getEntitiesByRefs.
6+
- Perf: cache K8s API clients per cluster and catalog lookups (30s TTL) to avoid redundant work across parallel requests.
7+
- Perf: strip metadata.managedFields from K8s and Kubearchive responses to reduce payload size.

‎workspaces/konflux/.changeset/fifty-moose-appear.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
'@red-hat-developer-hub/backstage-plugin-konflux': patch
3+
---
4+
5+
- Increase react-query staleTime from 30s to 5min and disable refetchOnWindowFocus to reduce unnecessary re-fetches.

‎workspaces/konflux/plugins/konflux-backend/src/helpers/__tests__/config.test.ts‎

Lines changed: 14 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -113,6 +113,7 @@ describe('config', () => {
113113
mockCatalog = {
114114
getEntities: jest.fn(),
115115
getEntityByRef: jest.fn(),
116+
getEntitiesByRefs: jest.fn().mockResolvedValue({ items: [] }),
116117
} as unknown as CatalogService;
117118

118119
mockCredentials = {} as BackstageCredentials;
@@ -269,8 +270,13 @@ describe('config', () => {
269270

270271
mockParseEntityKonfluxConfig.mockReturnValue(clusterConfigs);
271272

272-
const entity = createMockEntity('test-entity');
273-
(mockCatalog.getEntities as jest.Mock).mockResolvedValue({
273+
const entity = createMockEntity('test-entity', undefined, [
274+
{
275+
type: 'hasPart',
276+
targetRef: 'component:default/subcomponent1',
277+
},
278+
]);
279+
(mockCatalog.getEntitiesByRefs as jest.Mock).mockResolvedValue({
274280
items: [subcomponent1],
275281
});
276282

@@ -502,14 +508,18 @@ describe('config', () => {
502508
authProvider: 'serviceAccount',
503509
};
504510

505-
const entity = createMockEntity('test-entity');
506511
const subcomponent1 = createMockEntity('sub1', {}, [
507512
{ type: 'partOf', targetRef: 'component:default/test-entity' },
508513
]);
509514
const subcomponent2 = createMockEntity('sub2', {}, [
510515
{ type: 'partOf', targetRef: 'component:default/test-entity' },
511516
]);
512517

518+
const entity = createMockEntity('test-entity', undefined, [
519+
{ type: 'hasPart', targetRef: 'component:default/sub1' },
520+
{ type: 'hasPart', targetRef: 'component:default/sub2' },
521+
]);
522+
513523
mockParseSubcomponentClusterConfigurations.mockReturnValue([
514524
createMockSubcomponentClusterConfig('sub1', 'cluster1', 'ns1', [
515525
'app1',
@@ -519,7 +529,7 @@ describe('config', () => {
519529
]),
520530
]);
521531

522-
(mockCatalog.getEntities as jest.Mock).mockResolvedValue({
532+
(mockCatalog.getEntitiesByRefs as jest.Mock).mockResolvedValue({
523533
items: [subcomponent1, subcomponent2],
524534
});
525535

‎workspaces/konflux/plugins/konflux-backend/src/helpers/client-factory.ts‎

Lines changed: 63 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,10 +14,21 @@
1414
* limitations under the License.
1515
*/
1616
import { KonfluxConfig } from '@red-hat-developer-hub/backstage-plugin-konflux-common';
17-
import type { KubeConfig } from '@kubernetes/client-node';
17+
import type { KubeConfig, CustomObjectsApi } from '@kubernetes/client-node';
1818
import { KonfluxLogger } from './logger';
1919
import { getKubeClient } from './kube-client';
2020

21+
/**
22+
* Cache for CustomObjectsApi clients keyed by "cluster:apiUrl".
23+
*
24+
* Per-request auth headers are injected via middleware in the callers,
25+
* so cached clients are safe to share across users.
26+
*
27+
* Caching avoids creating a new KubeConfig, CustomObjectsApi, and TLS
28+
* connection on every API call.
29+
*/
30+
const clientCache = new Map<string, CustomObjectsApi>();
31+
2132
/**
2233
* Creates a KubeConfig for connecting to a Kubernetes cluster.
2334
*
@@ -94,3 +105,54 @@ export const createKubeConfig = async (
94105
return null;
95106
}
96107
};
108+
109+
/**
110+
* Returns a cached CustomObjectsApi client for the given cluster, creating
111+
* one on first access. The client is safe to share across requests because
112+
* auth headers are injected per-request via middleware.
113+
*
114+
* @param konfluxConfig - Konflux configuration
115+
* @param cluster - Cluster name
116+
* @param konfluxLogger - Logger instance
117+
* @param useKubearchiveUrl - If true, targets the kubearchive API URL
118+
* @returns CustomObjectsApi instance or null if client creation fails
119+
*/
120+
export const getOrCreateClient = async (
121+
konfluxConfig: KonfluxConfig | undefined,
122+
cluster: string,
123+
konfluxLogger: KonfluxLogger,
124+
useKubearchiveUrl = false,
125+
): Promise<CustomObjectsApi | null> => {
126+
if (!konfluxConfig) {
127+
return null;
128+
}
129+
130+
const clusterConfig = konfluxConfig.clusters?.[cluster];
131+
const apiUrl = useKubearchiveUrl
132+
? clusterConfig?.kubearchiveApiUrl
133+
: clusterConfig?.apiUrl;
134+
135+
const cacheKey = `${cluster}:${apiUrl}`;
136+
137+
const cached = clientCache.get(cacheKey);
138+
if (cached) {
139+
return cached;
140+
}
141+
142+
const kc = await createKubeConfig(
143+
konfluxConfig,
144+
cluster,
145+
konfluxLogger,
146+
clusterConfig?.serviceAccountToken,
147+
useKubearchiveUrl,
148+
);
149+
150+
if (!kc) {
151+
return null;
152+
}
153+
154+
const { CustomObjectsApi } = await getKubeClient();
155+
const client = kc.makeApiClient(CustomObjectsApi);
156+
clientCache.set(cacheKey, client);
157+
return client;
158+
};

‎workspaces/konflux/plugins/konflux-backend/src/helpers/config.ts‎

Lines changed: 16 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@ import {
2828
import { BackstageCredentials } from '@backstage/backend-plugin-api';
2929
import { Config } from '@backstage/config';
3030

31-
import { Entity, stringifyEntityRef } from '@backstage/catalog-model';
31+
import { Entity } from '@backstage/catalog-model';
3232
import { CatalogService } from '@backstage/plugin-catalog-node';
3333
import { KonfluxLogger } from './logger';
3434

@@ -37,8 +37,9 @@ import { KonfluxLogger } from './logger';
3737
*
3838
* In Konflux plugin, a "subcomponent" refers to a Backstage Component that has a
3939
* `subcomponentOf` relationship to the current Component being viewed.
40-
* Backstage automatically creates a `partOf` relation from the subcomponent
41-
* entity to the parent entity, which is what we query for.
40+
* Backstage automatically creates a `hasPart` relation on the parent entity
41+
* for each subcomponent, so we read those refs and batch-fetch the entities
42+
* instead of scanning the entire catalog.
4243
*
4344
* @param entity - The main/parent entity to find related entities for
4445
* @param credentials - Backstage credentials for authentication
@@ -53,20 +54,22 @@ const getRelatedEntities = async (
5354
): Promise<Entity[] | null> => {
5455
try {
5556
if (catalog) {
56-
const entityRef = stringifyEntityRef(entity);
57+
const hasPartRefs = (entity.relations || [])
58+
.filter(rel => rel.type === 'hasPart')
59+
.map(rel => rel.targetRef);
5760

58-
const allEntitiesForFiltering = await catalog.getEntities(
59-
{},
61+
if (hasPartRefs.length === 0) {
62+
return [];
63+
}
64+
65+
const response = await catalog.getEntitiesByRefs(
66+
{ entityRefs: hasPartRefs },
6067
{ credentials },
6168
);
62-
const filteredRelatedEntities = allEntitiesForFiltering.items.filter(
63-
item =>
64-
item.relations?.some(
65-
rel => rel.type === 'partOf' && rel.targetRef === entityRef,
66-
),
67-
);
6869

69-
return filteredRelatedEntities;
70+
return response.items.filter(
71+
(item): item is Entity => item !== undefined,
72+
);
7073
}
7174
return null;
7275
} catch (error) {

‎workspaces/konflux/plugins/konflux-backend/src/services/__tests__/kubearchive-service.test.ts‎

Lines changed: 9 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -16,13 +16,13 @@
1616

1717
import { KubearchiveService } from '../kubearchive-service';
1818
import { LoggerService } from '@backstage/backend-plugin-api';
19-
import type { KubeConfig, CustomObjectsApi } from '@kubernetes/client-node';
19+
import type { CustomObjectsApi } from '@kubernetes/client-node';
2020
import {
2121
KonfluxConfig,
2222
K8sResourceCommonWithClusterInfo,
2323
} from '@red-hat-developer-hub/backstage-plugin-konflux-common';
2424
import { KonfluxLogger } from '../../helpers/logger';
25-
import { createKubeConfig } from '../../helpers/client-factory';
25+
import { getOrCreateClient } from '../../helpers/client-factory';
2626
import { getKubeClient } from '../../helpers/kube-client';
2727

2828
jest.mock('../../helpers/logger');
@@ -32,7 +32,6 @@ jest.mock('../../helpers/kube-client');
3232
describe('KubearchiveService', () => {
3333
let mockLogger: jest.Mocked<LoggerService>;
3434
let mockKonfluxLogger: jest.Mocked<KonfluxLogger>;
35-
let mockKubeConfig: jest.Mocked<KubeConfig>;
3635
let mockCustomObjectsApi: jest.Mocked<CustomObjectsApi>;
3736
let service: KubearchiveService;
3837

@@ -104,18 +103,13 @@ describe('KubearchiveService', () => {
104103
listNamespacedCustomObjectWithHttpInfo: jest.fn(),
105104
} as unknown as jest.Mocked<CustomObjectsApi>;
106105

107-
mockKubeConfig = {
108-
makeApiClient: jest.fn().mockReturnValue(mockCustomObjectsApi),
109-
loadFromOptions: jest.fn(),
110-
} as unknown as jest.Mocked<KubeConfig>;
111-
112106
(
113107
KonfluxLogger as jest.MockedClass<typeof KonfluxLogger>
114108
).mockImplementation(() => mockKonfluxLogger);
115109

116110
(
117-
createKubeConfig as jest.MockedFunction<typeof createKubeConfig>
118-
).mockResolvedValue(mockKubeConfig);
111+
getOrCreateClient as jest.MockedFunction<typeof getOrCreateClient>
112+
).mockResolvedValue(mockCustomObjectsApi);
119113

120114
class MockObservable<T> {
121115
constructor(public value: T) {}
@@ -545,10 +539,10 @@ describe('KubearchiveService', () => {
545539
});
546540

547541
(
548-
createKubeConfig as jest.MockedFunction<typeof createKubeConfig>
542+
getOrCreateClient as jest.MockedFunction<typeof getOrCreateClient>
549543
).mockResolvedValue(null);
550544

551-
// provide OIDC token so it passes token check and reaches createKubeConfig
545+
// provide OIDC token so it passes token check and reaches getOrCreateClient
552546
await expect(
553547
service.fetchResources({
554548
konfluxConfig,
@@ -564,7 +558,7 @@ describe('KubearchiveService', () => {
564558
).rejects.toThrow(`Cluster '${cluster}' not found`);
565559

566560
expect(mockKonfluxLogger.error).toHaveBeenCalledWith(
567-
'Failed to create KubeConfig - cluster not found',
561+
'Failed to create API client - cluster not found',
568562
undefined,
569563
expect.objectContaining({
570564
cluster,
@@ -586,7 +580,7 @@ describe('KubearchiveService', () => {
586580
});
587581

588582
(
589-
createKubeConfig as jest.MockedFunction<typeof createKubeConfig>
583+
getOrCreateClient as jest.MockedFunction<typeof getOrCreateClient>
590584
).mockResolvedValue(null);
591585

592586
await expect(
@@ -602,7 +596,7 @@ describe('KubearchiveService', () => {
602596
).rejects.toThrow(`Cluster '${cluster}' not found`);
603597

604598
expect(mockKonfluxLogger.error).toHaveBeenCalledWith(
605-
'Failed to create KubeConfig - cluster not found',
599+
'Failed to create API client - cluster not found',
606600
undefined,
607601
expect.objectContaining({
608602
cluster,

‎workspaces/konflux/plugins/konflux-backend/src/services/__tests__/resource-fetcher.test.ts‎

Lines changed: 8 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -20,10 +20,10 @@ import {
2020
FetchOptions,
2121
} from '../resource-fetcher';
2222
import { LoggerService } from '@backstage/backend-plugin-api';
23-
import type { KubeConfig, CustomObjectsApi } from '@kubernetes/client-node';
23+
import type { CustomObjectsApi } from '@kubernetes/client-node';
2424
import { K8sResourceCommonWithClusterInfo } from '@red-hat-developer-hub/backstage-plugin-konflux-common';
2525
import { KubearchiveService } from '../kubearchive-service';
26-
import { createKubeConfig } from '../../helpers/client-factory';
26+
import { getOrCreateClient } from '../../helpers/client-factory';
2727
import { KonfluxLogger } from '../../helpers/logger';
2828
import { getKubeClient } from '../../helpers/kube-client';
2929

@@ -36,7 +36,6 @@ describe('ResourceFetcherService', () => {
3636
let mockLogger: jest.Mocked<LoggerService>;
3737
let mockKonfluxLogger: jest.Mocked<KonfluxLogger>;
3838
let mockKubearchiveService: jest.Mocked<KubearchiveService>;
39-
let mockKubeConfig: jest.Mocked<KubeConfig>;
4039
let mockCustomObjectsApi: jest.Mocked<CustomObjectsApi>;
4140
let service: ResourceFetcherService;
4241

@@ -122,10 +121,6 @@ describe('ResourceFetcherService', () => {
122121
listNamespacedCustomObjectWithHttpInfo: jest.fn(),
123122
} as unknown as jest.Mocked<CustomObjectsApi>;
124123

125-
mockKubeConfig = {
126-
makeApiClient: jest.fn().mockReturnValue(mockCustomObjectsApi),
127-
} as unknown as jest.Mocked<KubeConfig>;
128-
129124
(
130125
KonfluxLogger as jest.MockedClass<typeof KonfluxLogger>
131126
).mockImplementation(() => mockKonfluxLogger);
@@ -135,8 +130,8 @@ describe('ResourceFetcherService', () => {
135130
).mockImplementation(() => mockKubearchiveService);
136131

137132
(
138-
createKubeConfig as jest.MockedFunction<typeof createKubeConfig>
139-
).mockResolvedValue(mockKubeConfig);
133+
getOrCreateClient as jest.MockedFunction<typeof getOrCreateClient>
134+
).mockResolvedValue(mockCustomObjectsApi);
140135

141136
class MockObservable<T> {
142137
constructor(public value: T) {}
@@ -174,11 +169,10 @@ describe('ResourceFetcherService', () => {
174169

175170
expect(result.items).toEqual(mockItems);
176171
expect(result.continueToken).toBeUndefined();
177-
expect(createKubeConfig).toHaveBeenCalledWith(
172+
expect(getOrCreateClient).toHaveBeenCalledWith(
178173
context.konfluxConfig,
179174
'cluster1',
180175
mockKonfluxLogger,
181-
'service-token-123',
182176
);
183177
expect(
184178
mockCustomObjectsApi.listNamespacedCustomObjectWithHttpInfo,
@@ -301,11 +295,10 @@ describe('ResourceFetcherService', () => {
301295

302296
await service.fetchFromKubernetes(context);
303297

304-
expect(createKubeConfig).toHaveBeenCalledWith(
305-
expect.anything(),
298+
expect(getOrCreateClient).toHaveBeenCalledWith(
306299
expect.anything(),
300+
'cluster1',
307301
expect.anything(),
308-
'oidc-token-456',
309302
);
310303
expect(mockKonfluxLogger.debug).toHaveBeenCalledWith(
311304
'Using OIDC token for authentication',
@@ -423,7 +416,7 @@ describe('ResourceFetcherService', () => {
423416
it('should throw error when cluster not found', async () => {
424417
const context = createMockFetchContext();
425418
(
426-
createKubeConfig as jest.MockedFunction<typeof createKubeConfig>
419+
getOrCreateClient as jest.MockedFunction<typeof getOrCreateClient>
427420
).mockResolvedValue(null);
428421

429422
await expect(service.fetchFromKubernetes(context)).rejects.toThrow(

0 commit comments

Comments
 (0)