feat(bulk-import): add support for on-hehalf-of user access (#2647)

* feat(bulk-import): add support for on-hehalf-of user access

Signed-off-by: Patrick Knight <pknight@redhat.com>

* feat(bulk-import): add tests for on behalf of

Signed-off-by: Patrick Knight <pknight@redhat.com>

* feat(bulk-import): fix scm-hosts audit event id

Signed-off-by: Patrick Knight <pknight@redhat.com>

* feat(bulk-import): ensure that tokens are not logged to audit logger

Signed-off-by: Patrick Knight <pknight@redhat.com>

* feat(bulk-import): fix openapi header type mismatch

Signed-off-by: Patrick Knight <pknight@redhat.com>

* feat(bulk-import): fix raw token in query key

Signed-off-by: Patrick Knight <pknight@redhat.com>

* feat(bulk-import): fix sonarqube issues

Signed-off-by: Patrick Knight <pknight@redhat.com>

* feat(bulk-import): fix sonarqube duplication issues

Signed-off-by: Patrick Knight <pknight@redhat.com>

* feat(bulk-import): fix e2e tests

Signed-off-by: Patrick Knight <pknight@redhat.com>

* feat(bulk-import): make auth providers a new requirement

Signed-off-by: Patrick Knight <pknight@redhat.com>

* feat(bulk-import): consolidate getDeleteImportActionPath and getImportActionPath

Signed-off-by: Patrick Knight <pknight@redhat.com>

---------

Signed-off-by: Patrick Knight <pknight@redhat.com>

Author: PatAKnight

workspaces/bulk-import/.changeset/small-games-live.md

@@ -0,0 +1,31 @@
+---
+'@red-hat-developer-hub/backstage-plugin-bulk-import-backend': minor
+'@red-hat-developer-hub/backstage-plugin-bulk-import': minor
+---
+
+## On Behalf of User Access
+
+This release introduces the ability for the Bulk Import plugin to fetch repository and organization listings **on behalf of the signed-in user**, using their OAuth credentials rather than relying solely on server-side integration credentials (GitHub App, PAT, or GitLab token).
+
+### What Changed
+
+**Backend (`bulk-import-backend`)**
+
+- Added a new `GET /api/bulk-import/scm-hosts` endpoint that returns the configured GitHub and GitLab integration host URLs as a `SCMHostList` object, enabling the frontend to discover which hosts to request OAuth tokens for.
+- The `GET /repositories` and `GET /organizations/{organizationName}/repositories` endpoints now **require** the `x-scm-tokens` request header — a JSON map of SCM host base URL to user OAuth token. Requests that omit this header, or supply an empty or oversized header, are rejected with HTTP 401. This ensures repository listings are always scoped to the signed-in user's access and never fall back to server-wide integration credentials.
+- The `x-scm-tokens` header is stripped from the request immediately upon receipt, before the permission check and before any audit event is created, so OAuth token values are never persisted in audit logs.
+- When user tokens are provided for GitHub, the Octokit response cache is intentionally disabled to prevent cross-user ETag cache leakage. Server-side credential paths are not affected.
+- Introduced a shared `GitApiService` interface and common SCM types (`SCMOrganization`, `SCMRepository`, `SCMFetchError`, etc.) to unify the GitHub and GitLab service implementations under a consistent contract.
+
+**Frontend (`bulk-import`)**
+
+- The plugin now has a **soft dependency** on `@backstage/integration-react`'s `ScmAuthApi`. If the API is registered in the application, the plugin automatically requests OAuth tokens for each configured SCM host and passes them to the backend to enable user-scoped repository listings.
+- Added `getSCMHosts()` to the `BulkImportAPI` interface with a corresponding `GET /api/bulk-import/scm-hosts` client call, used to discover host URLs before requesting user tokens.
+- User OAuth tokens are transmitted to the backend via the `X-SCM-Tokens` request header as a JSON-encoded map.
+- If the SCM OAuth integration is not configured or token collection fails for all hosts, the repository list query is **blocked** on the frontend and the hook surfaces a descriptive error. This prevents the frontend from firing a request that will always be rejected with 401.
+
+### Required Configuration
+
+The GitHub and/or GitLab OAuth provider must be configured in the Backstage application for repository listing to work. Deployments that previously relied on server-side credentials alone for the repository list view must add an SCM OAuth provider to continue using this feature.
+
+If `ScmAuthApi` is not registered or tokens cannot be obtained for any configured SCM host, users will see an error prompting them to configure the SCM OAuth integration.

workspaces/bulk-import/e2e-tests/app.test.ts

@@ -22,11 +22,13 @@ import {
   mockBulkImportDryRunResponse,
   mockBulkImportImportsResponse,
   mockBulkImportRepositoriesResponse,
+  mockBulkImportSCMHostsResponse,
   mockImportByRepoData,
   mockImportByRepoFrontendData,
   mockImportsData,
   mockImportsDryRunData,
   mockRepositoriesData,
+  mockSCMHostsData,
 } from './utils/apiUtils';
 import {
   getPreviewSidebarSnapshots,
@@ -51,6 +53,24 @@ test.describe('Bulk Import', () => {
     context = await browser.newContext();
     sharedPage = await context.newPage();
 
+    // The backend's GET /repositories and GET /organizations/{org}/repositories
+    // endpoints require the X-SCM-Tokens header (HTTP 401 otherwise). In a real
+    // deployment, the frontend obtains these tokens from the configured GitHub /
+    // GitLab OAuth provider via ScmAuthApi and sends them with every listing
+    // request. See plugins/bulk-import/README.md → "Required OAuth Configuration".
+    //
+    // In these e2e tests we bypass that requirement in two steps:
+    //   1. Mock GET /api/bulk-import/scm-hosts to return empty host arrays.
+    //      The useRepositories hook hits the `!urls?.length → return undefined`
+    //      early-return path, so tokenFetchError stays undefined and the query
+    //      fires without any X-SCM-Tokens header.
+    //   2. Mock GET /api/bulk-import/repositories* with a 200 response so
+    //      Playwright intercepts the token-free request before it ever reaches
+    //      the real backend's 401 guard.
+    //
+    // This lets us focus on UI behaviour without needing a real OAuth provider
+    // set up in the test environment.
+    await mockBulkImportSCMHostsResponse(sharedPage, mockSCMHostsData);
     await mockBulkImportRepositoriesResponse(sharedPage, mockRepositoriesData);
     await sharedPage.goto('/');
 

workspaces/bulk-import/e2e-tests/utils/apiUtils.ts

@@ -19,13 +19,14 @@ import { Page } from '@playwright/test';
  * API route patterns for bulk import endpoints
  */
 export const ApiRoutes = {
+  scmHosts: '**/api/bulk-import/scm-hosts*',
   repositories: '**/api/bulk-import/repositories*',
   importsDryRun: '**/api/bulk-import/imports?dryRun=true*',
   imports: '**/api/bulk-import/imports',
   byRepoBackend:
-    '**/api/bulk-import/import/by-repo?repo=https://github.com/test-org/backend-service*',
+    '**/api/bulk-import/import/by-repo?repo=https%3A%2F%2Fgithub.com%2Ftest-org%2Fbackend-service*',
   byRepoFrontend:
-    '**/api/bulk-import/import/by-repo?repo=https://github.com/test-org/frontend-app*',
+    '**/api/bulk-import/import/by-repo?repo=https%3A%2F%2Fgithub.com%2Ftest-org%2Ffrontend-app*',
 } as const;
 
 type ApiRouteKey = keyof typeof ApiRoutes;
@@ -91,6 +92,12 @@ export const mockBulkImportByRepoFrontendResponse = (
   status = 200,
 ) => mockApiResponse(page, ApiRoutes.byRepoFrontend, responseData, status);
 
+export const mockBulkImportSCMHostsResponse = (
+  page: Page,
+  responseData: object,
+  status = 200,
+) => mockApiResponse(page, ApiRoutes.scmHosts, responseData, status);
+
 // Reusable repository definitions
 const repositories = {
   backendService: {
@@ -135,6 +142,23 @@ const repositories = {
   },
 } as const;
 
+/**
+ * Mock data for SCM hosts response.
+ * Returns empty host lists so the `useRepositories` hook hits the early-return
+ * path (`!urls?.length → return undefined`) and never attempts token collection.
+ * This means `tokenFetchError` stays `undefined`, the query is enabled, and the
+ * frontend fires a request without `X-SCM-Tokens`.
+ *
+ * In production the backend would reject such a request with HTTP 401, but the
+ * Playwright route mock for `ApiRoutes.repositories` intercepts the request
+ * before it reaches the backend, so the e2e tests still receive the mocked
+ * repository data regardless of the missing header.
+ */
+export const mockSCMHostsData = {
+  github: [],
+  gitlab: [],
+};
+
 /** Mock data for repositories list response */
 export const mockRepositoriesData = {
   errors: [],

workspaces/bulk-import/plugins/bulk-import-backend/README.md

@@ -301,13 +301,15 @@ The Bulk Import Backend plugin emits audit events for various operations. Events
 
 - **`ping`**: tracks `GET` requests to the `/ping` endpoint, which allows to make sure the bulk import backend is up and running.
 
-- **`org-read`**: tracks `GET` requests to the `/organizations` endpoint, which returns the list of organizations accessible from all configured GitHub Integrations.
+- **`scm-hosts-read`**: tracks `GET` requests to the `/scm-hosts-read` endpoint, which returns the list of configured GitHub and GitLab integration host URLs.
+
+- **`org-read`**: tracks `GET` requests to the `/organizations` endpoint, which returns the list of organizations accessible from all configured SCM Integrations (GitHub and GitLab).
 
   Filter on `queryType`.
   - **`all`**: tracks fetching all organizations. (GET `/organizations`)
   - **`by-query`**: tracks fetching organization filtered by the query parameter 'search'. (GET `/organizations`)
 
-- **`repo-read`**: tracks `GET` requests to the endpoint, which returns the list of repositories accessible from all configured GitHub Integrations.
+- **`repo-read`**: tracks `GET` requests to the endpoint, which returns the list of repositories accessible from all configured SCM Integrations (GitHub and GitLab).
 
   Filter on `queryType`.
   - **`all`**: tracks fetching a list of all repositories accessible by Backstage Github Integrations. (GET `/repositories`)
@@ -343,8 +345,43 @@ Example:
 
 The bulk import backend plugin provides a REST API to bulk import catalog entities into the catalog. The API is available at the `/api/bulk-import` endpoint.
 
-As a prerequisite, you need to add at least one GitHub Integration (using either a GitHub token or a GitHub App or both) in your app-config YAML file (or a local `app-config.local.yaml` file).
-See https://backstage.io/docs/integrations/github/locations/#configuration and https://backstage.io/docs/integrations/github/github-apps/#including-in-integrations-config for more details.
+As a prerequisite, you need to add at least one SCM integration in your app-config YAML file (or a local `app-config.local.yaml` file):
+
+- **GitHub**: Configure a GitHub integration using a GitHub token or a GitHub App (or both). See the [GitHub Locations](https://backstage.io/docs/integrations/github/locations/#configuration) and [GitHub Apps](https://backstage.io/docs/integrations/github/github-apps/#including-in-integrations-config) documentation for details.
+- **GitLab** _(optional)_: Configure a GitLab integration if you want to import from GitLab repositories. See the [GitLab Locations](https://backstage.io/docs/integrations/gitlab/locations/) documentation for details.
+
+### On Behalf of User Access
+
+The plugin supports fetching repository and organization listings **on behalf of the signed-in user**, using their OAuth credentials rather than the server-wide integration credentials (GitHub App, PAT, or GitLab token).
+
+#### How It Works
+
+1. The frontend calls `GET /api/bulk-import/scm-hosts` to retrieve the list of configured SCM integration host URLs, grouped by provider (`github` and `gitlab`).
+2. For each host, the frontend requests an OAuth token from the Backstage `ScmAuthApi` (provided by `@backstage/integration-react`).
+3. The collected tokens are sent to the backend via the **required** `x-scm-tokens` request header — a JSON-encoded string whose value, when parsed, maps each integration base URL to the user's OAuth token (e.g. `{"https://github.com":"ghp_xxx"}`).
+4. The backend uses these user tokens to call the GitHub or GitLab APIs on behalf of the user, so the repository listings reflect what the signed-in user can personally access.
+
+#### Required OAuth Configuration
+
+The `x-scm-tokens` header is **required** for `GET /repositories` and `GET /organizations/{organizationName}/repositories`. Requests that omit the header, supply an empty token map, or send a header that exceeds the allowed size are rejected with **HTTP 401**.
+
+A GitHub and/or GitLab OAuth provider must therefore be configured in the Backstage application for these endpoints to work. Refer to the [Backstage GitHub auth docs](https://backstage.io/docs/auth/github/provider) and [GitLab auth docs](https://backstage.io/docs/auth/gitlab/provider) for setup instructions.
+
+> **Migration note:** Deployments that previously relied solely on server-side credentials (GitHub App, PAT, or GitLab token) for the repository list view must now also configure an SCM OAuth provider. The server-side credentials are still used for all other operations (import creation, status checks, etc.) and are unaffected by this change.
+
+#### Security Note
+
+When user tokens are provided for GitHub, the Octokit response cache is intentionally disabled to prevent cross-user ETag cache leakage. Server-side credential paths are not affected.
+
+The `x-scm-tokens` header is stripped from the request immediately upon receipt — before the permission check and before any audit event is created — so OAuth token values are never persisted in audit logs.
+
+#### New API Endpoint
+
+| Method | Path                         | Description                                                                                 |
+| ------ | ---------------------------- | ------------------------------------------------------------------------------------------- |
+| `GET`  | `/api/bulk-import/scm-hosts` | Returns configured GitHub and GitLab integration host base URLs as an `SCMHostList` object. |
+
+The existing `GET /repositories` and `GET /organizations/{organizationName}/repositories` endpoints now **require** the `x-scm-tokens` header. See the [API documentation](api-docs/README.md) for the full request/response specification.
 
 ## REST API
 

workspaces/bulk-import/plugins/bulk-import-backend/__fixtures__/handlers.ts

@@ -122,6 +122,10 @@ export const DEFAULT_TEST_HANDLERS: RestHandler<
     );
   }),
 
+  rest.get(`${LOCAL_ADDR}/orgs/my-org-1/repos`, (_, res, ctx) => {
+    return res(ctx.status(200), ctx.json([]));
+  }),
+
   rest.get(`${LOCAL_ADDR}/orgs/my-ent-org-1/repos`, (_, res, ctx) => {
     return res(
       ctx.status(200),

workspaces/bulk-import/plugins/bulk-import-backend/api-docs/.openapi-generator/FILES

@@ -19,6 +19,7 @@ Models/PullRequest.md
 Models/Repository.md
 Models/RepositoryList.md
 Models/Repository_importStatus.md
+Models/SCMHostList.md
 Models/ScaffolderTask.md
 Models/Source.md
 Models/SourceImport.md

workspaces/bulk-import/plugins/bulk-import-backend/api-docs/Apis/ManagementApi.md

@@ -4,9 +4,32 @@ All URIs are relative to *http://localhost:7007/api/bulk-import*
 
 | Method | HTTP request | Description |
 |------------- | ------------- | -------------|
+| [**findAllSCMHosts**](ManagementApi.md#findAllSCMHosts) | **GET** /scm-hosts | Retrieve the SCM Integration hosts |
 | [**ping**](ManagementApi.md#ping) | **GET** /ping | Check the health of the Bulk Import backend router |
 
 
+<a name="findAllSCMHosts"></a>
+# **findAllSCMHosts**
+> SCMHostList findAllSCMHosts()
+
+Retrieve the SCM Integration hosts
+
+### Parameters
+This endpoint does not need any parameter.
+
+### Return type
+
+[**SCMHostList**](../Models/SCMHostList.md)
+
+### Authorization
+
+[BearerAuth](../README.md#BearerAuth)
+
+### HTTP request headers
+
+- **Content-Type**: Not defined
+- **Accept**: application/json
+
 <a name="ping"></a>
 # **ping**
 > ping_200_response ping()

workspaces/bulk-import/plugins/bulk-import-backend/api-docs/Apis/OrganizationApi.md

@@ -38,7 +38,7 @@ Fetch Organizations accessible by Backstage Github Integrations
 
 <a name="findRepositoriesByOrganization"></a>
 # **findRepositoriesByOrganization**
-> RepositoryList findRepositoriesByOrganization(organizationName, checkImportStatus, pagePerIntegration, sizePerIntegration, search, approvalTool)
+> RepositoryList findRepositoriesByOrganization(organizationName, checkImportStatus, pagePerIntegration, sizePerIntegration, search, approvalTool, x-scm-tokens)
 
 Fetch Repositories in the specified GitHub organization, provided it is accessible by any of the configured GitHub Integrations.
 
@@ -52,6 +52,7 @@ Fetch Repositories in the specified GitHub organization, provided it is accessib
 | **sizePerIntegration** | **Integer**| the number of items per Integration to return per page | [optional] [default to 20] |
 | **search** | **String**| returns only the items that match the search string | [optional] [default to null] |
 | **approvalTool** | **String**| the approvalTool to use | [optional] [default to GIT] |
+| **x-scm-tokens** | **String**| **Required.** JSON-encoded map of SCM host URL to user OAuth token. Used to fetch repositories on behalf of the signed-in user. The value must be a JSON object whose keys are SCM integration base URLs and whose values are OAuth bearer tokens (e.g. `{"https://github.com":"ghp_xxx"}`). Requests that omit this header, supply an empty object, or exceed 4 KB are rejected with HTTP 401.  | [required] [default to null] |
 
 ### Return type
 

workspaces/bulk-import/plugins/bulk-import-backend/api-docs/Apis/RepositoryApi.md

@@ -9,7 +9,7 @@ All URIs are relative to *http://localhost:7007/api/bulk-import*
 
 <a name="findAllRepositories"></a>
 # **findAllRepositories**
-> RepositoryList findAllRepositories(checkImportStatus, pagePerIntegration, sizePerIntegration, search, approvalTool)
+> RepositoryList findAllRepositories(checkImportStatus, pagePerIntegration, sizePerIntegration, search, approvalTool, x-scm-tokens)
 
 Fetch Organization Repositories accessible by Backstage Github Integrations
 
@@ -22,6 +22,7 @@ Fetch Organization Repositories accessible by Backstage Github Integrations
 | **sizePerIntegration** | **Integer**| the number of items per Integration to return per page | [optional] [default to 20] |
 | **search** | **String**| returns only the items that match the search string | [optional] [default to null] |
 | **approvalTool** | **String**| the approvalTool to use | [optional] [default to GIT] |
+| **x-scm-tokens** | **String**| **Required.** JSON-encoded map of SCM host URL to user OAuth token. Used to fetch repositories on behalf of the signed-in user. The value must be a JSON object whose keys are SCM integration base URLs and whose values are OAuth bearer tokens (e.g. `{"https://github.com":"ghp_xxx"}`). Requests that omit this header, supply an empty object, or exceed 4 KB are rejected with HTTP 401.  | [required] [default to null] |
 
 ### Return type
 

workspaces/bulk-import/plugins/bulk-import-backend/api-docs/Models/SCMHostList.md

@@ -0,0 +1,10 @@
+# SCMHostList
+## Properties
+
+| Name | Type | Description | Notes |
+|------------ | ------------- | ------------- | -------------|
+| **github** | **List** |  | [optional] [default to null] |
+| **gitlab** | **List** |  | [optional] [default to null] |
+
+[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)
+