Skip to content

feat(appkit): introduce modular SDK wrapper (PoC)#414

Draft
MarioCadenas wants to merge 1 commit into
mainfrom
poc/modular-sdk-migration
Draft

feat(appkit): introduce modular SDK wrapper (PoC)#414
MarioCadenas wants to merge 1 commit into
mainfrom
poc/modular-sdk-migration

Conversation

@MarioCadenas
Copy link
Copy Markdown
Collaborator

Wrap @databricks/sdk-experimental behind an AppKit-internal facade (packages/appkit/src/workspace-client/) and wire the modular Databricks SDK packages where viable.

Migrated service clients:

  • wrapper.files → @databricks/sdk-files (camelCase fields, mapping adapter at the connector boundary for AppKit's public types)
  • wrapper.warehouses → @databricks/sdk-warehouses
  • wrapper.vectorSearch → @databricks/sdk-vectorsearch
  • wrapper.http → @databricks/sdk-core/http + @databricks/sdk-auth (replaces apiClient.request for SCIM probe, serving SSE, internal telemetry; native AbortSignal — no Context bridge)

Delegated to legacy SDK via wrapper.toLegacyWorkspaceClient() with TODO(prod) markers:

  • wrapper.genie / wrapper.jobs (modular packages exist but the connectors need a non-trivial rewrite — method renames + waiter idiom + camelCase fields)
  • wrapper.statementExecution / wrapper.servingEndpoints / wrapper.currentUser (no modular package yet)
  • .toLegacyWorkspaceClient() bridge for @databricks/lakebase

Unified error handling — isApiError(e) / getApiErrorStatusCode(e) helpers work across both old-SDK and modular ApiError shapes (modular SDK throws @databricks/sdk-core/apierror; old SDK throws @databricks/sdk-experimental ApiError; instanceof can't unify them).

Upstream packaging-bug workarounds in patches/@databricks__sdk-*.patch (wired via pnpm-workspace.yaml patchedDependencies):

  • Add .js extensions to extensionless ESM imports across dist/
  • Add "default" condition to package.json exports (CJS-resolver support, needed by dev-playground's tsx)
  • Fix parseResponse to handle HTTP 204 empty bodies (was throwing "Error: [object Object]" on every void-response endpoint)

Incidental AppKit bug fix surfaced during testing — skipBodyParsing path matching used exact-string Set lookup against route templates (/api/files/:volumeKey/upload), which never matched concrete URLs (/api/files/files/upload), so every file upload bombed in body-parser. Replaced with compiled-regex matching.

Tests: 2591 passed | 51 skipped. Skipped suite is
connectors/files/tests/client.test.ts — heavy legacy-shape mocks need a rewrite against modular FilesClient methods (TODO(prod)).

Verified end-to-end against staging:

  • dev-playground boots, 42 routes, agents load
  • listDirectoryContentsIter, downloadFile, getFileMetadata, createDirectory, deleteDirectory all hit Databricks successfully
  • listWarehouses returns real warehouse list
  • Modular ApiError preserved through cache layer (no more 500 masking of real 404s)

Wrap @databricks/sdk-experimental behind an AppKit-internal facade
(packages/appkit/src/workspace-client/) and wire the modular Databricks
SDK packages where viable.

Migrated service clients:
  - wrapper.files → @databricks/sdk-files (camelCase fields, mapping
    adapter at the connector boundary for AppKit's public types)
  - wrapper.warehouses → @databricks/sdk-warehouses
  - wrapper.vectorSearch → @databricks/sdk-vectorsearch
  - wrapper.http → @databricks/sdk-core/http + @databricks/sdk-auth
    (replaces apiClient.request for SCIM probe, serving SSE,
    internal telemetry; native AbortSignal — no Context bridge)

Delegated to legacy SDK via wrapper.toLegacyWorkspaceClient() with
TODO(prod) markers:
  - wrapper.genie / wrapper.jobs (modular packages exist but the
    connectors need a non-trivial rewrite — method renames + waiter
    idiom + camelCase fields)
  - wrapper.statementExecution / wrapper.servingEndpoints /
    wrapper.currentUser (no modular package yet)
  - .toLegacyWorkspaceClient() bridge for @databricks/lakebase

Unified error handling — isApiError(e) / getApiErrorStatusCode(e)
helpers work across both old-SDK and modular ApiError shapes
(modular SDK throws @databricks/sdk-core/apierror; old SDK throws
@databricks/sdk-experimental ApiError; instanceof can't unify them).

Upstream packaging-bug workarounds in patches/@databricks__sdk-*.patch
(wired via pnpm-workspace.yaml patchedDependencies):
  - Add .js extensions to extensionless ESM imports across dist/
  - Add "default" condition to package.json exports (CJS-resolver
    support, needed by dev-playground's tsx)
  - Fix parseResponse to handle HTTP 204 empty bodies (was throwing
    "Error: [object Object]" on every void-response endpoint)

Incidental AppKit bug fix surfaced during testing — skipBodyParsing
path matching used exact-string Set lookup against route templates
(/api/files/:volumeKey/upload), which never matched concrete URLs
(/api/files/files/upload), so every file upload bombed in body-parser.
Replaced with compiled-regex matching.

Tests: 2591 passed | 51 skipped. Skipped suite is
connectors/files/tests/client.test.ts — heavy legacy-shape mocks need
a rewrite against modular FilesClient methods (TODO(prod)).

Verified end-to-end against staging:
  - dev-playground boots, 42 routes, agents load
  - listDirectoryContentsIter, downloadFile, getFileMetadata,
    createDirectory, deleteDirectory all hit Databricks successfully
  - listWarehouses returns real warehouse list
  - Modular ApiError preserved through cache layer (no more 500
    masking of real 404s)

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant