Skip to content

Return empty primary-key metadata for missing objects in Thrift - #553

Open
vuanhphung wants to merge 1 commit into
mainfrom
fix/thrift-primary-keys-not-found
Open

vuanhphung wants to merge 1 commit into
mainfrom
fix/thrift-primary-keys-not-found

Conversation

@vuanhphung

Copy link
Copy Markdown
Collaborator

When getPrimaryKeys() resolves a missing catalog, schema, or table, Thrift can throw while JDBC and kernel return an empty result set. Apply JDBC's not-found classification only to primary-key metadata, returning a successful empty result with the standard six-column schema for initial, direct-result, and polling failures. Preserve unrelated errors and server-handle cleanup.

Validated with 1,467 passing unit tests (36 new regressions), build, type checking, lint, and formatting. Replayed all 22 cases from https://github.com/databricks/databricks-driver-test/actions/runs/39944116811 using pecotesting in Thrift and kernel modes, with direct results enabled and disabled: all return zero rows with matching column names, types, and positions. Also verified real primary-key rows, omitted-catalog resolution, and ordinary SQL errors.


This PR was created with GitHub MCP.

Signed-off-by: Vu Anh Phung <vu.phung@databricks.com>

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: 1 Low

Looks good — a well-scoped, carefully-tested change that mirrors JDBC/kernel by returning a typed empty primary-key result for missing objects across the initial/direct/poll paths while preserving unrelated errors and handle cleanup. Imports, the no-param-reassign reassignment (rule is off in .eslintrc), and the reordered metadata assignment all check out. One low note inline about the breadth of the INVALID_PARAMETER_VALUE substring match potentially masking genuine invalid-argument errors on getPrimaryKeys.

return (
error.sqlState === '42704' ||
[error.errorMessage, error.displayMessage].some((message) =>
['NO_SUCH_CATALOG_EXCEPTION', 'SCHEMA_NOT_FOUND', 'TABLE_OR_VIEW_NOT_FOUND', 'INVALID_PARAMETER_VALUE'].some(

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Low — isObjectNotFound classifies any error whose errorMessage/displayMessage contains the substring INVALID_PARAMETER_VALUE as "object not found", which is broader than missing catalog/schema/table. For getPrimaryKeys, Spark raises INVALID_PARAMETER_VALUE for several non-missing-object conditions (e.g. a malformed/empty identifier argument). Because this classifier gates the empty-result recovery for primary keys (both in ThriftSessionBackend.getPrimaryKeys and via emptyResultOnNotFound in ThriftOperationBackend.processOperationStatusResponse), such a genuine invalid-argument failure would be silently converted into a successful empty result set instead of surfacing to the caller.

The match is deliberately scoped to primary-key metadata and mirrors JDBC, so this is low severity — but it's worth confirming JDBC actually anchors on the error class (not a loose substring) and, if so, tightening the match (e.g. require it to co-occur with the 42704/42P01 SQLSTATE, or match the bracketed [INVALID_PARAMETER_VALUE] prefix the tests use) so unrelated invalid-parameter errors still propagate.

This branch was successfully deployed

1 active deployment
azure-prod — b76134f4 Deployed Oct 24, 2026 by vuanhphung via e2e-test (20) #1694
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant