Repository navigation
Conversation
|
|
github-actions
Bot
added
Component: C++
Component: FlightRPC
awaiting review
4 tasks
lidavidm
reviewed
Oct 21, 2026
Comment on lines
+852
to
+857
| /// Retrieves a UTF-8 string containing a Substrait dialect: a YAML | ||
| /// document, conforming to the Substrait dialect schema, that lists the | ||
| /// types, relations, expressions and function signatures the server | ||
| /// accepts in a Substrait plan. Should only be returned if | ||
| /// FLIGHT_SQL_SERVER_SUBSTRAIT is true. Its absence means that the server | ||
| /// does not describe which part of Substrait it accepts. |
Member
There was a problem hiding this comment.
Can we just link to the Substrait docs?
| * A dialect can be tens of kilobytes. A client that retrieves it repeatedly | ||
| * can retrieve SQL_SUBSTRAIT_DIALECT_HASH first and only retrieve this value | ||
| * when the hash has changed. | ||
| */ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rationale for this change
Flight SQL can already report whether a server accepts Substrait plans and which versions it accepts (
FLIGHT_SQL_SERVER_SUBSTRAIT,_MIN_VERSION,_MAX_VERSION). It can't report which part of Substrait the server accepts. In practice no consumer implements the whole spec, so a producer only learns that a plan is unsupported when execution fails.Substrait now has a standard format for this: dialect files. A dialect is a YAML document that lists the types, relations, expressions and function signatures a system supports. substrait-java already publishes one for Spark (spark_dialect.yaml).
What changes are included in this PR?
Two new
SqlInfocodes inFlightSql.proto, mirrored in the C++SqlInfoOptions::SqlInfoenum:SQL_SUBSTRAIT_DIALECT = 577: a UTF-8 string holding a Substrait dialect YAML document. Should only be returned ifFLIGHT_SQL_SERVER_SUBSTRAITis true. If it isn't returned, the server doesn't describe which parts of Substrait it accepts; that doesn't mean it accepts all of Substrait.SQL_SUBSTRAIT_DIALECT_HASH = 578: an opaque string identifying the dialect's content, for client-side caching. It must change whenever the dialect changes (e.g. a hex-encoded SHA-256 digest of the dialect). Clients compare it for equality only.The hash code exists because of size. A dialect covering every function in the standard Substrait extensions plus all types, relations and expressions is about 34 KiB of YAML (about 3.6 KiB gzipped). Real dialects are subsets of that. This is fine to fetch once per connection, but chatty clients can fetch the hash first and only refetch the dialect when the hash changes.
Both codes are in the [500, 1000) range with the other SQL syntax codes, since they describe the syntax the server accepts. Because ADBC reuses Flight SQL's info codes, ADBC gets the same codes without a separate change.
Are these changes tested?
No. The change only adds enum values and documentation; no behavior changes.
Are there any user-facing changes?
Yes. Two new
SqlInfocodes are available to Flight SQL servers and clients. The change is backwards-compatible: no existing codes or types change.