Skip to content

Scope asset expressions to the assets the caller may read - #75180

Open
henry3260 wants to merge 1 commit into
apache:mainfrom
henry3260:redact-asset-expression-across-api
Open

henry3260 wants to merge 1 commit into
apache:mainfrom
henry3260:redact-asset-expression-across-api

Conversation

@henry3260

@henry3260 henry3260 commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Why

next_run_assets scopes the asset scheduling expression it serves to the caller: an asset leaf whose
id the caller may not read keeps its place in the boolean tree, so the shape of the schedule stays
honest, but its uri, name and id are blanked and hidden is set (#72864). The UI dependencies
graph followed the same rule (#73008).

Four endpoints still served DagModel.asset_expression verbatim:

Endpoint Guard Exposure
GET /api/v2/dags/{dag_id}/details Dag GET only one Dag, reachable with curl — no UI needed
GET /ui/dags Dag GET only every readable Dag in one response
GET /ui/partitioned_dag_runs asset GET, all-or-nothing every Dag in the page
GET /ui/pending_partitioned_dag_run/{dag_id} asset + Dag GET one Dag

Dag access and asset access are separate resources in the auth manager (is_authorized_dag vs
is_authorized_asset, with filter_authorized_assets as the batch API), and Dag source code is a
separate access entity again (DagAccessEntity.CODE), so a caller holding Dag GET is not entitled to
the identity of the assets that Dag waits on. Asset names and uris commonly encode bucket, table or
dataset names, which is why the rest of the asset API already scopes them.

Reachability is the same as for the endpoints already fixed: the FAB auth manager returns every asset
id and the simple auth manager grants all or nothing above its lowest role, so nothing is redacted
there. Deployments whose auth manager authorizes assets individually are the ones affected.

What

  • routes/public/dags.py — get_dag_details takes ReadableAssetsFilterDep and redacts the
    expression on the response. The redaction deliberately does not touch dag_model: that is a live
    ORM instance whose session commits when the request ends, so assigning the blanked leaves onto it
    would persist them.
  • datamodels/common.py — new validate_asset_expression. FastAPI does not validate the nested
    models of an already-built response, so an expression rewritten after model_validate has to be
    validated explicitly; without it the rewritten tree bypasses MaybeAssetExpression and a legacy
    stored shape would reach the client verbatim instead of as null.
  • routes/ui/dags.py — the Dag-list builder redacts each Dag's expression. The readable-asset id set
    is resolved once per request, not per Dag.
  • routes/ui/partitioned_dag_runs.py — both endpoints redact (the collection's per-Dag
    asset_expressions map and the detail's single asset_expression).
  • Tests — one per endpoint, each asserting a readable leaf survives intact and an unreadable one comes
    back blanked with hidden. All four fail against main. The Dag details test also asserts the
    stored expression is unchanged after the request, which is what pins the ORM caveat above. Two
    query-count assertions in the partitioned tests move up by the one AssetModel.id select the new
    dependency adds, and the comment enumerating them is updated.

related: #72864, #73008


Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code (Opus 5)

`next_run_assets` scopes the asset scheduling expression it serves to the
assets the caller may read (apache#72864), and the dependencies graph followed
(apache#73008). Four endpoints still served `DagModel.asset_expression` verbatim:
the public Dag details endpoint, the UI Dag list, and both partitioned Dag
run endpoints. A caller authorized on a Dag but not on its upstream assets
could read every name and uri out of them, and the public endpoint needs no
UI to reach. Asset names and uris commonly encode bucket, table or dataset
names, which is why the rest of the asset API already scopes them.

The sibling `assets` array of the pending partitioned Dag run response
carries the same names; scoping that is a separate change with its own UI
consumers.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:API Airflow's REST/HTTP API

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant