Repository navigation
Discover Dags in sub-folders of packaged (zip) Dag bundles - #71703
Closed
piotrlinski wants to merge 9 commits into
Closed
piotrlinski wants to merge 9 commits into
piotrlinski wants to merge 9 commits into
Conversation
Only .py/.pyc files at the root of a zip archive were imported; a Dag file placed in a sub-folder was silently skipped, giving no clue why it never appeared. Zip archives are a supported way to ship a Dag together with its vendored packages, and organising Dags into sub-packages inside the archive is a natural layout that previously did not work. A member in a sub-folder is now imported when that folder — and every folder above it — is an importable package (contains an __init__.py), which is what zipimport requires to reach it. Parent packages are imported before their sub-modules so autoregistered Dags are attributed to the right module, and stale modules the archive owns are evicted from sys.modules first so members resolve against this archive. A Dag file in a non-package folder cannot be imported from a zip, so instead of being dropped silently it is skipped and a warning names the unreachable file and how to fix it.
piotrlinski
requested review from
ephraimbuddy and
jedcunningham
as code owners
August 17, 2026 08:41
Member
|
Closing this. #73853 removed |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Only .py/.pyc files at the root of a zip archive were imported; a Dag file placed in a sub-folder was silently skipped, giving no clue why it never appeared. Zip archives are a supported way to ship a Dag together with its vendored packages, and organising Dags into sub-packages inside the archive is a natural layout that previously did not work.
A member in a sub-folder is now imported when that folder — and every folder above it — is an importable package (contains an
__init__.py), which is what zipimport requires to reach it. Parent packages are imported before their sub-modules so autoregistered Dags are attributed to the right module, and stale modules the archive owns are evicted from sys.modules first so members resolve against this archive. A Dag file in a non-package folder cannot be imported from a zip, so instead of being dropped silently it is skipped and a warning names the unreachable file and how to fix it.Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Opus 4.8) following the guidelines
{pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.