Add --extra-yaml flag to cci flow run/info and task run/info#3969
Open
Add --extra-yaml flag to cci flow run/info and task run/info#3969
Conversation
simple_salesforce has tightened its __all__ exports, so reading Salesforce/OrderedDict via the top-level namespace now trips pyright's reportPrivateImportUsage. Switch to the canonical submodule imports (simple_salesforce.api) and a setattr() monkey-patch that pyright doesn't flag. Also pin the pyright version in the pre-commit hook so node-side version drift can't silently reintroduce the same class of failure.
Introduces a pure function that reads one or more paths (from the --extra-yaml flag or CUMULUSCI_EXTRA_YAML env var) and returns the concatenated YAML content as a single string, suitable for passing as BaseProjectConfig's additional_yaml kwarg. Implements the motivating use case from #3725 (thanks @jlantz) with a different API: flag name, multi-file support, env var fallback, and proper Click wiring are all different.
Adds a method that rebuilds project_config with an additional_yaml kwarg and re-binds the keychain. Needed because CliRuntime is constructed in cci.main() before Click has parsed subcommand options, so flags like --extra-yaml cannot be applied at construction time.
Click option accepts multiple paths and honors CUMULUSCI_EXTRA_YAML as a fallback. The resolved YAML string is passed to CliRuntime.reload_project_config before flow resolution.
cci task run uses a custom RunTaskCommand MultiCommand. It must resolve --extra-yaml before get_task() runs. If it does not, any task defined only in the extra YAML is invisible to the runtime. RunTaskCommand.resolve_command reads --extra-yaml from the raw args list and calls CliRuntime.reload_project_config with the merged YAML. Both --extra-yaml PATH and --extra-yaml=PATH are accepted. The peek happens in resolve_command, not in get_command. Click's MultiCommand protocol calls resolve_command(ctx, args), which then calls get_command(ctx, cmd_name). By the time get_command runs, ctx.args is empty. The remaining args stay in the args parameter passed to resolve_command. We override resolve_command so that we can see --extra-yaml while the project config is still mutable. Adds "extra_yaml" to the specials list in core/tasks.py, alongside debug_before, debug_after, and no_prompt. Click passes extra_yaml as a kwarg on the task subcommand, which merges into task_config.config["options"]. Tasks that use the Pydantic Options form with extra='forbid' would reject it. The specials list already exists to handle exactly this case.
Exercises the full path from resolve_extra_yaml through CliRuntime.reload_project_config into BaseProjectConfig.get_task: - Option override preserves sibling keys (deep merge) - Multi-file last-wins ordering - New task definition via --extra-yaml is resolvable - class_path override imports the replacement class (documents the trust posture rather than preventing it)
Intentionally skipping docs/history.md — release notes there are auto-generated from PR titles between latest-start/latest-stop markers.
Splitting the env var on ":" broke on Windows where drive letters contain colons (e.g. C:\tmp\a.yml). Switch to comma, the convention used everywhere else in CCI for list-valued strings, via process_list_arg. Replace os.path.isfile/open with pathlib.Path. Update all CLI help strings, docstring, and docs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
--extra-yaml PATH(repeatable) tocci flow run,cci flow info,cci task run, andcci task info. It merges extra YAML into theproject config for one command. Also reads
CUMULUSCI_EXTRA_YAML(acomma-separated env var) as a fallback. The flag and env var are
opt-in; nothing changes for users who do not set them.
Supersedes #3725. Closes #3429.
Precedence (highest to lowest)
-o taskname__option value--extra-yamlfiles (last wins)CUMULUSCI_EXTRA_YAMLenv varcumulusci.ymlcumulusci.ymlcumulusci.ymlcumulusci.ymlSecurity
Extra YAML can redefine any
class_path, which runs arbitrary Pythonwhen the task starts. This is the same risk as
cumulusci.yml, butthe flag makes it easier to pass untrusted input. A stderr warning
prints on every use.
Incidental change
The first commit pins
pyrightin.pre-commit-config.yamland fixestwo pre-existing
reportPrivateImportUsageerrors fromsimple_salesforcetightening its re-exports. Without this, thepyright hook fails on
main. 4 lines across 3 files. Happy to splitinto a separate PR.
Test plan
flag overrides env, missing file, empty env segments
CliRuntime.reload_project_config: applies, rebinds keychain,no-ops on None, wraps ConfigError
--extra-yaml, including=PATHformwins, new task is resolvable, class_path override imports new
class
-owins over--extra-yamlfor the same optionDocs
docs/cli.md: new "The--extra-yamlFlag" subsectiondocs/config.md: new per-invocation scope section, updatedprecedence list
docs/env-var-reference.md:CUMULUSCI_EXTRA_YAMLentryFollow-up after merge
Cascading-Configuration-Fileswiki page to include--extra-yamlin the cascade diagram.