Skip to content

fix(loaders): handle self-referencing types to avoid infinite recursion#122

Open
Mxiansen wants to merge 1 commit into
TencentCloud:masterfrom
Mxiansen:fix/loaders-self-ref-infinite-recursion
Open

fix(loaders): handle self-referencing types to avoid infinite recursion#122
Mxiansen wants to merge 1 commit into
TencentCloud:masterfrom
Mxiansen:fix/loaders-self-ref-infinite-recursion

Conversation

@Mxiansen

@Mxiansen Mxiansen commented Jun 18, 2026

Copy link
Copy Markdown

Related TAPD: https://tapd.woa.com/tapd_fe/10161711/story/detail/1010161711126433671

Background

Some APIs (e.g. billing/CreateGatherRule, billing/CreateAllocationRule, billing/ModifyAllocationRule) declare self-referencing request types, typically AllocationRuleExpression.Children: list<AllocationRuleExpression>.

The original expansion logic in tccli/loaders.py (_get_param_info, _generate_param_skeleton, _get_unfold_param_info) walked the type graph without any cycle detection or depth bound, recursing forever and ending with RecursionError, which broke:

  • tccli <svc> <action> help
  • tccli <svc> <action> --generate-cli-skeleton
  • tccli <svc> <action> --cli-unfold-argument ...

After the initial fix landed (path-level visited + max_depth=20 truncation), users still couldn't pass any flat --Key.Path.Below.TruncationPoint value arguments under --cli-unfold-argument: every such token was rejected as "Unknown options", and the only escape was --cli-input-json file://.... This update extends the fix so that legitimate drill-down ≤ 30 segments past the truncation point goes through, while keeping the safety net for genuinely oversized inputs.

优化内容

  • _get_param_info: introduce path-level visited frozenset + max_depth=20 fallback; on cycle hit, the node degenerates to a placeholder leaf (members becomes the type-name string/list).
  • _generate_param_skeleton: same protection; emits "RecursiveRef<TypeName>" placeholder so users know to pass the field as a whole JSON via --cli-input-json.
  • _get_unfold_param_info / _recur_get_unfold_param_info: same protection; registers current path as a leaf when a cycle is hit.
  • _filling_unfold_param_info: detects truncated leaves (members is a placeholder string/list rather than dict), marks them type=Object and appends a JSON-input hint to the document.
  • visited is path-scoped (visited | {member} derives a new frozenset on each descent), so sibling fields sharing the same plain struct are not mistakenly truncated.
  • tccli/loaders.py: introduce MAX_INPUT_DEPTH = 30 and a shared RECURSIVE_HINT_FILE_OPTION constant. Simplify the truncation hint to a single --cli-input-json file://... alternative; the previous "(1) Drop --cli-unfold-argument" wording is removed by request.
  • tccli/document_handler.py (_json_format / _unfold_complex_object / _complex_object_doc): adapt the new contract — when members is a type-name string (truncated placeholder) instead of a dict, render it as <RecursiveRef:TypeName> and stop recursion, so help --detail no longer raises TypeError on self-referencing APIs.
  • tccli/cli_unfold_argument.py (build_action_parameters): accept an optional extra_unfold_args dict; when provided, those flat key/value pairs are fed through the same convert_to_dict + handle_array pipeline as the argparse Namespace, so the request body has one consistent shape. With extra_unfold_args=None (or empty) the behaviour is byte-identical to before.
  • tccli/command.py (ActionCommand.__call__): under --cli-unfold-argument, after argparse leaves a remaining list, walk it through the new _extract_deep_nested_args helper.
  • tccli/command.py (_extract_deep_nested_args + _resolve_inner_leaf_type + _match_truncated_prefix + _collect_truncated_prefixes): match each --k v / --k=v token against the longest recursive-truncated leaf prefix; for keys with flat depth ≤ MAX_INPUT_DEPTH forward them into extra_unfold_args, auto-wrapping a scalar value into [scalar] when the schema's leaf type is list (looked up via service_model.objects schema walk); for keys with flat depth > MAX_INPUT_DEPTH record them into oversized_tokens. The scan is driven by the function's input remaining (not sys.argv), so the helper can be exercised directly through ac(args, parsed_globals) from unit tests. An orphan-value drift fallback covers the case where argparse reorders consecutive unknown options and parks a value at the tail.
  • tccli/command.py (_build_recursive_hint / _build_oversized_hint): emit two distinct hint blocks — one for tokens that hit a truncation prefix but cannot be expanded (self-referencing context), one for tokens that exceed MAX_INPUT_DEPTH=30 — both pointing users to --cli-input-json file://<path/to/request.json> with the explicit file:// prefix requirement.

Compatibility

  • For non-recursive APIs and for self-referencing APIs that don't drill below the truncation point, output is byte-identical before and after this patch (verified against cvm/RunInstances, cvm/DescribeInstances, billing/DescribeBillSummary, etc.).
  • For self-referencing APIs that do drill below, requests with flat depth ≤ 30 now go through end-to-end; depth > 30 is rejected at the CLI layer with a targeted hint pointing at --cli-input-json file://.
  • Both --key value and --key=value argparse forms are supported on the deep-nesting path; the latter is kept as a compatibility path.

Internal MR: https://git.woa.com/tencentcloud-internal/tencentcloud-cli/merge_requests/18

@Mxiansen Mxiansen force-pushed the fix/loaders-self-ref-infinite-recursion branch from b198a1d to 34d3ae8 Compare June 24, 2026 06:59
Mxiansen added a commit to Mxiansen/tencentcloud-cli-intl-en that referenced this pull request Jun 24, 2026
Some APIs (e.g. billing/CreateGatherRule, billing/CreateAllocationRule,
billing/ModifyAllocationRule) declare self-referencing request types,
typically AllocationRuleExpression.Children: list<AllocationRuleExpression>.

The current expansion logic in tccli/loaders.py walks the type graph
without any cycle detection or depth bound, recursing forever and ending
with RecursionError, breaking:
- tccli <svc> <action> help
- tccli <svc> <action> --generate-cli-skeleton
- tccli <svc> <action> --cli-unfold-argument ...

Fix: introduce a path-level visited frozenset plus max_depth=20 fallback
on all three DFS expansion paths. When a cycle is hit (or depth is
exceeded), the node is rendered as a placeholder leaf instead of
recursing further. visited is path-scoped (visited | {member}), so
sibling fields sharing a normal struct are not mistakenly truncated.

In addition, when a self-referencing leaf is reached the truncated node
now carries stable flags (recursive_truncated / recursive_type) and an
extended document hint mentioning --cli-input-json and
--generate-cli-skeleton. tccli/command.py uses these flags to detect the
case where users keep drilling into a self-referencing path with
--cli-unfold-argument (e.g. --RuleList.RuleDetail.Children.0.RuleValue),
and prints a targeted hint pointing at the offending option, the
self-referencing type, and the recommended JSON-based usages, instead of
only raising the generic "Unknown options" error.

Same fix as TencentCloud/tencentcloud-cli#122.

Related TAPD: https://tapd.woa.com/tapd_fe/10161711/story/detail/1010161711126433671
Internal MR: https://git.woa.com/tencentcloud-internal/tencentcloud-cli/merge_requests/18
@Mxiansen Mxiansen force-pushed the fix/loaders-self-ref-infinite-recursion branch from 34d3ae8 to fb367f9 Compare June 24, 2026 07:48
Mxiansen added a commit to Mxiansen/tencentcloud-cli-intl-en that referenced this pull request Jun 24, 2026
Some APIs (e.g. billing/CreateGatherRule, billing/CreateAllocationRule,
billing/ModifyAllocationRule) declare self-referencing request types,
typically AllocationRuleExpression.Children: list<AllocationRuleExpression>.

The current expansion logic in tccli/loaders.py walks the type graph
without any cycle detection or depth bound, recursing forever and ending
with RecursionError, breaking:
- tccli <svc> <action> help
- tccli <svc> <action> --generate-cli-skeleton
- tccli <svc> <action> --cli-unfold-argument ...

Fix: introduce a path-level visited frozenset plus max_depth=20 fallback
on all three DFS expansion paths. When a cycle is hit (or depth is
exceeded), the node is rendered as a placeholder leaf instead of
recursing further. visited is path-scoped (visited | {member}), so
sibling fields sharing a normal struct are not mistakenly truncated.

In addition, when a self-referencing leaf is reached the truncated node
now carries stable flags (recursive_truncated / recursive_type) and an
extended document hint mentioning --cli-input-json and
--generate-cli-skeleton. tccli/command.py uses these flags to detect the
case where users keep drilling into a self-referencing path with
--cli-unfold-argument (e.g. --RuleList.RuleDetail.Children.0.RuleValue),
and prints a targeted hint pointing at the offending option, the
self-referencing type, and the recommended JSON-based usages, instead of
only raising the generic "Unknown options" error.

Same fix as TencentCloud/tencentcloud-cli#122.

Related TAPD: https://tapd.woa.com/tapd_fe/10161711/story/detail/1010161711126433671
Internal MR: https://git.woa.com/tencentcloud-internal/tencentcloud-cli/merge_requests/18
@Mxiansen Mxiansen force-pushed the fix/loaders-self-ref-infinite-recursion branch from fb367f9 to ac5ca5e Compare June 24, 2026 12:02
Mxiansen added a commit to Mxiansen/tencentcloud-cli-intl-en that referenced this pull request Jun 24, 2026
Some APIs (e.g. billing/CreateGatherRule, billing/CreateAllocationRule,
billing/ModifyAllocationRule) declare self-referencing request types,
typically AllocationRuleExpression.Children: list<AllocationRuleExpression>.

The current expansion logic in tccli/loaders.py walks the type graph
without any cycle detection or depth bound, recursing forever and ending
with RecursionError, breaking:
- tccli <svc> <action> help
- tccli <svc> <action> --generate-cli-skeleton
- tccli <svc> <action> --cli-unfold-argument ...

Fix: introduce a path-level visited frozenset plus max_depth=20 fallback
on all three DFS expansion paths. When a cycle is hit (or depth is
exceeded), the node is rendered as a placeholder leaf instead of
recursing further. visited is path-scoped (visited | {member}), so
sibling fields sharing a normal struct are not mistakenly truncated.

In addition, when a self-referencing leaf is reached the truncated node
now carries stable flags (recursive_truncated / recursive_type) and an
extended document hint mentioning --cli-input-json and
--generate-cli-skeleton. tccli/command.py uses these flags to detect the
case where users keep drilling into a self-referencing path with
--cli-unfold-argument (e.g. --RuleList.RuleDetail.Children.0.RuleValue),
and prints a targeted hint pointing at the offending option, the
self-referencing type, and the recommended JSON-based usages, instead of
only raising the generic "Unknown options" error.

Same fix as TencentCloud/tencentcloud-cli#122.

Related TAPD: https://tapd.woa.com/tapd_fe/10161711/story/detail/1010161711126433671
Internal MR: https://git.woa.com/tencentcloud-internal/tencentcloud-cli/merge_requests/18
@Mxiansen Mxiansen force-pushed the fix/loaders-self-ref-infinite-recursion branch 3 times, most recently from 677dafe to 5776ddb Compare June 26, 2026 08:04
Some APIs (e.g. billing/CreateGatherRule, billing/CreateAllocationRule,
billing/ModifyAllocationRule) declare self-referencing request types,
typically AllocationRuleExpression.Children: list<AllocationRuleExpression>.

The original expansion logic in tccli/loaders.py walked the type graph
without any cycle detection or depth bound, recursing forever and ending
with RecursionError, breaking:
- tccli <svc> <action> help
- tccli <svc> <action> --generate-cli-skeleton
- tccli <svc> <action> --cli-unfold-argument ...

Core fix:
- Introduce a path-level visited frozenset plus max_depth=20 fallback on
  all three DFS expansion paths in tccli/loaders.py. When a cycle is hit
  (or depth is exceeded), the node is rendered as a placeholder leaf
  instead of recursing further. visited is path-scoped (visited | {member}),
  so sibling fields sharing a normal struct are not mistakenly truncated.
- tccli/document_handler.py adapts to the new contract: when members is
  a type-name string (truncated placeholder) instead of a dict, render
  it as <RecursiveRef:TypeName> and stop recursion, so help --detail no
  longer raises TypeError on self-referencing APIs.

Beyond the truncation safety net, this change makes --cli-unfold-argument
actually usable for users who legitimately need to drill into a
self-referencing type for a few extra levels:

- tccli/loaders.py: introduce MAX_INPUT_DEPTH=30 and a shared
  RECURSIVE_HINT_FILE_OPTION constant. Simplify the truncation hint to
  a single --cli-input-json file://... alternative.

- tccli/cli_unfold_argument.py: build_action_parameters() accepts an
  optional extra_unfold_args dict; when provided, those flat key/value
  pairs are fed through the same convert_to_dict + handle_array pipeline
  as the argparse Namespace, so the request body has one consistent
  shape. With extra_unfold_args=None the behaviour is byte-identical.

- tccli/command.py: ActionCommand gains _extract_deep_nested_args /
  _resolve_inner_leaf_type / _build_oversized_hint and supporting
  helpers. Under --cli-unfold-argument, after argparse leaves a
  remaining list, walk it (--k v / --k=v, list nargs='*'), match
  the longest recursive-truncated prefix, and:
    * forward keys with depth <= MAX_INPUT_DEPTH into extra_unfold_args,
      auto-wrapping a scalar into [scalar] when the schema's leaf type
      is list (looked up via service_model.objects walk);
    * record keys whose flat depth exceeds MAX_INPUT_DEPTH into
      oversized_tokens, surfaced through a dedicated hint that points
      at --cli-input-json file://.
  The scan is driven by the function's input remaining (not sys.argv),
  so unit tests can exercise it directly through ac(args, globals).
  An orphan-value drift fallback covers the case where argparse
  reorders consecutive unknown options and parks a value at the tail.

Compatibility:
- For non-recursive APIs and for self-referencing APIs that don't drill
  below the truncation point, output is byte-identical before/after
  this patch (verified against cvm/RunInstances, cvm/DescribeInstances,
  billing/DescribeBillSummary).
- For self-referencing APIs that DO drill below, requests with flat
  depth <= 30 now go through; depth > 30 is rejected at CLI layer with
  a targeted hint pointing at --cli-input-json file://.
- Both --key value and --key=value argparse forms are supported.

Related TAPD: https://tapd.woa.com/tapd_fe/10161711/story/detail/1010161711126433671
Internal MR: https://git.woa.com/tencentcloud-internal/tencentcloud-cli/merge_requests/18
@Mxiansen Mxiansen force-pushed the fix/loaders-self-ref-infinite-recursion branch from 5776ddb to 9b9c6ba Compare June 26, 2026 08:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant