Skip to content

feat: Add Pydantic V2 support with V1 compatibility layer#3924

Merged
aditya-balachander merged 5 commits intomainfrom
pydantic-v2-production
Oct 13, 2025
Merged

feat: Add Pydantic V2 support with V1 compatibility layer#3924
aditya-balachander merged 5 commits intomainfrom
pydantic-v2-production

Conversation

@jstvz
Copy link
Copy Markdown
Contributor

@jstvz jstvz commented Oct 1, 2025

Summary

This PR adds Pydantic V2 support to CumulusCI using the V1 compatibility layer. This enables working with Pydantic V2 ecosystem while maintaining stability.

Approach: V1 Compatibility Layer

Instead of converting all models to V2 syntax, we use Pydantic's V1 compatibility namespace (pydantic.v1). This approach minimizes breaking changes and allows gradual migration. FastAPI and other projects use this same approach for complex migrations.

Changes Made

  • Dependencies: Pydantic 1.10.22 → 2.11.9, point Snowfakery to migrate_to_pydantic_v2 branch
  • Core modules: Replace from pydantic import with from pydantic.v1 import across 17 files
  • Test compatibility: Fix validation imports and exception handling for V1 namespace

All existing functionality is preserved. All tests pass with the compatibility layer.

Update dependencies for Pydantic V2 migration:
- Pydantic: 1.10.22 → 2.11.9 (major version upgrade)
- Snowfakery: Point to migrate_to_pydantic_v2 branch
- Add hatch direct references support for Git dependencies

This enables Pydantic V2 features while maintaining compatibility.
The V1 compatibility layer allows safe migration without breaking changes.

Major frameworks like FastAPI use this approach successfully.
Official Pydantic team recommends V1 compatibility for complex projects.
Implement systematic V1 compatibility across core modules:
- Replace 'from pydantic import' with 'from pydantic.v1 import'
- Update core configuration, task execution, and OAuth components
- Maintain all existing functionality without breaking changes

Updated modules:
- Core: Configuration models, dependency resolution, version handling
- Tasks: Metadata ETL, bulk data operations, package management
- OAuth: Authentication and client models
- Utils: YAML parsing, options processing, parallel execution

This approach is used by FastAPI and other major Python projects.
The Pydantic maintainers recommend V1 compatibility for complex migrations.
Enables gradual migration with minimal performance impact.
Fix validation error handling and field imports in tests:
- Update ValidationError imports to use pydantic.v1 namespace
- Fix Field imports in model parser tests
- Update pydantic.Extra usage for V1 compatibility
- Resolve exception type mismatches in test suite

Updated components:
- Test files: Model validation, dependency tests, transform tests
- Core files: Task validation, project config, deploy tasks

All tests are now passing with V1 compatibility layer.
Copy link
Copy Markdown
Contributor

@aditya-balachander aditya-balachander left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @jstvz,

Went through all references of pydantic and you have ensured that all references have now been shifted to use pydantic.v1 import. Everything looks good!

There is one linting error: umulusci/utils/yaml/cumulusci_yml.py:283:5: F824 global has_shown_yaml_error_message is unused: name is never assigned in scope which you can check and remove that line.

Additionally regarding the test failures in test_select_utils.py, they are passing on 3.12 but not on 3.11 and 3.13. And when I tested in local with 3.11 and 3.13, they seem to pass. I dont exactly know what the reason for that is.

Approving, so that you can proceed.

@jstvz jstvz marked this pull request as ready for review October 7, 2025 20:15
@jstvz jstvz requested a review from a team as a code owner October 7, 2025 20:15
jstvz added 2 commits October 9, 2025 22:10
Remove the global declaration for 'has_shown_yaml_error_message'.
This variable was deleted in a previous change, but the declaration
was left behind by mistake. This fixes flake8 error F824.
Copy link
Copy Markdown
Contributor

@aditya-balachander aditya-balachander left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @jstvz,

Looks like there is a bug in Annoy package version 1.17.3 with Python 3.11 which is why the optional tests are failing. Other tests pass. So approving

@aditya-balachander aditya-balachander merged commit d4405e3 into main Oct 13, 2025
40 of 47 checks passed
@aditya-balachander aditya-balachander deleted the pydantic-v2-production branch October 13, 2025 04:23
rupeshjSFDC pushed a commit to jorgesolebur/CumulusCI that referenced this pull request Nov 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants