|
1 | | -# Test |
| 1 | +# E2E Tests |
2 | 2 |
|
3 | | -## Run e2e tests locally |
| 3 | +## Overview |
4 | 4 |
|
5 | | -To run e2e tests locally, run the following command |
| 5 | +The end-to-end (E2E) test suite validates the full lifecycle of the Cluster API Operator in a real Kubernetes cluster. Tests cover provider creation, upgrade, downgrade, deletion, air-gapped installations, OCI registry support, compressed manifests, and Helm chart rendering. |
| 6 | + |
| 7 | +## Running E2E Tests |
| 8 | + |
| 9 | +### Quick Start (Local) |
6 | 10 |
|
7 | 11 | ```bash |
8 | 12 | make test-e2e-local |
9 | 13 | ``` |
10 | 14 |
|
11 | | -## Compatibility notice |
| 15 | +This creates a local Kind cluster, deploys cert-manager and the operator, and runs the full E2E suite. |
| 16 | + |
| 17 | +### Using an Existing Cluster |
| 18 | + |
| 19 | +```bash |
| 20 | +USE_EXISTING_CLUSTER=true make test-e2e |
| 21 | +``` |
| 22 | + |
| 23 | +### Running Specific Tests |
| 24 | + |
| 25 | +Use Ginkgo's `--focus` flag to run a subset of tests: |
| 26 | + |
| 27 | +```bash |
| 28 | +# Run only air-gapped tests |
| 29 | +make test-e2e GINKGO_ARGS="--focus='air gapped'" |
| 30 | + |
| 31 | +# Run only CoreProvider tests |
| 32 | +make test-e2e GINKGO_ARGS="--focus='CoreProvider'" |
| 33 | +``` |
| 34 | + |
| 35 | +### Skipping Cleanup |
| 36 | + |
| 37 | +For debugging failed tests, set `SKIP_CLEANUP=true` to preserve cluster state: |
| 38 | + |
| 39 | +```bash |
| 40 | +SKIP_CLEANUP=true make test-e2e-local |
| 41 | +``` |
| 42 | + |
| 43 | +## Test Suite Structure |
| 44 | + |
| 45 | +``` |
| 46 | +test/e2e/ |
| 47 | +├── e2e_suite_test.go # Suite setup, Kind cluster management, cert-manager |
| 48 | +├── helpers_test.go # Shared test utilities and helper functions |
| 49 | +├── minimal_configuration_test.go # Core provider lifecycle tests (create/upgrade/delete) |
| 50 | +├── air_gapped_test.go # ConfigMap-based air-gapped installation tests |
| 51 | +├── compressed_manifests_test.go # Large manifest compression via OCI |
| 52 | +├── helm_test.go # Helm chart rendering and golden-file tests |
| 53 | +├── config/ # E2E configuration YAML files |
| 54 | +├── resources/ # Test resource manifests |
| 55 | +└── doc.go # Package documentation |
| 56 | +``` |
| 57 | + |
| 58 | +### Test Files |
| 59 | + |
| 60 | +| File | Tests | Description | |
| 61 | +|------|-------|-------------| |
| 62 | +| `minimal_configuration_test.go` | 11 | Provider create, upgrade, downgrade, delete for all 7 types; OCI fetching; manifest patches | |
| 63 | +| `air_gapped_test.go` | 3 | ConfigMap-based install/upgrade without network access | |
| 64 | +| `compressed_manifests_test.go` | 4 | Large OCI manifests exceeding ConfigMap 1MB limit | |
| 65 | +| `helm_test.go` | 16 | Helm chart install + 15 golden-file template comparison tests | |
| 66 | + |
| 67 | +## Test Framework |
| 68 | + |
| 69 | +The E2E tests use: |
| 70 | + |
| 71 | +- **[Ginkgo v2](https://onsi.github.io/ginkgo/)** — BDD test framework |
| 72 | +- **[Gomega](https://onsi.github.io/gomega/)** — Matcher library with `Eventually`/`Consistently` support |
| 73 | +- **[CAPI test framework](https://pkg.go.dev/sigs.k8s.io/cluster-api/test/framework)** — Kubernetes cluster management utilities |
| 74 | +- **Custom framework** (`test/framework/`) — Operator-specific helpers (`HaveStatusConditionsTrue`, `For().In().ToSatisfy()`) |
| 75 | + |
| 76 | +### Key Patterns |
| 77 | + |
| 78 | +#### Condition Checking |
| 79 | + |
| 80 | +Use the `HaveStatusConditionsTrue` helper to verify provider conditions: |
| 81 | + |
| 82 | +```go |
| 83 | +HaveStatusConditionsTrue( |
| 84 | + provider, |
| 85 | + operatorv1.PreflightCheckCondition, |
| 86 | + operatorv1.ProviderInstalledCondition, |
| 87 | +) |
| 88 | +``` |
| 89 | + |
| 90 | +#### Eventually / Consistently |
| 91 | + |
| 92 | +Always use `Eventually` for async operations (provider creation, deployment readiness) and `Consistently` to assert that a state holds over time: |
| 93 | + |
| 94 | +```go |
| 95 | +// Wait for provider to become ready |
| 96 | +Eventually(func() bool { |
| 97 | + // ... check condition |
| 98 | +}, e2eConfig.GetIntervals(...)...).Should(BeTrue()) |
| 99 | + |
| 100 | +// Verify condition stays true |
| 101 | +Consistently(func() bool { |
| 102 | + // ... check condition |
| 103 | +}, e2eConfig.GetIntervals(...)...).Should(BeTrue()) |
| 104 | +``` |
| 105 | + |
| 106 | +#### Configurable Intervals |
| 107 | + |
| 108 | +Test timeouts and poll intervals are configured in `config/` YAML files, not hard-coded: |
| 109 | + |
| 110 | +```yaml |
| 111 | +intervals: |
| 112 | + default/wait-providers: ["5m", "10s"] |
| 113 | + default/wait-controllers: ["3m", "10s"] |
| 114 | +``` |
| 115 | +
|
| 116 | +Access them with: |
| 117 | +
|
| 118 | +```go |
| 119 | +e2eConfig.GetIntervals("default", "wait-providers") |
| 120 | +``` |
| 121 | + |
| 122 | +## Writing New E2E Tests |
| 123 | + |
| 124 | +### 1. Add a Test File |
| 125 | + |
| 126 | +Create a new file in `test/e2e/` with the `e2e` build tag: |
| 127 | + |
| 128 | +```go |
| 129 | +//go:build e2e |
| 130 | + |
| 131 | +package e2e |
| 132 | + |
| 133 | +import ( |
| 134 | + . "github.com/onsi/ginkgo/v2" |
| 135 | + . "github.com/onsi/gomega" |
| 136 | + operatorv1 "sigs.k8s.io/cluster-api-operator/api/v1alpha2" |
| 137 | + . "sigs.k8s.io/cluster-api-operator/test/framework" |
| 138 | +) |
| 139 | +``` |
| 140 | + |
| 141 | +### 2. Use Ginkgo Containers |
| 142 | + |
| 143 | +Structure tests with `Describe`, `Context`, and `It`: |
| 144 | + |
| 145 | +```go |
| 146 | +var _ = Describe("My Feature", func() { |
| 147 | + It("should do something", func() { |
| 148 | + // Test implementation |
| 149 | + }) |
| 150 | +}) |
| 151 | +``` |
| 152 | + |
| 153 | +For ordered tests that share state, use `Ordered`: |
| 154 | + |
| 155 | +```go |
| 156 | +var _ = Describe("Sequential tests", Ordered, func() { |
| 157 | + It("step 1", func() { /* ... */ }) |
| 158 | + It("step 2", func() { /* ... */ }) |
| 159 | +}) |
| 160 | +``` |
| 161 | + |
| 162 | +### 3. Create Provider Resources |
| 163 | + |
| 164 | +Use the standard pattern from existing tests: |
| 165 | + |
| 166 | +```go |
| 167 | +coreProvider := &operatorv1.CoreProvider{ |
| 168 | + ObjectMeta: metav1.ObjectMeta{ |
| 169 | + Name: "cluster-api", |
| 170 | + Namespace: operatorNamespace, |
| 171 | + }, |
| 172 | + Spec: operatorv1.CoreProviderSpec{ |
| 173 | + ProviderSpec: operatorv1.ProviderSpec{ |
| 174 | + Version: "v1.9.0", |
| 175 | + }, |
| 176 | + }, |
| 177 | +} |
| 178 | + |
| 179 | +Expect(bootstrapClusterProxy.GetClient().Create(ctx, coreProvider)).To(Succeed()) |
| 180 | +``` |
| 181 | + |
| 182 | +### 4. Wait for Conditions |
| 183 | + |
| 184 | +```go |
| 185 | +Eventually( |
| 186 | + For(coreProvider). |
| 187 | + In(bootstrapClusterProxy.GetClient()). |
| 188 | + ToSatisfy( |
| 189 | + HaveStatusConditionsTrue( |
| 190 | + coreProvider, |
| 191 | + operatorv1.PreflightCheckCondition, |
| 192 | + operatorv1.ProviderInstalledCondition, |
| 193 | + ), |
| 194 | + ), |
| 195 | + e2eConfig.GetIntervals("default", "wait-providers")..., |
| 196 | +).Should(BeTrue()) |
| 197 | +``` |
| 198 | + |
| 199 | +### 5. Clean Up Resources |
| 200 | + |
| 201 | +Always clean up after tests to avoid interfering with other specs: |
| 202 | + |
| 203 | +```go |
| 204 | +AfterEach(func() { |
| 205 | + Expect(bootstrapClusterProxy.GetClient().Delete(ctx, coreProvider)).To(Succeed()) |
| 206 | + // Wait for deletion to complete |
| 207 | + Eventually(func() bool { |
| 208 | + err := bootstrapClusterProxy.GetClient().Get(ctx, client.ObjectKeyFromObject(coreProvider), coreProvider) |
| 209 | + return apierrors.IsNotFound(err) |
| 210 | + }, e2eConfig.GetIntervals("default", "wait-providers")...).Should(BeTrue()) |
| 211 | +}) |
| 212 | +``` |
| 213 | + |
| 214 | +### 6. Add Golden Files (Helm Tests) |
| 215 | + |
| 216 | +For Helm template tests, add expected output in `test/e2e/resources/` and compare: |
| 217 | + |
| 218 | +```go |
| 219 | +rendered := helmTemplate(chartPath, releaseName, namespace, values) |
| 220 | +expected := loadGoldenFile("resources/expected-output.yaml") |
| 221 | +Expect(rendered).To(Equal(expected)) |
| 222 | +``` |
| 223 | + |
| 224 | +## Environment Variables |
| 225 | + |
| 226 | +| Variable | Description | Default | |
| 227 | +|----------|-------------|---------| |
| 228 | +| `USE_EXISTING_CLUSTER` | Use existing cluster instead of Kind | `false` | |
| 229 | +| `SKIP_CLEANUP` | Skip resource cleanup after tests | `false` | |
| 230 | +| `E2E_CONFIG_PATH` | Path to E2E config YAML | `test/e2e/config/operator.yaml` | |
| 231 | +| `ARTIFACTS_FOLDER` | Folder for test artifacts/logs | `_artifacts` | |
| 232 | +| `GINKGO_ARGS` | Additional Ginkgo CLI arguments | — | |
| 233 | + |
| 234 | +## Debugging Tips |
| 235 | + |
| 236 | +1. **Preserve cluster state**: Use `SKIP_CLEANUP=true` to keep resources after failure. |
| 237 | +2. **Collect logs**: Artifacts are stored in the `ARTIFACTS_FOLDER` directory including pod logs and cluster state. |
| 238 | +3. **Run focused tests**: Use `--focus` to isolate failing tests. |
| 239 | +4. **Check provider conditions**: When a provider isn't becoming ready, examine its `.status.conditions` for error details. |
| 240 | +5. **Inspect deployments**: Provider components are deployed in the provider's namespace; check controller-manager pod logs. |
| 241 | + |
| 242 | +## Compatibility Notice |
12 | 243 |
|
13 | 244 | This package is not subject to deprecation notices or compatibility guarantees. |
14 | 245 |
|
|
0 commit comments