Skip to content

Commit 0979fdf

Browse files
author
alexander-demicev
committed
Add more details to e2e readme
Signed-off-by: alexander-demicev <alexandr.demicev@suse.com>
1 parent 26b08b4 commit 0979fdf

File tree

1 file changed

+235
-4
lines changed

1 file changed

+235
-4
lines changed

test/e2e/README.md

Lines changed: 235 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,245 @@
1-
# Test
1+
# E2E Tests
22

3-
## Run e2e tests locally
3+
## Overview
44

5-
To run e2e tests locally, run the following command
5+
The end-to-end (E2E) test suite validates the full lifecycle of the Cluster API Operator in a real Kubernetes cluster. Tests cover provider creation, upgrade, downgrade, deletion, air-gapped installations, OCI registry support, compressed manifests, and Helm chart rendering.
6+
7+
## Running E2E Tests
8+
9+
### Quick Start (Local)
610

711
```bash
812
make test-e2e-local
913
```
1014

11-
## Compatibility notice
15+
This creates a local Kind cluster, deploys cert-manager and the operator, and runs the full E2E suite.
16+
17+
### Using an Existing Cluster
18+
19+
```bash
20+
USE_EXISTING_CLUSTER=true make test-e2e
21+
```
22+
23+
### Running Specific Tests
24+
25+
Use Ginkgo's `--focus` flag to run a subset of tests:
26+
27+
```bash
28+
# Run only air-gapped tests
29+
make test-e2e GINKGO_ARGS="--focus='air gapped'"
30+
31+
# Run only CoreProvider tests
32+
make test-e2e GINKGO_ARGS="--focus='CoreProvider'"
33+
```
34+
35+
### Skipping Cleanup
36+
37+
For debugging failed tests, set `SKIP_CLEANUP=true` to preserve cluster state:
38+
39+
```bash
40+
SKIP_CLEANUP=true make test-e2e-local
41+
```
42+
43+
## Test Suite Structure
44+
45+
```
46+
test/e2e/
47+
├── e2e_suite_test.go # Suite setup, Kind cluster management, cert-manager
48+
├── helpers_test.go # Shared test utilities and helper functions
49+
├── minimal_configuration_test.go # Core provider lifecycle tests (create/upgrade/delete)
50+
├── air_gapped_test.go # ConfigMap-based air-gapped installation tests
51+
├── compressed_manifests_test.go # Large manifest compression via OCI
52+
├── helm_test.go # Helm chart rendering and golden-file tests
53+
├── config/ # E2E configuration YAML files
54+
├── resources/ # Test resource manifests
55+
└── doc.go # Package documentation
56+
```
57+
58+
### Test Files
59+
60+
| File | Tests | Description |
61+
|------|-------|-------------|
62+
| `minimal_configuration_test.go` | 11 | Provider create, upgrade, downgrade, delete for all 7 types; OCI fetching; manifest patches |
63+
| `air_gapped_test.go` | 3 | ConfigMap-based install/upgrade without network access |
64+
| `compressed_manifests_test.go` | 4 | Large OCI manifests exceeding ConfigMap 1MB limit |
65+
| `helm_test.go` | 16 | Helm chart install + 15 golden-file template comparison tests |
66+
67+
## Test Framework
68+
69+
The E2E tests use:
70+
71+
- **[Ginkgo v2](https://onsi.github.io/ginkgo/)** — BDD test framework
72+
- **[Gomega](https://onsi.github.io/gomega/)** — Matcher library with `Eventually`/`Consistently` support
73+
- **[CAPI test framework](https://pkg.go.dev/sigs.k8s.io/cluster-api/test/framework)** — Kubernetes cluster management utilities
74+
- **Custom framework** (`test/framework/`) — Operator-specific helpers (`HaveStatusConditionsTrue`, `For().In().ToSatisfy()`)
75+
76+
### Key Patterns
77+
78+
#### Condition Checking
79+
80+
Use the `HaveStatusConditionsTrue` helper to verify provider conditions:
81+
82+
```go
83+
HaveStatusConditionsTrue(
84+
provider,
85+
operatorv1.PreflightCheckCondition,
86+
operatorv1.ProviderInstalledCondition,
87+
)
88+
```
89+
90+
#### Eventually / Consistently
91+
92+
Always use `Eventually` for async operations (provider creation, deployment readiness) and `Consistently` to assert that a state holds over time:
93+
94+
```go
95+
// Wait for provider to become ready
96+
Eventually(func() bool {
97+
// ... check condition
98+
}, e2eConfig.GetIntervals(...)...).Should(BeTrue())
99+
100+
// Verify condition stays true
101+
Consistently(func() bool {
102+
// ... check condition
103+
}, e2eConfig.GetIntervals(...)...).Should(BeTrue())
104+
```
105+
106+
#### Configurable Intervals
107+
108+
Test timeouts and poll intervals are configured in `config/` YAML files, not hard-coded:
109+
110+
```yaml
111+
intervals:
112+
default/wait-providers: ["5m", "10s"]
113+
default/wait-controllers: ["3m", "10s"]
114+
```
115+
116+
Access them with:
117+
118+
```go
119+
e2eConfig.GetIntervals("default", "wait-providers")
120+
```
121+
122+
## Writing New E2E Tests
123+
124+
### 1. Add a Test File
125+
126+
Create a new file in `test/e2e/` with the `e2e` build tag:
127+
128+
```go
129+
//go:build e2e
130+
131+
package e2e
132+
133+
import (
134+
. "github.com/onsi/ginkgo/v2"
135+
. "github.com/onsi/gomega"
136+
operatorv1 "sigs.k8s.io/cluster-api-operator/api/v1alpha2"
137+
. "sigs.k8s.io/cluster-api-operator/test/framework"
138+
)
139+
```
140+
141+
### 2. Use Ginkgo Containers
142+
143+
Structure tests with `Describe`, `Context`, and `It`:
144+
145+
```go
146+
var _ = Describe("My Feature", func() {
147+
It("should do something", func() {
148+
// Test implementation
149+
})
150+
})
151+
```
152+
153+
For ordered tests that share state, use `Ordered`:
154+
155+
```go
156+
var _ = Describe("Sequential tests", Ordered, func() {
157+
It("step 1", func() { /* ... */ })
158+
It("step 2", func() { /* ... */ })
159+
})
160+
```
161+
162+
### 3. Create Provider Resources
163+
164+
Use the standard pattern from existing tests:
165+
166+
```go
167+
coreProvider := &operatorv1.CoreProvider{
168+
ObjectMeta: metav1.ObjectMeta{
169+
Name: "cluster-api",
170+
Namespace: operatorNamespace,
171+
},
172+
Spec: operatorv1.CoreProviderSpec{
173+
ProviderSpec: operatorv1.ProviderSpec{
174+
Version: "v1.9.0",
175+
},
176+
},
177+
}
178+
179+
Expect(bootstrapClusterProxy.GetClient().Create(ctx, coreProvider)).To(Succeed())
180+
```
181+
182+
### 4. Wait for Conditions
183+
184+
```go
185+
Eventually(
186+
For(coreProvider).
187+
In(bootstrapClusterProxy.GetClient()).
188+
ToSatisfy(
189+
HaveStatusConditionsTrue(
190+
coreProvider,
191+
operatorv1.PreflightCheckCondition,
192+
operatorv1.ProviderInstalledCondition,
193+
),
194+
),
195+
e2eConfig.GetIntervals("default", "wait-providers")...,
196+
).Should(BeTrue())
197+
```
198+
199+
### 5. Clean Up Resources
200+
201+
Always clean up after tests to avoid interfering with other specs:
202+
203+
```go
204+
AfterEach(func() {
205+
Expect(bootstrapClusterProxy.GetClient().Delete(ctx, coreProvider)).To(Succeed())
206+
// Wait for deletion to complete
207+
Eventually(func() bool {
208+
err := bootstrapClusterProxy.GetClient().Get(ctx, client.ObjectKeyFromObject(coreProvider), coreProvider)
209+
return apierrors.IsNotFound(err)
210+
}, e2eConfig.GetIntervals("default", "wait-providers")...).Should(BeTrue())
211+
})
212+
```
213+
214+
### 6. Add Golden Files (Helm Tests)
215+
216+
For Helm template tests, add expected output in `test/e2e/resources/` and compare:
217+
218+
```go
219+
rendered := helmTemplate(chartPath, releaseName, namespace, values)
220+
expected := loadGoldenFile("resources/expected-output.yaml")
221+
Expect(rendered).To(Equal(expected))
222+
```
223+
224+
## Environment Variables
225+
226+
| Variable | Description | Default |
227+
|----------|-------------|---------|
228+
| `USE_EXISTING_CLUSTER` | Use existing cluster instead of Kind | `false` |
229+
| `SKIP_CLEANUP` | Skip resource cleanup after tests | `false` |
230+
| `E2E_CONFIG_PATH` | Path to E2E config YAML | `test/e2e/config/operator.yaml` |
231+
| `ARTIFACTS_FOLDER` | Folder for test artifacts/logs | `_artifacts` |
232+
| `GINKGO_ARGS` | Additional Ginkgo CLI arguments ||
233+
234+
## Debugging Tips
235+
236+
1. **Preserve cluster state**: Use `SKIP_CLEANUP=true` to keep resources after failure.
237+
2. **Collect logs**: Artifacts are stored in the `ARTIFACTS_FOLDER` directory including pod logs and cluster state.
238+
3. **Run focused tests**: Use `--focus` to isolate failing tests.
239+
4. **Check provider conditions**: When a provider isn't becoming ready, examine its `.status.conditions` for error details.
240+
5. **Inspect deployments**: Provider components are deployed in the provider's namespace; check controller-manager pod logs.
241+
242+
## Compatibility Notice
12243

13244
This package is not subject to deprecation notices or compatibility guarantees.
14245

0 commit comments

Comments
 (0)