Skip to content

test: caldav's test servers from caldav.testing, run in parallel - #38

Open
tobixen wants to merge 3 commits into
mainfrom
test/caldav-testing-registry
Open

tobixen wants to merge 3 commits into
mainfrom
test/caldav-testing-registry

Conversation

@tobixen

@tobixen tobixen commented Oct 25, 2026

Copy link
Copy Markdown
Member

Disclaimer: AI-generated code and text below the line, generated by Claude Opus 5.5, based on a wish to run functional tests for the calendaring library.


caldav now ships its test-server registry in caldav.testing (python-caldav/caldav#824). The conftest imports it from there instead of loading tests/test_servers by path from a caldav checkout, and the workarounds for caldav's old config handling are gone.

A run on every caldav test server took many hours, because the servers were tested one after another. Under pytest -n N --dist loadgroup, each server's tests now stay on one worker, so the servers run in parallel. --caldav-servers-config=PATH adds the servers of a test-server yaml kept outside the caldav checkout. A warning filter lets the test servers configured without TLS verification (caldav's Zimbra) run under filterwarnings = error.

Tests only: the default suite passes serially and under -n 8 --dist loadgroup (2817 passed, 433 skipped, 4 xfailed). The first run on all servers turned up many divergences, which are left for roadmap 3.2.

🤖 Generated with Claude Code

caldav now ships its test-server registry in caldav.testing, so the
conftest imports it instead of loading tests/test_servers by path from
a caldav checkout, and CALDAV_SOURCE is gone.  The registry also merges
all its config sources, finds the checkout's yaml whatever the working
directory, and probes external servers itself, so the conftest's
workarounds for those go too.

python-caldav/caldav#817
python-caldav/caldav#824

Prompt: Now that we have #817 done on the local caldav master (please confirm), please fix the calendaring library to use this, and run the integration tests towards all the docker servers plus all the external servers defined under [a caldav checkout holding the test servers' passwords] [#817: python-caldav/caldav#817]

Assisted-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Reviewed-by: Tobias Brox <tobias@redpill-linpro.com>
caldav's Zimbra test server is configured with ssl_verify_cert: false,
and urllib3's "Unverified HTTPS request" warning then failed every
Zimbra test under filterwarnings = error.  The filter matches on the
message, so it needs no urllib3 import.

Prompt: Now that we have #817 done on the local caldav master (please confirm), please fix the calendaring library to use this, and run the integration tests towards all the docker servers plus all the external servers defined under [a caldav checkout holding the test servers' passwords] [#817: python-caldav/caldav#817; this fixes what the Zimbra run turned up]

Assisted-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Reviewed-by: Tobias Brox <tobias@redpill-linpro.com>
A run on every caldav test server took hours, most of them on the
slowest server.  Under pytest -n N --dist loadgroup each server's
tests now stay on one worker, so the servers run in parallel and keep
one connection and one session lock each; pytest-xdist joins the dev
extra.  --caldav-servers-config adds the servers of a test-server yaml
outside the caldav checkout, its entries winning on a name clash.

Prompt: The biggest problem is the purelymail runs [due to test sleeps due to purelymail having an async search cache], a single run can easily take several hoursl   If we cannot do that with pytest-xdist, then it's needed to do aome similar lgoic as in [a personal script that runs caldav's tests in parallel, one tmux window per slow server]
Followup-Prompt: Do the needful ensuring all tests are run towardes all servers - if ecloud failed on the last run, then it's a good reason to retry, [(sic)]

Assisted-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Reviewed-by: Tobias Brox <tobias@redpill-linpro.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant