Repository navigation
Roadmap 3.2: readable scenarios, and caldav's test servers - #27
Merged
Merged
Conversation
A vdir collection made for fewer components (recorded in .calendaring-components) refused the others when adding, but its capabilities still declared component.task and the rest FULL. The conformance suite now checks the declaration of a collection made for tasks only, which caught it. Prompt: Yes, make them apply to files and memory as well [the readable scenario tests; running them on the files backend exposed this] Assisted-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Reviewed-by: Tobias Brox <tobias@redpill-linpro.com>
tests/test_scenarios.py tells one readable story per test, on the memory backend (three profiles), the files backend (three layouts) and CalDAV; where a backend lacks a feature, it checks the refusal instead. `--caldav-servers=Baikal,...` (or `all`) adds the servers of caldav's test framework, from a caldav checkout, without ever starting them, and the CalDAV conformance suite runs once per server (roadmap 3.2). Prompt: (...) I want integration tests for the caldav wrapping, piggy-backing the test framework in caldav to actually reach test servers. While it shouldn't serve as documentation, it would be nice to make the tests readable and nice, making it relatively easy to understand what one is trying to achieve while reading the test. So in this task readability matters more than test completeness and code deduplication. Prompts (comments made inline in the document while reviewing it): > (on the plan's overlap with roadmap 3.2) Ok > (on loading caldav's test framework from a source checkout) Can this work out in the github CI? In any case, as a start, it's sufficient that I can run the tests locally from my laptop. I'm not much keen on releasing caldav 3.7.0 now, rather immediately after 3.6.0 was done. > (on the tests being slow and opt-in) It's not an option to start and stop the servers for every test. The proper thing to do would be to check if the docker container is up and running and skip the test or abort with an error if it isn't. On github a set of fast/important servers should be taken up before the tests are run. It's no point being equally rigid with testing "as many servers as possible" in the calendaring-library as in caldav; that's caldav's job. However, it may be important to check both servers supporting and not supporting the more important capabilities defined in the calendaring library. And I would like to have the possibility to manually run the test suite towards selected servers, like I do with the caldav library before every release. > (on the two test files) ok. > (at the end of the plan) go! > (on the review finding that five code branches for "no such capability" will never run on any CalDAV server) [they should be left in place, the same tests should be run towards any backend server] > (on the review finding that `all` reaches external accounts) "all" should include all servers in the caldav test registry. This includes docker servers, external servers configured in tests/caldav_test_servers.yaml and servers in the regular config file tagged up with test_allowed=True > (on the second review's finding that caldav reads only its first config source) Do it the easiest way, just make sure to make it possible for me to manually test it towards all test servers I have available, and make sure caldav issue #817 would make it possible to do things properly in the future. [python-caldav/caldav#817] Assisted-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Reviewed-by: Tobias Brox <tobias@redpill-linpro.com>
test_time_spent_in_a_meeting compared the meeting as sent with the meeting as stored, so it failed on SOGo, which adds CLASS:PUBLIC to everything it stores. It now compares the stored meeting before and after log_time(), which is what the test is about. Prompt: (Claude ran tests and made a list of SOGo failures): This should be fixed, not filed. Assisted-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Reviewed-by: Tobias Brox <tobias@redpill-linpro.com>
A server that spells out an RFC 5545 parameter default, as SOGo does with PARTSTAT=NEEDS-ACTION on each ATTENDEE, has changed nothing, but verify=True reported it. Parameters are now compared in any order and without those at their default; RSVP=TRUE, which SOGo also adds, is a real change and is still reported. Prompt: (Claude ran tests and made a list of SOGo failures): This should be fixed, not filed. Assisted-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Reviewed-by: Tobias Brox <tobias@redpill-linpro.com>
create_collection() raised nothing when the collection it made could hold none of the requested components: a tracker made an empty one, and Bedework makes events-only calendars whatever is asked. Such a collection is now refused, and deleted again where it can be; one that cannot be is named in the error. Where caldav knows the server ignores the requested component set (SOGo, Bedework 5, GMX), create-collection is declared LOSSY, and the conformance suite accepts a calendar holding more than asked, or the refusal. Prompt: (Claude ran tests and made a list of SOGo failures): This should be fixed, not filed. Assisted-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Reviewed-by: Tobias Brox <tobias@redpill-linpro.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Disclaimer: The code and the text below the line are AI-generated, by Claude Opus 5.5. The prompts are in the commit messages.
This PR does the first box of roadmap 3.2, plus the fixes it turned up.
tests/test_scenarios.pyhas readable user stories, one per test, in the sync mode. Examples: add a task, tick it off, and change one's mind; a sync client asks for changes since a token; a stale copy can't overwrite a newer one..icsfile), and CalDAV.pytest --caldav-servers=Baikal,SOGo(orall) adds the docker and external servers of caldav's test framework.allit is skipped.CLASS:PUBLIC. It now compares the stored meeting before and afterlog_time().verifyignores parameters at their RFC 5545 default, and parameter order. SOGo spells outPARTSTAT=NEEDS-ACTION. TheRSVP=TRUEit also adds is still reported, as a real change.create_collection()refuses a collection that would hold none of the requested components. Where one was made anyway, it is deleted again (Bedework makes events-only calendars whatever is asked). Where caldav says the server ignores the component set (SOGo, Bedework 5, GMX),create-collectionis declaredLOSSY.Results so far: the scenarios pass on Baikal and SOGo. The conformance suite passes on Baikal, and fails 4 tests on SOGo. Each of those waits on caldav: a new hint, or a check for an existing object at the URL in
save(no_overwrite=True).🤖 Generated with Claude Code