diff --git a/CHANGELOG.md b/CHANGELOG.md index 824612a7..8b38cb6d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,11 +8,9 @@ As of v3.x, **niquests** is the supported and recommended package for HTTP commu ### Configuration and "features" -This is "old news", but the 3.4-release may hold a regression making this more relevant (see "Breaking changes" for the 3.4-release below). - When configuring the server it's now possible to pass a `features`-parameter. This may contain a dict describing the server capabilities, but for end-users the intended usage pattern is to pass a server name, e.g. `features: cyrus`. -Support for configuring things like server credentials through a **configuration file** or through **environment** was added a while ago - the idea being that independent applications built using the caldav library can share the same configuration file / environment settings. Use `get_davclient()` rather than `DAVClient()` to support fetching server configuration through a config file or environment. +Support for configuring things like server credentials through a **configuration file** or through **environment** was added a while ago - the idea being that independent applications built using the caldav library can share the same configuration file / environment settings. Use `get_davclient()` rather than `DAVClient()` to support fetching server configuration through a config file or environment. As of 3.6, the config file is read through the [calendaring-config](https://github.com/pycalendar/calendaring-config) package, shared with plann and other tools. ## Meta @@ -26,6 +24,101 @@ After 3.5.0, changes since the last release are kept as fragments in [changes/]( +## [3.6.0] - 2026-10-22 + +I apologize for the 3.6.0 release coming so shortly after the 3.5.0 release; it is fuelled by demands from the [calendaring-sync](https://github.com/pycalendar/calendaring-sync) and [calendaring](https://github.com/pycalendar/calendaring) projects. If you're doing packaging or maintaining dependencies and find bumping the version costly, then wait two weeks - maybe there will be a 3.7.0 release. + +### Overview + +The configuration file is now read through the new [calendaring-config](https://github.com/pycalendar/calendaring-config) package, shared with plann, so the two read the same file the same way - with new features like `password_command`, keyring lookups and per-key environment overrides. `save(no_overwrite=True)` and `save(no_create=True)` now leave the check to the server through wildcard preconditions (`If-None-Match: *`, `If-Match: *`), closing a race with other clients. Calendar objects keep the raw data as the server sent it (`raw_data`), and a single occurrence can be cancelled with `cancel()`. More server profiles have been checked and updated. + +Also released without the niquests dependency as 3.6.0a0 - install requests (for the sync client) and/or httpx2 (for the async client) alongside it. + +### Security + +* Configuration files are now read with PyYAML's safe loader (by calendaring-config). The full loader was used before, so a YAML tag such as `!!python/object/apply:` in a config file could run arbitrary Python code when `get_davclient()` or `get_calendars()` read it - also from the system-wide `/etc/calendar.conf`. A config file using such tags is now refused with a `ValueError`; plain YAML and JSON files are unaffected. https://github.com/python-caldav/caldav/pull/785 +* The new `password_command` key runs a program named in the config file, so a config file is, by design, able to run code. It is guarded where the YAML tags were not: the command is only run if the file it comes from is owned by the user or by root and is not writable by group or others, and it must be given as a list of the program and its arguments, never through a shell. A config file written by someone else, or one others can write to, should not be trusted for the same reason as before. https://github.com/python-caldav/caldav/pull/804 + +### Potentially breaking changes + +These are bug fixes or improvements, but code or configuration relying on the old behaviour may notice them. + +* Most of the config file handling logic was separated out in a new package calendaring-config (see Added). The changes introduced are unlikely to break anything for anyone, and hence considered allowable for the minor release (https://github.com/python-caldav/caldav/issues/786, https://github.com/python-caldav/caldav/pull/804): + * **This introduces a new dependency**. The package has the same maintainer as the caldav library, and the same co-maintainers as the icalendar-searcher library. + * An empty config file found first in the search path is used, rather than skipped for the next one. + * If a disabled or missing section is named, or a glob matches nothing, an error is now raised. + * A name in `contains` that matches nothing gives a warning. + * A disabled meta-section leaves out its members too. + * `*` now excludes meta-sections. + * `inherits` naming a missing section or a parent of another kind (a `caldav_*` section inheriting from a `kind: jmap` one), or a cycle of `inherits`, raises an error (they used to be ignored, merged in, or recurse forever). + * A section no longer inherits its parent's `contains` and `disable`, and `config_section()` leaves `inherits` out of the section it returns. + * A config file named explicitly must exist. +* With a server that does not honour the wildcard preconditions (SOGo, Zimbra, Bedework, GMX, iCloud, Posteo, Radicale, Yahoo), `save(no_overwrite=True)` and `save(no_create=True)` (and `add_event(..., no_overwrite=True)` and friends) may now overwrite an existing object, create a missing one or refuse a legitimate update, unless the server is configured with its `features` (e.g. `features: sogo`), or `save.if-none-match` and `save-load.mutable.if-match-wildcard` are marked correctly in a hand-written feature list. See "Changed" below, https://github.com/python-caldav/caldav/pull/802 +* `DAVClient.password` is now the `str` that was passed in, as on `AsyncDAVClient`; it used to be encoded to UTF-8 `bytes`. The return type of `Calendar.multiget()` is annotated as a list, or a coroutine of one for an async client, like `search()`. https://github.com/python-caldav/caldav/issues/799, https://github.com/python-caldav/caldav/pull/807 + +### Added + +#### Library API + +* `cancel()` marks an object, or a single occurrence from `search(..., expand=True)`, as `STATUS:CANCELLED` and saves it - for an occurrence that was planned but will not take place. A new compatibility feature, `save-load.event.recurrences.exception.cancelled`, records that Zimbra turns a cancelled occurrence into an `EXDATE`. https://github.com/python-caldav/caldav/issues/598, https://github.com/python-caldav/caldav/pull/810 +* Calendar objects keep the calendar data exactly as the server sent it, before `vcal.fix()` or any other normalisation, in the new read-only `raw_data` attribute (`bytes`; `None` for objects not read from a server, copies, and after `save()`). After `load()` it is the GET response body byte for byte; from a REPORT (`search()`, `multiget()`) it is the text of the `calendar-data` element, with XML entities decoded and line breaks turned into LF. `save(raw_data=...)` sends the given (UTF-8) bytes as the PUT body unchanged - no `vcal.fix()`, no re-serialisation, no `SEQUENCE` bump - still with `If-Match` / `If-Schedule-Tag-Match`. This lets sync tools compare and copy what servers store. https://github.com/python-caldav/caldav/issues/763, https://github.com/python-caldav/caldav/pull/803 +* `DAVError` and its subclasses have a new `status` attribute with the HTTP status code of the failed response (or `None`), so an `AuthorizationError` tells a 401 (bad or missing credentials) from a 403 (authenticated, but not allowed). https://github.com/python-caldav/caldav/issues/805, https://github.com/python-caldav/caldav/pull/807 + +#### Configuration + +* The config file is now read through the [calendaring-config](https://github.com/pycalendar/calendaring-config) package, which the file's other users (plann, calendaring) share; see https://github.com/python-caldav/caldav/issues/786 and https://github.com/python-caldav/caldav/pull/804. This is a new dependency, `calendaring-config>=0.5` (it depends only on PyYAML), which matters to packagers. New in the format: + + - `~/.config/pycal/calendar.yaml` is searched first, before the old paths. + - A section may say `kind: caldav` and drop the `caldav_` prefix. Sections of other kinds (`kind: jmap`, a directory of `.ics` files) are skipped. + - `password_command` runs a program for the password, and `password_keyring` looks it up in the system keyring (this needs the `keyring` package, e.g. `pip install calendaring-config[keyring]`). + - `CALDAV_*` environment variables override the section's keys one by one: a `CALDAV_PASSWORD` alone no longer makes caldav ignore the config file (https://github.com/python-caldav/caldav/issues/780). `true`/`false` in a variable is a boolean now, and `CALDAV_HEADERS` is read as JSON. + - `extract_conn_params_from_section()`, which plann uses, reads the new format too; pass it the config file's path as `config_file` to let a `password_command` run, and the section's name as `section_name` for error messages. + - An unknown key in a section is logged as a warning. `x_*` and `extra_config` are reserved for applications. + +#### Server compatibility + +* Compatibility features `save-load.todo.uncomplete` (a completed task can be made pending again with `Todo.uncomplete()`; GMX keeps it completed) and `search.time-range.todo.no-dtstart.no-due` (a date search finds a task with neither DTSTART nor DUE, as RFC 4791 section 9.9 says it should; Zimbra does not). They replace the last two old-style flags in the Zimbra and GMX profiles. https://github.com/python-caldav/caldav/issues/585, https://github.com/python-caldav/caldav/pull/811 +* Several server profiles in `compatibility_hints` are updated from caldav-server-tester runs, so with `features: ` configured, `is_supported()` may give a different answer for the features named here. https://github.com/python-caldav/caldav/pull/781, https://github.com/python-caldav/caldav/pull/812 + + - `calendar-color` (a hex value) and `calendar-order` are measured instead of left at the default `fragile`: both are `full`, and colour names work too, for Xandikos, Nextcloud (and so ecloud, which inherits its profile), CCS, Stalwart, Cyrus, Posteo, Purelymail and Robur; Bedework 3 stores both colour forms but not `calendar-order`; SOGo takes a hex colour but not a colour name; OX App Suite takes a hex colour, ignores colour names, and does not store `calendar-order`. + - Sync tokens: DAViCal, SOGo and Bedework 3 now state `sync-token.invalid-token`, `sync-token.limit` and `sync-token.limit.initial-sync`, and Bedework 5 the two `limit` features, where they used to inherit `fragile` from `sync-token` or default to `full`. DAViCal and SOGo ignore `DAV:limit` and answer a never-issued sync token with a 207 and no error. Bedework 3 refuses a token it never issued, as it should, and answers a `DAV:limit` with a 500. Bedework 5 cuts the reply at `DAV:limit` without the 507 the RFC asks for: following the tokens loses nothing, but a client takes the cut reply as complete and gets the rest only on its next sync. + - OX App Suite: `scheduling` is `full` (it was `unknown`), with `scheduling.schedule-tag` unsupported; inbox delivery and auto-scheduling stay `unknown`. + - Purelymail: `search.is-not-defined`, `search.time-range.event` and `search.time-range.todo` are `full` again (they were `fragile`). + - Stalwart: `search.time-range.todo.no-dtstart.no-due` is `unsupported`. + - GMX: `search.recurrences.includes-implicit.infinite-scope` is `fragile` (it was `full`): it failed in one run and passed in the next. + - Infomaniak: the wait after deleting a calendar (`synchronous-write.delete-calendar`) is raised from 12 to 20 seconds; a deleted calendar could still be there after 12. + - `well-known`: `unsupported` for Posteo, `full` for Robur. + +### Changed + +* `calendar-color` is now about a hex value such as `#FF0000FF`, the format clients use in practice. Setting the colour by name (`blue`) is the new `calendar-color.name`; a server that rejects or ignores colour names is `unsupported` there, not `broken`. Zimbra, SOGo, iCloud and OX App Suite are therefore `full` for `calendar-color` and `unsupported` for `calendar-color.name`. For Synology, Purelymail, GMX and Infomaniak only colour names were measured, so their hex support is assumed. `calendar-color.hex` is deprecated: a configuration setting it still works and gives a DeprecationWarning, and its `calendar-color` is then read as `calendar-color.name`. https://github.com/python-caldav/caldav/pull/813 +* `save(no_overwrite=True)` and `save(no_create=True)` (and `add_event(..., no_overwrite=True)` and friends) now send `If-None-Match: *` and `If-Match: *`, so on an RFC-compliant server the flag is enforced at the same URL in the same request as the write, and another client can no longer slip in between the check and the PUT. Against a server that does not honour the wildcard preconditions, an existing object may be overwritten, a missing one created, or a legitimate update refused (see "Potentially breaking changes" above). Servers known not to honour them, as of this release: + + - `If-None-Match: *`: SOGo, Zimbra + - `If-Match: *`: Bedework, GMX, iCloud, Posteo, Radicale, SOGo, Yahoo, Zimbra + + A refusal still raises `ConsistencyError`. A subclass overriding `_create()` or `_put()` (or their async variants) must accept a new `wildcard` parameter after `precondition`. https://github.com/python-caldav/caldav/pull/802 +* When `save(no_overwrite=True)` or `save(no_create=True)` is refused because another object (at another URL) holds the UID, the error is now the new `UidConflictConsistencyError`, which is both a `UidConflictError` and a `ConsistencyError`, whether the server refused the UID or caldav's own UID search found it. https://github.com/python-caldav/caldav/issues/806, https://github.com/python-caldav/caldav/pull/802, https://github.com/python-caldav/caldav/pull/807 + +### Deprecated + +* Config file keys starting with `test_` are now reserved for test tooling, so other programs sharing the config file can drop them. A section is marked for testing with `test_allowed: true` rather than `testing_allowed: true`, and the order test servers are tried in is set with `test_priority` rather than `priority`. The old names still work, but emit a `DeprecationWarning`. `test_priority` (or `priority`) in a section of the main config file was ignored before; it is now honoured. https://github.com/python-caldav/caldav/issues/786, https://github.com/python-caldav/caldav/pull/804 +* `FeatureSet.copyFeatureSet()` in `caldav.compatibility_hints` has been renamed to `copy_feature_set()`, in line with the naming of the other methods. The old name still works but emits a `DeprecationWarning`. https://github.com/python-caldav/caldav/pull/790 + +### Removed + +* The old-style compatibility flags are gone: `caldav.compatibility_hints.incompatibility_description`, and the `old_flags` key in a feature set. They were only ever used by the test suite to skip or adjust tests, and every flag still in use is now a compatibility feature. https://github.com/python-caldav/caldav/issues/585, https://github.com/python-caldav/caldav/pull/811 + +### Fixed + +* `complete(handle_rrule=True)` on a recurring task with fixed times (`BY*` parts in the `RRULE`, e.g. `FREQ=YEARLY;BYMONTH=4,9`) did not advance the task when it was completed before its `DTSTART`: the next occurrence was looked for after the completion time, which found the occurrence just completed. It is now looked for after `DTSTART`. https://github.com/python-caldav/caldav/issues/782, https://github.com/python-caldav/caldav/pull/808 +* `aio.get_calendars()` ignored meta-sections, `*` and glob patterns in `config_section`, as well as the `calendar_name` and `calendar_url` filters of a config section. It now finds the same calendars as the sync `get_calendars()`, and behaves like it on errors too: it no longer sends an OPTIONS probe first, so an unreachable server raises the error from the principal lookup rather than a `DAVError` saying "Failed to connect", and with `raise_errors=False` such an error is logged and the server skipped rather than raised. https://github.com/python-caldav/caldav/issues/793, https://github.com/python-caldav/caldav/pull/800 +* Creating an `AsyncDAVClient` failed with an `ImportError` when only an httpx-family library was installed. It works now; RFC 6764 discovery still needs niquests or requests, and is skipped with a warning without them. `aio.get_davclient()` and `aio.get_calendars()` blocked the event loop while reading the config file and doing RFC 6764 discovery; they now do both in a worker thread. https://github.com/python-caldav/caldav/issues/794, https://github.com/python-caldav/caldav/pull/800 +* The `supported-report-set` and `current-user-privilege-set` properties from `propfind()` were the raw lxml element. They are now lists of the report and privilege names in Clark notation, e.g. `["{urn:ietf:params:xml:ns:caldav}calendar-query", "{DAV:}sync-collection"]`. Code that walked the returned element has to read the list instead. https://github.com/python-caldav/caldav/issues/795, https://github.com/python-caldav/caldav/pull/807 +* `caldav.testing`: `RadicaleServer` ignored its configured port, so Radicale always listened on 5232 and any other port timed out at startup. `RadicaleServer.stop()` now closes both ends of its shutdown socketpair, and `XandikosServer.stop()` closes its event loop, so neither leaves a `ResourceWarning` behind. https://github.com/python-caldav/caldav/issues/797, https://github.com/python-caldav/caldav/pull/800 +* A `200 OK` reply to the PUT of `save()` is now taken as success (RFC 9110 section 9.3.4 allows it for an overwrite). Before, it was retried, and the retry raised `PutError` when the server answered `200` again. https://github.com/python-caldav/caldav/pull/802 +* A long UID may be folded over several lines, and for an object that was not parsed yet `.id` returned only the first line. As a result `save(no_create=True)` of such an existing object raised `ConsistencyError`, `save(no_overwrite=True)` overwrote it instead of raising, and `save(increase_seqno=False, only_this_recurrence=False)` wrote the cut UID back to the server. `.id` now returns the whole UID, and a UID with a backslash escape such as `\,` is unescaped there too. https://github.com/python-caldav/caldav/pull/787 + ## [3.5.0] - 2026-10-15 Yet another release primarily focusing on bugfixes and compatibility. iCloud has now been tested and got its compatibility profile in `compatibility_hints.py`. Test runs towards GMX, Posteo and Purelymail have been skipped for the last few releases; with 3.5.0 the tests pass towards those. More work on getting synchronization right, and on avoiding overwriting changes when multiple clients update the same object. diff --git a/caldav/config.py b/caldav/config.py index f316f222..eca99920 100644 --- a/caldav/config.py +++ b/caldav/config.py @@ -11,15 +11,21 @@ log = logging.getLogger("caldav") -""" -This configuration parsing code was just copied from my plann library (and will be removed from there at some point in the future). Test coverage is poor as for now. -""" +## The config file format and its parsing belong to calendaring-config. What +## is left here is caldav's own: mapping a section to DAVClient keyword +## arguments, resolving feature profiles, and the order of the sources +## (explicit arguments, test server, environment, file). +## +## TODO: the public expand_config_section, config_section, read_config and +## extract_conn_params_from_section are kept for plann, and should be +## deprecated (with warnings) once plann gets its calendars from +## get_calendars() (https://github.com/PyCalendar/plann/issues/84). +## +## TODO: verify that we have test code for all kinds of corner cases +## +## TODO: verify that the config format is well enough documented -## This is being moved from my plann library. The code itself will be introduced into caldav 2.0, but proper test code and documentation will come in a later release (2.1?) - -## TODO TODO TODO - write test code for all the corner cases -## TODO TODO TODO - write documentation of config format def expand_config_section(config, section="default", blacklist=None): """ The leaf sections that ``section`` expands to, as a list. @@ -43,13 +49,13 @@ def expand_config_section(config, section="default", blacklist=None): def config_section(config, section="default"): - if section in config and "inherits" in config[section]: - ret = config_section(config, config[section]["inherits"]) - else: - ret = {} - if section in config: - ret.update(config[section]) - return ret + """Section ``section`` with what it ``inherits`` copied in, or ``{}`` when + there is no such section. ``inherits`` and the parents' ``contains`` and + ``disable`` are left out; the section's own are kept. The rules are + calendaring-config's ``merged_section``.""" + if section not in config: + return {} + return calendaring_config.merged_section(config, section) def read_config(fn, interactive_error=False): @@ -356,6 +362,9 @@ def _caldav_connections( return [params for params in found if params] +## TODO: the test server selection below, get_test_setting and +## get_all_test_servers are test tooling, not config; move them to +## caldav/testing.py (planned for 3.6.1). def _get_test_server_config( name: str | None, environment: bool, config_file: str | None = None ) -> dict[str, Any] | None: @@ -616,7 +625,12 @@ def get_all_test_servers( result: dict[str, dict[str, Any]] = {} for section_name in cfg: - section_data = config_section(cfg, section_name) + try: + section_data = config_section(cfg, section_name) + except calendaring_config.ConfigError as e: + ## A broken section that is not a test server must not hide the others + log.warning(f"config section {section_name} skipped: {e}") + continue if get_test_setting(section_data, "test_allowed"): conn_params = extract_conn_params_from_section(section_data, str(path), section_name) if conn_params: diff --git a/changes/+calendar-color-name.changed.md b/changes/+calendar-color-name.changed.md deleted file mode 100644 index 9479f280..00000000 --- a/changes/+calendar-color-name.changed.md +++ /dev/null @@ -1 +0,0 @@ -`compatibility_hints`: `calendar-color` is now about a hex value such as `#FF0000FF`, the format clients use in practice. Setting the colour by name (`blue`) is the new `calendar-color.name`; a server that rejects or ignores colour names is `unsupported` there, not `broken`. Zimbra, SOGo, iCloud and OX App Suite are therefore `full` for `calendar-color` and `unsupported` for `calendar-color.name`. For Synology, Purelymail, GMX and Infomaniak only colour names were measured, so their hex support is assumed. `calendar-color.hex` is deprecated: a configuration setting it still works and gives a DeprecationWarning, and its `calendar-color` is then read as `calendar-color.name`. diff --git a/changes/+calendaring-config.changed.md b/changes/+calendaring-config.changed.md deleted file mode 100644 index 1bc8219d..00000000 --- a/changes/+calendaring-config.changed.md +++ /dev/null @@ -1,10 +0,0 @@ -The config file is now read through the [calendaring-config](https://github.com/pycalendar/calendaring-config) package, which the file's other users (plann, calendaring) share; see https://github.com/python-caldav/caldav/issues/786. New in the format: - -- `~/.config/pycal/calendar.yaml` is searched first, before the old paths. -- A section may say `kind: caldav` and drop the `caldav_` prefix. Sections of other kinds (`kind: jmap`, a directory of `.ics` files) are skipped. -- `password_command` runs a program for the password, and `password_keyring` looks it up in the system keyring. -- `CALDAV_*` environment variables override the section's keys one by one: a `CALDAV_PASSWORD` alone no longer makes caldav ignore the config file (https://github.com/python-caldav/caldav/issues/780). `true`/`false` in a variable is a boolean now, and `CALDAV_HEADERS` is read as JSON. -- `extract_conn_params_from_section()`, which plann uses, reads the new format too; pass it the config file's path as `config_file` to let a `password_command` run, and the section's name as `section_name` for error messages. -- An unknown key in a section is logged as a warning. `test_*`, `x_*` and `extra_config` are reserved for test tooling and applications. - -Some files are read differently: an empty file found first is used, rather than skipped for the next one; a section named directly that is missing or disabled, or a glob that matches nothing, is an error; a name in `contains` that matches nothing gives a warning; a disabled meta-section leaves out its members too; `*` is only the sections that are not meta-sections; and a config file named explicitly must exist. diff --git a/changes/+config-safe-yaml.security.md b/changes/+config-safe-yaml.security.md deleted file mode 100644 index f1fbd98a..00000000 --- a/changes/+config-safe-yaml.security.md +++ /dev/null @@ -1 +0,0 @@ -Configuration files are now read with PyYAML's safe loader. The full loader was used before, so a YAML tag such as `!!python/object/apply:` in a config file could run arbitrary Python code when `get_davclient()` or `get_calendars()` read it - also from the system-wide `/etc/calendar.conf`. A config file using such tags is now refused with a `ValueError`; plain YAML and JSON files are unaffected. diff --git a/changes/+copy-feature-set.deprecated.md b/changes/+copy-feature-set.deprecated.md deleted file mode 100644 index 8cdfc4f2..00000000 --- a/changes/+copy-feature-set.deprecated.md +++ /dev/null @@ -1 +0,0 @@ -`FeatureSet.copyFeatureSet()` in `caldav.compatibility_hints` has been renamed to `copy_feature_set()`, in line with the naming of the other methods. The old name still works but emits a `DeprecationWarning`. diff --git a/changes/+folded-long-uid.fixed.md b/changes/+folded-long-uid.fixed.md deleted file mode 100644 index 9adff68f..00000000 --- a/changes/+folded-long-uid.fixed.md +++ /dev/null @@ -1 +0,0 @@ -A long UID may be folded over several lines, and for an object that was not parsed yet `.id` returned only the first line. As a result `save(no_create=True)` of such an existing object raised `ConsistencyError`, `save(no_overwrite=True)` overwrote it instead of raising, and `save(increase_seqno=False, only_this_recurrence=False)` wrote the cut UID back to the server. `.id` now returns the whole UID, and a UID with a backslash escape such as `\,` is unescaped there too. diff --git a/changes/+hints-2026-10-20.changed.md b/changes/+hints-2026-10-20.changed.md deleted file mode 100644 index 1df48c0c..00000000 --- a/changes/+hints-2026-10-20.changed.md +++ /dev/null @@ -1,7 +0,0 @@ -`compatibility_hints`: several server profiles are updated from a caldav-server-tester run, so with `features: ` configured, `is_supported()` may give a different answer for the features named here. - -- `calendar-color` (a hex value) and `calendar-order` are measured instead of left at the default `fragile`: both are `full`, and colour names work too, for Xandikos, Nextcloud (and so ecloud, which inherits its profile), CCS, Stalwart, Cyrus, Posteo and Robur; Bedework 3 stores both colour forms but not `calendar-order`; SOGo takes a hex colour but not a colour name; OX App Suite takes a hex colour, ignores colour names, and does not store `calendar-order`. -- OX App Suite: `scheduling` is `full` (it was `unknown`), with `scheduling.schedule-tag` unsupported; inbox delivery and auto-scheduling stay `unknown`. -- Stalwart: `search.time-range.todo.no-dtstart.no-due` is `unsupported`. -- GMX: `search.recurrences.includes-implicit.infinite-scope` is `fragile` (it was `full`): it failed in one run and passed in the next. -- `well-known`: `unsupported` for Posteo, `full` for Robur. diff --git a/changes/+hints-sync-token-probes.changed.md b/changes/+hints-sync-token-probes.changed.md deleted file mode 100644 index c54987b0..00000000 --- a/changes/+hints-sync-token-probes.changed.md +++ /dev/null @@ -1,5 +0,0 @@ -`compatibility_hints`: several server profiles are updated from a caldav-server-tester run, so with `features: ` configured, `is_supported()` may give a different answer for the features named here. - -- DAViCal, SOGo and Bedework 3 now state `sync-token.invalid-token`, `sync-token.limit` and `sync-token.limit.initial-sync`, and Bedework 5 the two `limit` features, where they used to inherit `fragile` from `sync-token` or default to `full`. DAViCal and SOGo ignore `DAV:limit` and answer a never-issued sync token with a 207 and no error. Bedework 3 refuses a token it never issued, as it should, and answers a `DAV:limit` with a 500. Bedework 5 cuts the reply at `DAV:limit` without the 507 the RFC asks for: following the tokens loses nothing, but a client takes the cut reply as complete and gets the rest only on its next sync. -- Purelymail: `search.is-not-defined`, `search.time-range.event` and `search.time-range.todo` are `full` again (they were `fragile`), and `calendar-color` and `calendar-order` are `full`. -- Infomaniak: the wait after deleting a calendar (`synchronous-write.delete-calendar`) is raised from 12 to 20 seconds; a deleted calendar could still be there after 12. diff --git a/changes/+no-overwrite-if-none-match.changed.md b/changes/+no-overwrite-if-none-match.changed.md deleted file mode 100644 index 7a7f09c3..00000000 --- a/changes/+no-overwrite-if-none-match.changed.md +++ /dev/null @@ -1 +0,0 @@ -`save(no_overwrite=True)` and `save(no_create=True)` (and `add_event(..., no_overwrite=True)` and friends) have improved handling on RFC-compliant servers: the server enforces the flag at the same URL in the same request as the write, so another client can no longer slip in between the check and the PUT there. A refusal still raises `ConsistencyError`. When the refusal is because another object (at another URL) holds the UID, it is the subclass `UidConflictConsistencyError`, whether the server refused the UID or caldav's own UID search found it (https://github.com/python-caldav/caldav/issues/806). diff --git a/changes/+put-200-ok.fixed.md b/changes/+put-200-ok.fixed.md deleted file mode 100644 index 7a0b3af4..00000000 --- a/changes/+put-200-ok.fixed.md +++ /dev/null @@ -1 +0,0 @@ -A `200 OK` reply to the PUT of `save()` is now taken as success (RFC 9110 section 9.3.4 allows it for an overwrite). Before, it was retried, and the retry raised `PutError` when the server answered `200` again. diff --git a/changes/+wildcard-preconditions.breaking.md b/changes/+wildcard-preconditions.breaking.md deleted file mode 100644 index d15b7ef3..00000000 --- a/changes/+wildcard-preconditions.breaking.md +++ /dev/null @@ -1,6 +0,0 @@ -Potentially breaking, on servers that are themselves broken: `save(no_overwrite=True)` and `save(no_create=True)` now rely on the server honouring `If-None-Match: *` and `If-Match: *`, which every CalDAV server is required to do. Against a server that does not, an existing object may be overwritten, a missing one created, or a legitimate update refused. The fix is to tell the client that the server is broken: configure it with the server's `features` (e.g. `features: sogo`), or mark `save.if-none-match` and `save-load.mutable.if-match-wildcard` correctly in a hand-written feature list. Servers known not to honour them, as of this release: - -- `If-None-Match: *`: SOGo, Zimbra -- `If-Match: *`: Bedework, GMX, iCloud, Posteo, Radicale, SOGo, Yahoo, Zimbra - -A subclass overriding `_create()` or `_put()` must accept their new `wildcard=` keyword. diff --git a/changes/585.added.md b/changes/585.added.md deleted file mode 100644 index b94819a5..00000000 --- a/changes/585.added.md +++ /dev/null @@ -1 +0,0 @@ -Compatibility features `save-load.todo.uncomplete` (a completed task can be made pending again with `Todo.uncomplete()`; GMX keeps it completed) and `search.time-range.todo.no-dtstart.no-due` (a date search finds a task with neither DTSTART nor DUE, as RFC 4791 section 9.9 says it should; Zimbra does not). They replace the last two old-style flags in the Zimbra and GMX profiles. diff --git a/changes/585.removed.md b/changes/585.removed.md deleted file mode 100644 index c92da999..00000000 --- a/changes/585.removed.md +++ /dev/null @@ -1 +0,0 @@ -The old-style compatibility flags are gone: `caldav.compatibility_hints.incompatibility_description`, and the `old_flags` key in a feature set. They were only ever used by the test suite to skip or adjust tests, and every flag still in use is now a compatibility feature. diff --git a/changes/598.added.md b/changes/598.added.md deleted file mode 100644 index 9f6f6fd6..00000000 --- a/changes/598.added.md +++ /dev/null @@ -1 +0,0 @@ -`cancel()` marks an object, or a single occurrence from `search(..., expand=True)`, as `STATUS:CANCELLED` and saves it - for an occurrence that was planned but will not take place. A new compatibility feature, `save-load.event.recurrences.exception.cancelled`, records that Zimbra turns a cancelled occurrence into an `EXDATE`. diff --git a/changes/763.added.md b/changes/763.added.md deleted file mode 100644 index 95f4c104..00000000 --- a/changes/763.added.md +++ /dev/null @@ -1 +0,0 @@ -Calendar objects keep the calendar data exactly as the server sent it, before `vcal.fix()` or any other normalisation, in the new read-only `raw_data` attribute (`bytes`; `None` for objects not read from a server, copies, and after `save()`). After `load()` it is the GET response body byte for byte; from a REPORT (`search()`, `multiget()`) it is the text of the `calendar-data` element, with XML entities decoded and line breaks turned into LF. `save(raw_data=...)` sends the given (UTF-8) bytes as the PUT body unchanged - no `vcal.fix()`, no re-serialisation, no `SEQUENCE` bump - still with `If-Match` / `If-Schedule-Tag-Match`. This lets sync tools compare and copy what servers store. diff --git a/changes/782.fixed.md b/changes/782.fixed.md deleted file mode 100644 index ad2be0f2..00000000 --- a/changes/782.fixed.md +++ /dev/null @@ -1 +0,0 @@ -`complete(handle_rrule=True)` on a recurring task with fixed times (`BY*` parts in the `RRULE`, e.g. `FREQ=YEARLY;BYMONTH=4,9`) did not advance the task when it was completed before its `DTSTART`: the next occurrence was looked for after the completion time, which found the occurrence just completed. It is now looked for after `DTSTART`. diff --git a/changes/786.deprecated.md b/changes/786.deprecated.md deleted file mode 100644 index b68245a2..00000000 --- a/changes/786.deprecated.md +++ /dev/null @@ -1 +0,0 @@ -Config file keys starting with `test_` are now reserved for test tooling, so other programs sharing the config file can drop them. A section is marked for testing with `test_allowed: true` rather than `testing_allowed: true`, and the order test servers are tried in is set with `test_priority` rather than `priority`. The old names still work, but emit a `DeprecationWarning`. `test_priority` (or `priority`) in a section of the main config file was ignored before; it is now honoured. diff --git a/changes/793.fixed.md b/changes/793.fixed.md deleted file mode 100644 index c4cac8fe..00000000 --- a/changes/793.fixed.md +++ /dev/null @@ -1 +0,0 @@ -`aio.get_calendars()` ignored meta-sections, `*` and glob patterns in `config_section`, as well as the `calendar_name` and `calendar_url` filters of a config section. It now finds the same calendars as the sync `get_calendars()`, and behaves like it on errors too: it no longer sends an OPTIONS probe first, so an unreachable server raises the error from the principal lookup rather than a `DAVError` saying "Failed to connect", and with `raise_errors=False` such an error is logged and the server skipped rather than raised. diff --git a/changes/794.fixed.md b/changes/794.fixed.md deleted file mode 100644 index c59706d5..00000000 --- a/changes/794.fixed.md +++ /dev/null @@ -1 +0,0 @@ -Creating an `AsyncDAVClient` failed with an `ImportError` when only an httpx-family library was installed. RFC 6764 discovery still needs niquests or requests, and is skipped with a warning without them. `aio.get_davclient()` and `aio.get_calendars()` now read the config file and do RFC 6764 discovery in a worker thread, so they no longer block the event loop. diff --git a/changes/795.fixed.md b/changes/795.fixed.md deleted file mode 100644 index 8f330771..00000000 --- a/changes/795.fixed.md +++ /dev/null @@ -1 +0,0 @@ -The `supported-report-set` and `current-user-privilege-set` properties from `propfind()` were the raw lxml element. They are now lists of the report and privilege names in Clark notation, e.g. `["{urn:ietf:params:xml:ns:caldav}calendar-query", "{DAV:}sync-collection"]`. Code that walked the returned element has to read the list instead. diff --git a/changes/797.fixed.md b/changes/797.fixed.md deleted file mode 100644 index d391d52e..00000000 --- a/changes/797.fixed.md +++ /dev/null @@ -1 +0,0 @@ -`caldav.testing`: `RadicaleServer` ignored its configured port, so Radicale always listened on 5232 and any other port timed out at startup. `RadicaleServer.stop()` now closes both ends of its shutdown socketpair, and `XandikosServer.stop()` closes its event loop, so neither leaves a `ResourceWarning` behind. diff --git a/changes/799.changed.md b/changes/799.changed.md deleted file mode 100644 index 7b470f43..00000000 --- a/changes/799.changed.md +++ /dev/null @@ -1 +0,0 @@ -`DAVClient.password` is now the `str` that was passed in, as on `AsyncDAVClient`; it used to be encoded to UTF-8 `bytes`. The return type of `Calendar.multiget()` is annotated as a list, or a coroutine of one for an async client, like `search()`. diff --git a/changes/805.added.md b/changes/805.added.md deleted file mode 100644 index 81d4d627..00000000 --- a/changes/805.added.md +++ /dev/null @@ -1 +0,0 @@ -`DAVError` and its subclasses have a new `status` attribute with the HTTP status code of the failed response (or `None`), so an `AuthorizationError` tells a 401 (bad or missing credentials) from a 403 (authenticated, but not allowed). diff --git a/docs/source/configfile.rst b/docs/source/configfile.rst index 665094e5..dae2fa13 100644 --- a/docs/source/configfile.rst +++ b/docs/source/configfile.rst @@ -186,7 +186,10 @@ sections share the same server or credentials: calendar_name: Tasks Inheritance is recursive: a section can inherit from a section that itself -inherits from another. +inherits from another. A parent's ``contains`` and ``disable`` are not +inherited. Inheriting from a missing section, a cycle of ``inherits``, or a +parent of another kind (a ``caldav_*`` section inheriting from a +``kind: jmap`` one) is an error. Meta-sections (collections) ============================ diff --git a/pyproject.toml b/pyproject.toml index 642852a4..6be285c5 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -85,7 +85,7 @@ dependencies = [ "dnspython", "python-dateutil", "PyYAML", - "calendaring-config>=0.3", ## reads the config file, see issue 786 + "calendaring-config>=0.5", ## reads the config file, see issue 786 ] dynamic = ["version"] diff --git a/tests/test_caldav_unit.py b/tests/test_caldav_unit.py index ec239f33..334b6153 100755 --- a/tests/test_caldav_unit.py +++ b/tests/test_caldav_unit.py @@ -4550,6 +4550,91 @@ def test_inherits_recursive(self): assert result["caldav_username"] == "alice" assert result["caldav_password"] == "secret" + def test_result_is_a_leaf(self): + """plann hands the result to extract_conn_params_from_section, which + refused it as long as the inherits key was left in""" + from caldav.config import config_section, extract_conn_params_from_section + + config = { + "base": {"caldav_url": "https://dav.example.com/", "contains": ["x"]}, + "default": {"inherits": "base", "caldav_user": "bob"}, + "x": {}, + } + result = config_section(config, "default") + assert "inherits" not in result and "contains" not in result + assert extract_conn_params_from_section(result) == { + "url": "https://dav.example.com/", + "username": "bob", + } + + def test_test_server_with_inherits(self, tmp_path): + import json + + from caldav.config import get_all_test_servers + + config_file = tmp_path / "calendar.conf" + config_file.write_text( + json.dumps( + { + "base": {"caldav_url": "https://dav.example.com/"}, + "srv": {"inherits": "base", "test_allowed": True}, + } + ) + ) + assert get_all_test_servers(str(config_file))["srv"]["url"] == "https://dav.example.com/" + + def test_broken_section_does_not_hide_test_servers(self, tmp_path): + import json + + from caldav.config import get_all_test_servers + + config_file = tmp_path / "calendar.conf" + config_file.write_text( + json.dumps( + { + "srv": {"caldav_url": "https://dav.example.com/", "test_allowed": True}, + "broken": {"inherits": "nosuch"}, + "a": {"inherits": "b"}, + "b": {"inherits": "a"}, + } + ) + ) + assert list(get_all_test_servers(str(config_file))) == ["srv"] + + def test_own_contains_and_disable_are_kept(self): + """only the parents' contains and disable are dropped""" + from caldav.config import config_section + + config = {"meta": {"contains": ["a"], "disable": True}, "a": {}} + assert config_section(config, "meta") == {"contains": ["a"], "disable": True} + + def test_parent_disable_is_not_inherited(self): + from caldav.config import config_section + + config = { + "base": {"caldav_url": "https://dav.example.com/", "disable": True}, + "child": {"inherits": "base"}, + } + assert config_section(config, "child") == {"caldav_url": "https://dav.example.com/"} + + def test_kind_mismatch_is_refused(self): + import calendaring_config + import pytest + + from caldav.config import config_section + + config = { + "jmap": {"kind": "jmap", "url": "https://jmap.example.com/"}, + "child": {"inherits": "jmap", "caldav_url": "https://dav.example.com/"}, + } + with pytest.raises(calendaring_config.ConfigError): + config_section(config, "child") + + def test_missing_section_is_empty(self): + from caldav.config import config_section + + assert config_section({"a": {}}, "nosuch") == {} + class TestExpandEnvVars: """Unit tests for caldav.config.expand_env_vars."""