Skip to content

[AIT-30] LiveObjects Path-based API spec (pre-squash)#427

Closed
VeskeR wants to merge 43 commits into
integration/liveobjects-path-based-apifrom
AIT-30/liveobjects-path-based-api-spec
Closed

[AIT-30] LiveObjects Path-based API spec (pre-squash)#427
VeskeR wants to merge 43 commits into
integration/liveobjects-path-based-apifrom
AIT-30/liveobjects-path-based-api-spec

Conversation

@VeskeR
Copy link
Copy Markdown
Contributor

@VeskeR VeskeR commented Feb 24, 2026

Note: This PR is based on #470; please review that one first.

Resolves AIT-30.

@VeskeR VeskeR force-pushed the AIT-30/liveobjects-path-based-api-spec branch from 13dee45 to 1e518c6 Compare February 24, 2026 13:46
@VeskeR VeskeR force-pushed the AIT-30/liveobjects-path-based-api-spec branch from 1fa3eeb to 46261f4 Compare February 24, 2026 15:52
@VeskeR VeskeR force-pushed the AIT-313/protocol-v6-state-message branch 3 times, most recently from 49f0364 to 47a9d51 Compare February 27, 2026 15:52
Base automatically changed from AIT-313/protocol-v6-state-message to main February 27, 2026 15:53
@ttypic ttypic force-pushed the AIT-30/liveobjects-path-based-api-spec branch from 46261f4 to 3608895 Compare March 9, 2026 10:54
@github-actions github-actions Bot temporarily deployed to staging/pull/427 March 9, 2026 10:55 Inactive
@lawrence-forooghian lawrence-forooghian force-pushed the AIT-30/liveobjects-path-based-api-spec branch from 3608895 to b4ad764 Compare May 12, 2026 18:41
@github-actions github-actions Bot temporarily deployed to staging/pull/427 May 12, 2026 18:41 Inactive
@lawrence-forooghian lawrence-forooghian added live-objects Related to LiveObjects functionality. labels May 12, 2026
@lawrence-forooghian lawrence-forooghian changed the base branch from main to rename-channel-objects-to-object May 12, 2026 19:37
@lawrence-forooghian lawrence-forooghian force-pushed the rename-channel-objects-to-object branch 2 times, most recently from 7738c92 to fa2a54e Compare May 12, 2026 19:45
@lawrence-forooghian lawrence-forooghian force-pushed the AIT-30/liveobjects-path-based-api-spec branch from b4ad764 to 1eb4dd8 Compare May 12, 2026 19:51
@github-actions github-actions Bot temporarily deployed to staging/pull/427 May 12, 2026 19:52 Inactive
@lawrence-forooghian lawrence-forooghian force-pushed the AIT-30/liveobjects-path-based-api-spec branch from 1eb4dd8 to 035aef9 Compare May 12, 2026 19:56
@github-actions github-actions Bot temporarily deployed to staging/pull/427 May 12, 2026 19:57 Inactive
@lawrence-forooghian lawrence-forooghian force-pushed the AIT-30/liveobjects-path-based-api-spec branch from 035aef9 to babc296 Compare May 12, 2026 20:09
@github-actions github-actions Bot temporarily deployed to staging/pull/427 May 12, 2026 20:10 Inactive
lawrence-forooghian and others added 23 commits May 27, 2026 10:55
added in f9dc077 but actually would be
better off in the UTS
`Subscription` (returned by `subscribe`) is now the sole
deregistration mechanism, matching the ably-js public API.

RTLO4c is retained as a "This clause has been deleted" stub since
it existed on main; RTPO20 and RTINS17 are removed outright as
they were introduced earlier in this PR branch. The corresponding
`unsubscribe` declarations are also removed from the IDL.

Lifted from Sachin's spec-alignment PR [1].

[1] #480

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds SUB2b clarifying that repeated calls to `Subscription#unsubscribe`
are a no-op, matching the ably-js implementation across all three
subscription factories (LiveObject EventEmitter.off, the
PathObjectSubscriptionRegister Map.delete, and Instance which delegates
to LiveObject).

Lifted from Sachin's spec-alignment PR [1].

[1] #480

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The internal `count` (RTLCV2a) and `entries` (RTLMV2a) properties
were already specified in prose but missing from the IDL block.
Add them, matching the private `_count` and `_entries` fields on
ably-js's `LiveCounterValueType` and `LiveMapValueType`.

Lifted from Sachin's spec-alignment PR [1].

[1] #480

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Stubs out the new `parentReferences` internal property on
`LiveObject`, needed by `getFullPaths` (RTLO4f). The detailed
maintenance rules (across `MAP_SET`, `MAP_REMOVE`, `MAP_CLEAR`,
`LiveMap` tombstoning, and post-sync rebuild) are deferred to a
follow-up by Sachin; the in-progress draft is at [1].

ably-js stores `parentReferences` as a map keyed by a direct
`LiveMap` reference; the placeholder instead keys by `objectId`,
for consistency with how the rest of the LiveObjects spec models
inter-object references (forward references in `LiveMap` entries
are already objectIds resolved via the `ObjectsPool` on demand).

This is also load-bearing for languages without automatic cycle
collection. The protocol allows cyclic `LiveMap` graphs (e.g.
`A.x = B`, `B.y = A`), and `getFullPaths` is being specified to
handle them; under ARC in Swift, direct parent references in such
a cycle would form an unbreakable retain cycle on the two
`LiveMap`s. Keying by `objectId` lets the `ObjectsPool` remain the
single owner and sidesteps the issue.

Implementations remain explicitly permitted to store a direct
`LiveMap` reference if more idiomatic in their language -- e.g.
to avoid an `ObjectsPool` lookup at each `getFullPaths` traversal
step -- as ably-js does today, provided they handle the cycle
concern.

[1] #480

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Lifts the `getFullPaths` definition verbatim from commit ecf85df of
Sachin's spec-alignment PR [1].

The only departure from the source is renumbering: Sachin places
`getFullPaths` under `RTLO3 LiveObject properties` as `RTLO3g`;
this commit places it under `RTLO4 LiveObject methods` as `RTLO4f`
(with sub-clauses `RTLO4f1`-`RTLO4f7`) since `getFullPaths` is a
function, not a property. Cross-references in RTO24b1 and RTLO3f
are updated to match.

Lawrence has not reviewed the lifted content yet; the imported
clauses retain Sachin's capitalised RFC 2119 keywords and the
NetworkX references, both of which may be tightened in follow-up
commits.

[1] #480

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The term "outermost" was unclear in RTLM20e7g2 and RTLMV4d2. Replace
it with "final element in the list/array", leveraging RTLMV4k's
ordering guarantee that the value type's own MAP_CREATE comes last
in the returned array.

RTLM20e7g1 is also tweaked to explicitly normalise both branches
(RTLCV4 returns a single ObjectMessage; RTLMV4 returns an array)
into an ordered list, so that RTLM20e7g2's "final element" wording
applies uniformly for both LiveCounterValueType and
LiveMapValueType.

Addresses [1] and [2].

[1] #427 (comment)
[2] #427 (comment)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This was a transcription error in d4662fa, which intended to base
the `PublicAPI::ObjectMessage` (PAOM2) field types on ably-js's
public `ObjectMessage` type in `liveobjects.d.ts`. That type has
`connectionId?: string` (optional), but PAOM2c was written as
required.

Fix both the prose and the IDL to mark `connectionId` as optional,
matching ably-js.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PAOM3 constructs a `PublicAPI::ObjectMessage` from a source
`ObjectMessage`, and references the source's `operation` field
(both directly in PAOM3d and transitively via PAOOP3, which
expects an `ObjectOperation`). All three call sites (RTO24b2b2,
RTPO19d2, RTINS16d2) already gate the call on `operation` being
populated, and PAOM1 frames the type as the user-facing
representation of an `ObjectMessage` "that carried an operation",
but the procedure itself didn't state the precondition.

Add a PAOM3a "Preconditions" subclause stating that callers must
ensure the source has its `operation` field populated, and shift
the existing steps to PAOM3b-d.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
These values are not populated for `ObjectMessage`s created by
apply-on-ACK (RTO20d2).

Matches the corresponding change in ably-js#2230.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PR #480 [1] proposed specifying that ably-js deregisters all
LiveObject#subscribe listeners on tombstone. Adopt that proposal
with refined wording and a new LiveObjectUpdate.tombstone field
that makes the trigger condition explicit. Also add the related
ably-js refactor (commit 1d98cc3 [2]) that has tombstone() return
the cleared LiveObjectUpdate rather than dispatching it inline.

[1] #480
[2] ably/ably-js@1d98cc3

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Imports the parentReferences bookkeeping spec from PR #480 [1] onto
this integration branch, resolving the committed conflict marker at
RTLO3f and the duplicate clause IDs introduced by the import.

Imported from #480 verbatim:

- RTO5c10: post-sync rebuild of every parentReferences map.
- addParentReference and removeParentReference internal methods,
  with set-merge / set-remove / empty-set-delete semantics.
- Tombstone-time children walk for LiveMap, stripping parent
  references from each referenced child before the data is cleared.
- MAP_SET, MAP_REMOVE and MAP_CLEAR parent-reference maintenance
  (RTLM7a3, RTLM7i, RTLM8a3, RTLM24e1c).
- IDL declarations for the two new internal methods.

The Primitive type alias added in #480 was deliberately not
imported, as it is unrelated to the parentReferences work.

Conflicts reconciled:

- The committed <<<<<<< / >>>>>>> block at RTLO3f. Kept the
  objectId-keyed Dict<String, Set<String>> description from this
  branch (consistent with #480's own IDL line and its set-style
  manipulation contracts; the alternative half mandated a specific
  in-memory representation that ably-js does not match literally).
  The "set to an empty map on initialisation" clause from #480 was
  moved to RTLO3f2; the prior RTLO3f2 TODO is deleted, since the
  imported maintenance rules now resolve it. RTO5c10a's
  back-reference was updated to point at the new RTLO3f2.
- Duplicate clause IDs introduced by #480 were renamed per the
  "rename the later addition" convention in CONTRIBUTING.md:
    - addParentReference: RTLO4f -> RTLO4g
    - removeParentReference: RTLO4g -> RTLO4h
    - tombstone children walk: RTLO4e5* -> RTLO4e9*
  All cross-references to the renamed clauses were updated
  accordingly. The pre-existing RTLO4f (getFullPaths) and
  RTLO4e5-e8 (Compute LiveObjectUpdate through Return) are
  untouched.

Linter passes. Still needs human review.

[1] #480

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Follow-up to 54a3a02. The clauses pulled in from PR #480 use the
uppercase RFC 2119 convention (MUST etc.); lowercase them for
consistency with the prose style preferred on this branch.

Touches the 10 occurrences of MUST in RTO5c10, RTLO4g1-g2,
RTLO4h1-h3, RTLM7a3, RTLM7i, RTLM8a3 and RTLM24e1c. The
pre-existing uppercase keywords in RTLO4f1-f7 (getFullPaths) are
intentionally left alone.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The IDL entries imported from PR #480 declared these two methods
without argument types. Annotate them as (LiveMap parent, String
key), matching the conventional style used for multi-arg methods
elsewhere in the IDL and the parent/key descriptions in the
RTLO4g/RTLO4h prose.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The RTLO4f getFullPaths clause was added to the prose spec but
missed from the IDL. Add it as `getFullPaths() -> String[][]`,
positioned between tombstone (RTLO4e) and addParentReference
(RTLO4g) to preserve clause-letter ordering. The return type
reflects RTLO4f, which describes the result as a list of distinct
paths, each being an ordered sequence of string keys.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Make the objectId-keyed lookup convention explicit at the point of
use, rather than relying on the reader to infer it from RTLO3f.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drop the explicit ordering language (it's implied by the surrounding
RTO5c sequence), merge the entries-iteration and addParentReference
sub-clauses into one, and defer to LiveMap#entries to determine when
a value is a LiveObject.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fix the key argument (Sachin's version passed entry.value, not the
entry's key), align terminology with ObjectsMapEntry naming used
elsewhere in the file, flatten the nesting, and re-position relative
to RTLO4e4 by referencing the previous value of LiveMap.data
instead of imposing a "before RTLO4e4" ordering.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fold RTLM7i's parent-reference recording into RTLM7g as RTLM7g2,
removing the duplicated MapSet.value.objectId presence check.

Also replace "the operation's key" with "the specified key" in
RTLM7a3b, RTLM7g2 and RTLM8a3b, matching the wording used by the
surrounding RTLM7a/b/b4 and RTLM8a/b/b1 clauses.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
RTLM7a3, RTLM8a3, RTLM24e1c and RTLO4e9 now all share the same
"Before [target] is applied: { fetch from ObjectsPool; if found
call removeParentReference }" shape, dropping the imprecise
"ObjectsMapEntry is of type LiveObject" / "parent reference
recorded on existing ObjectsMapEntry" wording.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Nest RTLM7a3 and RTLM8a3 inside RTLM7a2 / RTLM8a2 (the "Otherwise,
apply" branches) so their "Otherwise" pair with the noop check
isn't obscured, and reword all four parent clauses (RTLM7a3,
RTLM8a3, RTLM24e1c, RTLO4e9) to name the data modification each
one precedes (set / cleared / removed / reset).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace the seven-clause MUST-style spec with four: define a directed
graph G over parentReferences, return the *key-paths* corresponding
to G's simple paths from root to this LiveObject. The new term
*key-path* (matching PathObject's "path" concept) is used here to
distinguish from the graph-theoretical "simple path". Edge cases
(root, orphan, multi-key, multi-ancestor, cycles) fall out of the
definition.

There's a tension here: the most universal contract would just say
"returns the key-paths from root to this LiveObject" and leave the
mechanism to SDK implementers. But any SDK implementing
`getFullPaths` will probably want a `parentReferences`-equivalent
data structure, and keeping that structure consistent across the
many places where `LiveMap.data` is mutated (`MAP_SET`, `MAP_REMOVE`,
`MAP_CLEAR`, tombstone, sync rebuild) is the part SDKs are likely to
get wrong. The prescriptive `parentReferences`-based formulation
pays for itself by making those bookkeeping responsibilities
explicit at each mutation site. If we hadn't already specified
`parentReferences` and its maintenance, we might not have bothered
— but we have, so let's use it.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Move the OBJECT_SUBSCRIBE mode + channel-state check (access API
preconditions) and the OBJECT_PUBLISH mode + channel-state +
echoMessages check (write API preconditions) out of the
LiveMap/LiveCounter/LiveObject public methods and into two new
common clauses (RTO25 and RTO26). Each PathObject and Instance
public method that accesses or mutates data now references the
applicable preconditions and renumbers its sub-clauses so the
check sits in a logical position (after Expects, before any data
work). External cross-references to the renumbered sub-clauses,
including the IDL section, are updated.

Two motivations:

1. Previously the spec placed these checks on LiveMap/LiveCounter,
   which delegating PathObject/Instance methods triggered only
   after path resolution and type checks. A call against a stale
   or detached channel could then yield a "wrong type" result
   (empty array etc.) instead of a state error. ably-js already
   moved the checks to the public entry points for this reason
   (commit a7462b14, "Handle channel configuration checks on
   PathObject/Instance level instead of LiveMap/LiveCounter").

2. With the checks lifted out, the underlying LiveMap/LiveCounter
   methods become non-throwing for channel-state reasons. This
   matters for internal callers that invoke them in a non-throwing
   context, e.g. RTO5c10b iterating LiveMap#entries during the
   post-sync parentReferences rebuild. See [1].

The displaced LiveMap/LiveCounter/LiveObject sub-clauses are kept
as "replaced by RTO25/RTO26" markers rather than deleted.

[1] #477 (comment)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@lawrence-forooghian
Copy link
Copy Markdown
Collaborator

Have rebased on the integration branch to lose the merge commit

Comment thread specifications/objects-features.md Outdated
- `(RTO11f1)` If `entries` is null or not of type `Dict`, the library should throw an `ErrorInfo` error with `statusCode` 400 and `code` 40003, indicating that `entries` must be a `Dict`. Note that `entries` is an optional argument, and if omitted, this error must not be thrown
- `(RTO11f2)` If any of the keys provided in `entries` are not of type `String`, the library should throw an `ErrorInfo` error with `statusCode` 400 and `code` 40003, indicating that keys must be `String`
- `(RTO11f3)` If any of the values provided in `entries` are not of an expected type, the library should throw an `ErrorInfo` error with `statusCode` 400 and `code` 40013, indicating that such data type is unsupported
- `(RTO23d)` Returns a new `PathObject` ([RTPO1](#RTPO1)) with `path` ([RTPO2a](#RTPO2)) set to an empty list and `root` ([RTPO2b](#RTPO2)) set to the `LiveMap` with id `root` from the internal `ObjectsPool`
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some spec id mismatches here it seems:

([RTPO2a](#RTPO2)) set to an empty list and `root` ([RTPO2b](#RTPO2))

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, fixed d479b8d

Comment thread specifications/objects-features.md Outdated
- `(RTO24b2a1)` The first (most-preferred) candidate is `pathToThis` itself
- `(RTO24b2a2)` If the `LiveObjectUpdate` is a `LiveMapUpdate`, then for each key in `LiveMapUpdate.update`, append a further candidate consisting of `pathToThis` extended by that key
- `(RTO24b2b)` For each registered subscription, find the first `eventPath` in `candidatePaths` that the subscription covers per [RTO24c1](#RTO24c1). If no such `eventPath` exists, do nothing for this subscription. Otherwise, call the subscription's listener exactly once with a `PathObjectSubscriptionEvent` that has:
- `(RTO24b2b1)` `object` - a new `PathObject` ([RTPO1](#RTPO1)) with `path` ([RTPO2a](#RTPO2)) set to `eventPath` and `root` ([RTPO2b](#RTPO2)) set to the `LiveMap` with id `root` from the internal `ObjectsPool`
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mismatch spec ids:

([RTPO2a](#RTPO2)) set to `eventPath` and `root` ([RTPO2b](#RTPO2))

Copy link
Copy Markdown
Collaborator

@lawrence-forooghian lawrence-forooghian May 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, fixed 83a0404

Comment thread specifications/objects-features.md Outdated
- `(RTLMV4c)` If any of the values in the internal `entries` are not of an expected type, the library should throw an `ErrorInfo` error with `statusCode` 400 and `code` 40013, indicating that such data type is unsupported
- `(RTLMV4d)` Build entries for the `MapCreate` object. For each key-value pair in the internal `entries` (if present), create an `ObjectsMapEntry` for the value:
- `(RTLMV4d1)` If the value is of type `LiveCounterValueType`, evaluate it per [RTLCV4](#RTLCV4) to generate a `COUNTER_CREATE` `ObjectMessage`. Collect the generated `ObjectMessage` and set `ObjectsMapEntry.data.objectId` to the `objectId` from the `ObjectMessage`
- `(RTLMV4d2)` If the value is of type `LiveMapValueType`, recursively evaluate it per [RTLMV4](#RTLMV4) to generate an ordered array of `ObjectMessages`. Collect all generated `ObjectMessages` and set `ObjectsMapEntry.data.objectId` to the `objectId` from the final `ObjectMessage` in the array
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Appreciate final sounds better than outermost from the initial spec.
However, I find it still a bit confusing on the first read. Maybe we can benefit from adding an additional note here, clarifying that by "final" we means the last ObjectMessage, which is produced by RTLMV4k?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wasn't sure exactly what was the unclear part here but I've added a bit of descriptive text to explain what this message is 9e1cd88

lawrence-forooghian and others added 3 commits May 27, 2026 14:56
Both clauses linked `RTPO2a` and `RTPO2b` to `#RTPO2` (the parent
clause). Point them at their own anchors instead, matching the
convention already used by RTPO19f.

Reported by @VeskeR on PR #427 [1] [2].

[1] #427 (comment)
[2] #427 (comment)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
In RTLMV4d2 and RTLM20e7g2, the spec instructs implementers to take
the objectId from the "final" ObjectMessage produced by recursively
evaluating a LiveMapValueType (or LiveCounterValueType). "Final" is
unambiguous as the last element of an ordered array, but doesn't on
its own explain why that particular message is the right one to
reference. Spell out that it is the create operation for the
LiveObject whose creation the value type represents.

Reported by @VeskeR on PR #427 [1].

[1] #427 (comment)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The LiveObjects data model is a directed graph that may contain
cycles (cyclic structures can be created via the REST API), not a
tree. RTLO4f already frames it explicitly in graph-theoretic terms.
Several descriptions of `PathObject`, `Instance`, and `getFullPaths`
introduced in this PR referred to it as a tree; switch them to
"graph" (and reword the "subtree" reference in RTPO19c1, where
depth is actually about path nesting below the subscription path,
not tree depth from the root).

Reported by @VeskeR on PR #427 [1].

[1] #427 (comment)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@lawrence-forooghian
Copy link
Copy Markdown
Collaborator

Replaced by #484; keeping the non-squashed history around here for future reference

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

live-objects Related to LiveObjects functionality.

Development

Successfully merging this pull request may close these issues.

5 participants