Skip to content

architecture.md

Latest commit

 

History

History
282 lines (238 loc) · 15.3 KB

architecture.md

File metadata and controls

282 lines (238 loc) · 15.3 KB
Raw

Architecture

This document describes the internal architecture of mpt-api-python-client.

Overview

mpt-api-python-client is a Python API client that provides a typed, fluent interface for the SoftwareONE Marketplace Platform (MPT) REST API. It supports both synchronous and asynchronous usage and is built on top of httpx.

API Reference: The full upstream API contract is described by the MPT OpenAPI Spec. The client mirrors this spec's resource structure.

The client exposes every MPT API domain (catalog, commerce, billing, etc.) as a resource group, where each resource is a service object composed from reusable HTTP operation mixins.

Directory Structure

mpt_api_client/
├── __init__.py              # Public API: MPTClient, AsyncMPTClient, RQLQuery
├── mpt_client.py            # Client entry points
├── constants.py             # Shared constants (content types)
├── exceptions.py            # Error hierarchy (MPTError, MPTHttpError, MPTAPIError)
│
├── http/                    # HTTP transport layer
│   ├── client.py            # Sync HTTPClient (httpx.Client)
│   ├── async_client.py      # Async AsyncHTTPClient (httpx.AsyncClient)
│   ├── base_service.py      # ServiceBase — shared service logic
│   ├── service.py           # Service — sync service (extends ServiceBase)
│   ├── async_service.py     # AsyncService — async service (extends ServiceBase)
│   ├── query_state.py       # Query parameter accumulation
│   ├── client_utils.py      # URL validation helpers
│   ├── file_utils.py        # Upload content-type declaration
│   ├── types.py             # Type aliases (Response, HeaderTypes, etc.)
│   ├── jsonl_lines.py       # JSONL record-line splitting (newlines only)
│   └── mixins/              # Composable HTTP operation mixins
│       ├── collection_mixin.py
│       ├── create_mixin.py
│       ├── create_file_mixin.py
│       ├── update_mixin.py
│       ├── update_file_mixin.py
│       ├── delete_mixin.py
│       ├── get_mixin.py
│       ├── enable_mixin.py
│       ├── disable_mixin.py
│       ├── download_file_mixin.py
│       ├── file_operations_mixin.py
│       ├── stream_jsonl_mixin.py
│       ├── streaming_mixin.py
│       ├── queryable_mixin.py
│       └── resource_mixins.py
│
├── models/                  # Response models
│   ├── model.py             # Model base class (camelCase ↔ snake_case mapping)
│   ├── model_collection.py  # ModelCollection — paginated result set
│   ├── meta.py              # Meta / Pagination metadata
│   └── file_model.py        # FileModel for binary responses
│
├── resources/               # API domain modules
│   ├── accounts/            # Account, Users, Buyers, Sellers, API Tokens, …
│   ├── audit/               # Audit records, Event types
│   ├── billing/             # Invoices, Ledgers, Journals, Statements, Credit memos, …
│   ├── catalog/             # Products, Listings, Price lists, Authorizations, …
│   ├── commerce/            # Agreements, Orders, Subscriptions, Assets
│   ├── exchange/            # Exchange resources
│   ├── helpdesk/            # Cases, Chats, Queues, Forms, …
│   ├── integration/         # Integration resources
│   ├── notifications/       # Messages, Batches, Subscribers, …
│   ├── program/             # Program resources
│   └── spotlight/           # Spotlight resources
│
└── rql/                     # RQL query builder
    ├── query_builder.py     # RQLQuery, RQLProperty, RQLValue
    ├── encoding.py          # Percent-encoding for RQL field names and values
    └── constants.py         # RQL operator constants

Layered Architecture

The client is organized into four layers:

┌─────────────────────────────────────────────┐
│           MPTClient / AsyncMPTClient        │  Entry point
├─────────────────────────────────────────────┤
│         Resource Groups (domains)           │  catalog, commerce, billing, …
├─────────────────────────────────────────────┤
│   Service + Mixins (HTTP operations)        │  get, create, update, delete, iterate, …
├─────────────────────────────────────────────┤
│       HTTPClient / AsyncHTTPClient          │  httpx transport
└─────────────────────────────────────────────┘

1. Client Layer — mpt_client.py

MPTClient (sync) and AsyncMPTClient (async) are the public entry points.

Each client holds an HTTP client instance and exposes domain-specific resource groups as properties:

client = MPTClient.from_config(authentication=BearerTokenAuthentication("..."), base_url="...")
client.catalog  # Catalog
client.commerce  # Commerce
client.billing  # Billing
client.accounts  # Accounts
client.audit  # Audit
client.helpdesk  # Helpdesk
client.notifications  # Notifications

2. Resource Groups — resources/

Each resource group (e.g. Catalog, Commerce) is a plain class that groups related service objects. For example, Catalog exposes products, listings, price_lists, authorizations, pricing_policies, items, and units_of_measure.

Resource groups pass the HTTP client down to each service.

3. Service Layer — http/service.py, http/async_service.py

Service and AsyncService extend ServiceBase and represent a single REST resource endpoint (e.g. /catalog/products).

Services are composed using mixins that add HTTP operations:

Mixin Operation
CollectionMixin iterate() — paginated listing; inherits StreamingMixin, adding stream_snapshot()
GetMixin get(id) — retrieve single resource
CreateMixin create(data) — create resource
UpdateMixin update(id, data) — update resource
DeleteMixin delete(id) — delete resource
CreateFileMixin create with file upload
UpdateFileMixin update with file upload
DownloadFileMixin download binary content
EnableMixin / DisableMixin enable/disable actions
QueryableMixin filter(), order_by(), select() — RQL query chaining
StreamingMixin stream_snapshot() — streaming read mode, opted into with the MPT-Streaming header
StreamJSONLMixin stream() — JSONL endpoints that define their own meaning for application/jsonl (billing statement charges)
FilesOperationsMixin combined file create / update / download operations

The table lists the synchronous names; every mixin except QueryableMixin, which is shared, has an Async* counterpart for composition with AsyncService.

The platform streaming contract (StreamingMixin / AsyncStreamingMixin) exposes stream_snapshot(), while the endpoint-specific JSONL contract (StreamJSONLMixin / AsyncStreamJSONLMixin) exposes stream(), so a service can expose both without the method names colliding. stream() belongs to the JSONL contract because that is what it meant through 6.x; the next major version gives the name back to the streaming read. CollectionMixin and AsyncCollectionMixin inherit the platform streaming mixins, so every collection service carries stream_snapshot() without composing them explicitly; do not list StreamingMixin before a collection service's CollectionMixin base, because that ordering cannot produce a consistent MRO. The platform mixins request the streaming read mode on a regular collection route and require the API to echo the MPT-Streaming response header, raising MPTStreamingNotEnabledError when it does not. They also verify completeness: the declared MPT-Item-Count is read before the first record — raising MPTStreamingItemCountMissingError when absent or unusable — and compared with the count of raw records consumed once the body is fully consumed, raising MPTStreamingIncompleteError on mismatch. The count is taken before skip_deleted withholds any stub, so a filtered stub still counts. An iterator closed early skips the comparison. A record marked with $meta.deleted is a deletion stub rather than data, and is yielded as a DeletionStub instead of a model, so it still counts towards the declared item count but cannot be ingested as a record. A consumer that ingests no deletions can opt out with the keyword-only skip_deleted=True, which withholds stubs at yield time — after the completeness bookkeeping and the progress tick — and is typed with overloads, so the call narrows to an iterator of models. The JSONL mixins serve endpoints that assign application/jsonl their own meaning outside streaming mode.

stream_snapshot() always sends Accept: application/jsonl and reads one record object per line. The body is parsed as it arrives, so the memory bound is one record rather than one export, and a body the client cannot parse raises json.JSONDecodeError. Record lines are split by http/jsonl_lines.py; see the streaming guide for the record-boundary contract. A short read surfaces one of three ways: a transport failure once the body has started raises MPTStreamingTruncatedError, since retry cannot re-request a stream in flight; a completed message whose last record is unterminated raises the decode error; and a completed, well-formed body carrying fewer records than MPT-Item-Count declared raises MPTStreamingIncompleteError. A progress receiver's total comes from the MPT-Item-Count header; see the streaming guide for the consumer-facing progress contract.

StreamingMixin.stream_snapshot() takes limit and offset and forwards them to the collection route unchanged, omitting whichever is unset. Validating them locally is deliberately out of scope: the server owns pagination-input validation, and the inputs it accepts in streaming mode are still changing.

See the streaming guide for the consumer-facing contract these mixins implement.

Example service definition:

class ProductsService(
    Service[Model],
    CollectionMixin,
    GetMixin,
    CreateFileMixin,
    UpdateFileMixin,
    DeleteMixin,
):
    _endpoint = "/catalog/products"
    _model_class = Model

4. HTTP Transport — http/client.py, http/async_client.py

HTTPClient and AsyncHTTPClient wrap httpx.Client / httpx.AsyncClient with:

  • pluggable authentication via an Authentication provider (BearerTokenAuthentication, EnvTokenAuthentication)
  • base URL resolution
  • retry transport (configurable)
  • error transformation into MPTHttpError / MPTAPIError
  • multipart file upload support; http/file_utils.py declares a content type for an upload whose extension mimetypes does not recognise, such as the .jsonl a billing journal carries, which httpx would otherwise send as application/octet-stream

Transport-level settings (base_url, timeout, retries) are grouped in the TransportSettings dataclass (http/transport_settings.py), passed to the client constructors as transport=TransportSettings(...). Timeouts resolve per connection phase: connect_timeout, read_timeout, write_timeout and pool_timeout each fall back to timeout, and the dataclass exposes two profiles — request_timeout for regular requests and stream_timeout, whose read phase is the larger of stream_read_timeout and read_timeout, because a streamed response defers its first byte until the server has built the result set. To resolve the base URL from the MPT_API_BASE_URL environment variable instead, pass EnvTransportSettings() (the default when no transport is given); the clients themselves never read the environment. The resolved settings are handed to the authentication provider through Authentication.configure(transport) at client construction time. The authentication provider is always passed explicitly.

Cross-Cutting Concerns

RQL Query Builder — rql/

See the RQL guide for the fluent query builder, filter chaining, value encoding, and usage examples.

Model Layer — models/

Model is a lightweight base class that:

  • converts API responses from camelCase to snake_case attribute access
  • supports nested model parsing
  • provides to_dict() serialization back to camelCase

Collection[Model] wraps paginated API responses with metadata (Meta, Pagination).

DeletionStub (models/deletion_stub.py) is deliberately not a Model. It carries only the id of a row deleted after a stream's membership snapshot, which is the only property the platform guarantees on a $meta.deleted stub, so the type keeps a stub from being mistaken for a record with unset fields. It is distinct from the domain DELETED status, which is a state of a full record.

Error Handling — exceptions.py

Client, transport, and API errors use the following hierarchy:

MPTError
├── MPTStreamingError                       # base for streaming-mode failures
│   ├── MPTStreamingNotEnabledError         # response did not confirm streaming mode
│   ├── MPTStreamingFormatMismatchError     # Content-Type differed from the requested format
│   ├── MPTStreamingItemCountMissingError   # no usable MPT-Item-Count declared
│   ├── MPTStreamingIncompleteError         # record count differed from MPT-Item-Count
│   └── MPTStreamingTruncatedError          # body ended before the HTTP message completed
├── MPTMaxRetryError                        # retry attempts exhausted
└── MPTHttpError                            # generic HTTP error (status_code, message, body)
    ├── MPTAPIError                         # structured API error (payload, title, detail, trace_id)
    ├── MPTStreamingNotSupportedError       # 501, resource cannot stream (also MPTStreamingError)
    ├── MPTStreamingNotAcceptableError      # 406, format unsupported (also MPTStreamingError)
    └── MPTStreamingOverCapError            # 413, export over cap (also MPTStreamingError)

An error response becomes an MPTAPIError only when its body parses as a JSON object, whose members populate title, detail, trace_id and errors. httpx detects the JSON encoding before parsing, so an object arriving as UTF-16 or UTF-32 keeps its members too — tolerance of a non-conforming body, not an endorsement of it: RFC 8259 requires UTF-8 for interchange. Any other body — valid JSON that is not an object (a bare string, null, an array), a body that is not JSON at all, or bytes that do not decode — carries no members to read and becomes a plain MPTHttpError whose body holds the raw diagnostic, with undecodable bytes replaced. Either way status_code is preserved, so a caller can always branch on the status.