|
| 1 | +# ESB Support |
| 2 | + |
| 3 | +> Status: **completed**. This document is the original feature request plus the |
| 4 | +> implementation plan and per-milestone progress notes recorded as the work was |
| 5 | +> carried out. |
| 6 | +
|
| 7 | +## Overview |
| 8 | + |
| 9 | +Goal: implement the required minimal changes in this application to support https://github.com/DecaturMakers/equipment-status-board/issues/10. |
| 10 | + |
| 11 | +At a high level, this means: |
| 12 | + |
| 13 | +1. If we don't already have one, a simple API for retrieving a list of all machines and their current status. |
| 14 | +2. Ability to fire a webhook (non-blocking/async or from a background worker) when a machine's status is changed (user login/logout, unauthorized RFID fob, oops/lockout/clear), using a webhook destination configured via environment variable. The webhook should include information on the currently-logged-in user if there is one. This should only fire on meaningful status changes, not on every update from the MCU. |
| 15 | +3. If we don't already have them, API endpoints to oops a machine, put it in maintenance lockout, or clear oops/lockout. |
| 16 | + |
| 17 | +Our approach is to minimize change in this application, and let ESB do the heavy lifting; we just need to send events to it via webhook and give it the appropriate API endpoints to call. |
| 18 | + |
| 19 | +## Implementation Plan |
| 20 | + |
| 21 | +Commit-message prefix for this feature: ``ESB Support - {Milestone}.{Task}``. |
| 22 | + |
| 23 | +### Findings / Scope |
| 24 | + |
| 25 | +Mapping the three high-level requirements onto the current codebase: |
| 26 | + |
| 27 | +1. **Machine list/status API** — *does not exist as JSON.* Today status is only |
| 28 | + available via the Prometheus ``/metrics`` endpoint and the Slack ``status`` |
| 29 | + command. We will add a small read-only JSON endpoint. |
| 30 | +2. **Status-change webhook** — *does not exist.* The meaningful status changes |
| 31 | + already have well-defined chokepoints in ``models/machine.py`` where Slack |
| 32 | + notifications fire (login, logout, unauthorized/unknown fob, override login, |
| 33 | + oops, un-oops, lockout, unlock, reboot). We will fire the webhook from those |
| 34 | + same sites so it only fires on meaningful changes, never on ordinary MCU |
| 35 | + heartbeats. |
| 36 | +3. **Oops / lockout / clear API endpoints** — *already exist* in |
| 37 | + ``views/machine.py``: ``POST``/``DELETE /api/machine/oops/<name>`` and |
| 38 | + ``POST``/``DELETE /api/machine/locked_out/<name>``. ESB "clear" = ``DELETE`` |
| 39 | + on whichever of the two is set. No new control endpoints are required; we |
| 40 | + only verify and document them. |
| 41 | + |
| 42 | +The ESB issue also mentions storing 100–500 recent activity events per machine. |
| 43 | +Per the "let ESB do the heavy lifting" directive, **ESB stores that history from |
| 44 | +the webhook events**; MAC does not add history storage. |
| 45 | + |
| 46 | +### Design Decisions (confirmed with maintainer) |
| 47 | + |
| 48 | +* **Webhook destination:** ``STATUS_WEBHOOK_URL`` environment variable. When |
| 49 | + unset, webhook firing is disabled entirely (mirrors how Slack is disabled |
| 50 | + without its tokens). |
| 51 | +* **Authentication:** none. The POST is sent with no auth header. |
| 52 | +* **Delivery:** fire-and-forget (``asyncio.create_task``, like Slack) so the MCU |
| 53 | + response is never blocked, with a bounded **retry-with-backoff** in the |
| 54 | + background task and a per-attempt timeout. Failures are logged after retries |
| 55 | + are exhausted; ESB can reconcile via the status API. |
| 56 | + |
| 57 | +### Shared status representation |
| 58 | + |
| 59 | +A single ``status_dict`` (and derived ``status`` string) will be used by **both** |
| 60 | +the list API and the webhook payload so they never drift: |
| 61 | + |
| 62 | +```json |
| 63 | +{ |
| 64 | + "name": "planer", |
| 65 | + "display_name": "Planer", |
| 66 | + "status": "in_use", // one of: idle | in_use | oops | locked_out | unknown |
| 67 | + "relay": true, |
| 68 | + "oops": false, |
| 69 | + "locked_out": false, |
| 70 | + "current_user": { "account_id": "123", "full_name": "Jane Doe" }, // or null |
| 71 | + "last_checkin": 1720000000.0, // epoch seconds, or null if never |
| 72 | + "last_update": 1720000000.0 // epoch seconds, or null if never |
| 73 | +} |
| 74 | +``` |
| 75 | + |
| 76 | +The webhook payload is this dict plus an ``event`` field and a firing |
| 77 | +``timestamp``. Event values: ``login``, ``logout``, ``unauthorized``, |
| 78 | +``unknown_fob``, ``override_login``, ``oops``, ``unoops``, ``lockout``, |
| 79 | +``unlock``, ``reboot``. |
| 80 | + |
| 81 | +### Milestone 1 — ESB-facing status & control API — ✅ COMPLETE |
| 82 | + |
| 83 | +Added ``Machine.status`` and ``Machine.status_dict`` (shared representation), |
| 84 | +the ``GET /api/machines`` endpoint with ``MachineStatus`` / ``MachinesListResponse`` |
| 85 | +schemas, and unit tests for both the model properties and the endpoint. Verified |
| 86 | +the existing oops/lockout endpoints already satisfy requirement #3 and are |
| 87 | +well-covered (no code change needed). All ``nox -s tests`` pass at 97% coverage. |
| 88 | + |
| 89 | +* **1.1** Add a ``status`` property (derived string) and a ``status_dict`` |
| 90 | + property on ``Machine`` (reading ``self.state``), producing the shared |
| 91 | + representation above. ``current_user`` serializes to ``{account_id, full_name}`` |
| 92 | + or ``null``. |
| 93 | +* **1.2** Add ``GET /api/machines`` returning ``{"machines": [status_dict, ...]}`` |
| 94 | + for all configured machines, sorted by name. Add ``pydantic`` response schemas |
| 95 | + in ``models/api_schemas.py`` and document the route with ``quart_schema`` |
| 96 | + (tagged ``Admin``). Read-only and unauthenticated, consistent with the existing |
| 97 | + API surface. |
| 98 | +* **1.3** Confirm the existing oops/lockout endpoints satisfy ESB's control |
| 99 | + needs; add any missing test coverage. No behavior change expected. |
| 100 | +* **1.4** Unit tests for ``status``/``status_dict`` and the new endpoint |
| 101 | + (idle/in-use/oops/locked-out, with and without a current user). |
| 102 | + |
| 103 | +### Milestone 2 — Status-change webhook — ✅ COMPLETE |
| 104 | + |
| 105 | +Added ``src/dm_mac/webhook.py`` (``WebhookNotifier``: fire-and-forget delivery |
| 106 | +via ``aiohttp`` with bounded exponential-backoff retry, enabled by |
| 107 | +``STATUS_WEBHOOK_URL``), wired it into ``create_app`` as ``WEBHOOK_NOTIFIER``, |
| 108 | +and fired ``notify(...)`` from the meaningful status-change sites in |
| 109 | +``models/machine.py`` (login / logout / unauthorized / unknown_fob / |
| 110 | +override_login / oops / unoops / lockout / unlock / reboot), including the |
| 111 | +always-enabled RFID-tracking path. The notifier is resolved via |
| 112 | +``current_app`` in the request-context (MCU/API) paths and via the passed |
| 113 | +Slack handler's app (``slack.quart``) in the Slack-command path, which runs |
| 114 | +without a request context. Payloads reuse ``Machine.status_dict`` plus |
| 115 | +``event``, ``timestamp``, and a distinct ``user`` (event actor) field. Unit |
| 116 | +tests cover the notifier (payload/retry/backoff/disabled) and the wiring |
| 117 | +(each event fires; heartbeats do not; Slack path; disabled no-op). All |
| 118 | +``nox -s tests`` and ``nox -s mypy`` pass; ``webhook.py`` at 100% coverage. |
| 119 | + |
| 120 | +* **2.1** New module ``src/dm_mac/webhook.py`` with a ``WebhookNotifier`` class: |
| 121 | + * Constructed from ``STATUS_WEBHOOK_URL``. |
| 122 | + * ``notify(machine, event, user=None)`` builds the payload from |
| 123 | + ``machine.status_dict`` + ``event`` + ``timestamp`` and spawns a |
| 124 | + fire-and-forget ``asyncio.create_task`` delivery coroutine. |
| 125 | + * Delivery coroutine POSTs JSON via ``aiohttp`` (already a dependency) with a |
| 126 | + per-attempt timeout and retry-with-backoff (a small, bounded number of |
| 127 | + attempts with exponential backoff); logs a single error after exhaustion. |
| 128 | +* **2.2** In ``create_app``/``main`` (``__init__.py``), instantiate the notifier |
| 129 | + when ``STATUS_WEBHOOK_URL`` is set and store it as |
| 130 | + ``app.config["WEBHOOK_NOTIFIER"]`` (default ``None``), exactly like |
| 131 | + ``SLACK_HANDLER``. |
| 132 | +* **2.3** Fire ``notify(...)`` from the same async sites that emit Slack |
| 133 | + messages, so webhooks track meaningful changes only: |
| 134 | + * ``MachineState._handle_rfid_insert`` → ``login`` / ``unauthorized`` / |
| 135 | + ``unknown_fob`` / ``override_login`` |
| 136 | + * ``MachineState._handle_rfid_remove`` → ``logout`` |
| 137 | + * ``MachineState._handle_reboot`` → ``reboot`` (and logout of any prior user) |
| 138 | + * ``Machine.oops`` / ``MachineState._handle_oops`` → ``oops`` |
| 139 | + * ``Machine.unoops`` → ``unoops``; ``Machine.lockout`` → ``lockout``; |
| 140 | + ``Machine.unlock`` → ``unlock`` |
| 141 | + Each site fetches the notifier via ``current_app.config.get("WEBHOOK_NOTIFIER")`` |
| 142 | + and no-ops when ``None``. |
| 143 | +* **2.4** Unit tests: payload shape per event, disabled-when-unset, retry/backoff |
| 144 | + on failure (mock ``aiohttp``), and non-firing on ordinary heartbeat updates. |
| 145 | + |
| 146 | +### Milestone 3 — Acceptance Criteria — ✅ COMPLETE |
| 147 | + |
| 148 | +Updated documentation (`docs/source/configuration.rst` env-var table; |
| 149 | +`docs/source/http-api.rst` with the Machine Status API and Status-Change |
| 150 | +Webhook sections; `CLAUDE.md` app-config, API endpoints, env vars, and a new |
| 151 | +Status-Change Webhook subsection). `README.rst` is a pointer to the full docs |
| 152 | +and needed no change. `GET /api/machines` is picked up automatically by the |
| 153 | +generated OpenAPI spec, and `dm_mac.webhook` by `sphinx-apidoc`. New code has |
| 154 | +unit-test coverage (`webhook.py` at 100%). All `nox` sessions pass: `tests`, |
| 155 | +`mypy`, `pre-commit`, `typeguard`, `docs`, and `safety`. |
| 156 | + |
| 157 | +Note: `nox -s safety` was failing on `main` due to a pre-existing transitive |
| 158 | +`msgpack==1.2.0` advisory (GHSA-6v7p-g79w-8964), unrelated to this feature. To |
| 159 | +satisfy the "all nox sessions passing" criterion, `msgpack ^1.2.1` is pinned in |
| 160 | +`pyproject.toml`. Because `msgpack` is only pulled in via `cachecontrol` (a |
| 161 | +dev-group dependency) and is not used at runtime, the pin lives in |
| 162 | +`group.dev.dependencies` so it does not expand the production dependency set. |
| 163 | + |
| 164 | +* **3.1** Update documentation: ``README.md`` (env var + endpoints if listed), |
| 165 | + ``docs/source/configuration.rst`` (add ``STATUS_WEBHOOK_URL`` to the env-var |
| 166 | + table), ``docs/source/http-api.rst`` (document ``GET /api/machines`` and the |
| 167 | + webhook payload/events), and ``CLAUDE.md`` (env vars + architecture notes). |
| 168 | + Match the existing style and verbosity. |
| 169 | +* **3.2** Ensure all new code has appropriate unit-test coverage. |
| 170 | +* **3.3** All ``nox`` sessions pass (``tests``, ``mypy``, ``pre-commit``, |
| 171 | + ``safety``, ``typeguard``, ``docs``). |
| 172 | +* **3.4** Move this file from ``docs/features/`` to ``docs/features/completed/``. |
0 commit comments