Skip to content

feat(metrics): record deployment, device class, mail provider, owner and attempt on every auth event - #306

Merged
Bccorb merged 1 commit into
mainfrom
feat/auth-path-instrumentation
Sep 12, 2026
Merged

Bccorb merged 1 commit into
mainfrom
feat/auth-path-instrumentation

Conversation

@Bccorb

@Bccorb Bccorb commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

Closes #154. The write-path half: the dimensions that cannot be backfilled, recorded on every auth_events row before the Seamless Idea fleet starts producing sign-ins, and the one query the issue asks for. The read-side numbers already answerable from existing rows shipped as #304.

What changed

Five nullable columns on auth_events, stamped by AuthEventService on every row:

Column Value Source
deployment_id APP_ID The environment. The same value every tier's terraform already puts on sessions.infraId.
device_class ios, android, macos, windows, linux, chromeos, bot, unknown The request user agent.
mail_provider a consumer provider name, other, or null The subject's address, when the row is about the request principal or the handler names the subject.
owner true, false, or null Whether the subject's address is in OWNER_EMAIL. Null when the subject is unknown.
attempt_id UUID or null The ephemeral token's new jti.

user_agent widens to text in the same migration. It was varchar(255), never hit while the adapter wrote its own short user agent, and would have failed the whole audit write for a long browser one.

Attempt id. /login and /registration/register mint a UUID, write it on the row that starts the attempt, and sign the ephemeral token with it as jti. verifyBearerAuth reads it back onto req.attemptId, AuthEventService.log defaults to it, and OTP resends and decoy re-mints pass the one they received so the attempt stays whole. This is the optional item from the scoping comment; it is what makes drop-off per step and time to login attributable exactly rather than by adjacency.

Deliverability. Every OTP send is now one otp_success row with metadata.channel (email or sms), including the send registration makes, which used to write none. A send the provider refuses is a new otp_failed event with reason: 'Delivery failed', told apart from a server fault by a typed DeliveryError thrown from messagingService. otp_failed is not in LOCKOUT_FAILURE_TYPES, so lockout is unchanged.

Forwarded user agent. The adapter is the only client this API sees, so every row records the adapter's user agent. A new x-seamless-client-user-agent header is honoured under the same service-token rule as x-seamless-client-ip and replaces the request header in place, so the audit row, the session record and the magic link device binding all see the browser's. The middleware is renamed applyTrustedClientContext.

The query. GET /internal/metrics/sign-ins?from&to, behind admin:read like the funnel endpoint:

{
  "deploymentId": "a1b2c3",
  "attempts": { "started": 412, "delivered": 130, "presented": 388, "completed": 371 },
  "signIns": { "success": 371, "failed": 21, "successRate": 0.946 },
  "breakdown": [
    { "method": "passkey", "deviceClass": "ios", "mailProvider": "gmail", "owner": false, "success": 202, "failed": 4 }
  ]
}

The unit is a method presented within an attempt, not an event, so a completed OTP sign-in (which logs verify_otp_success twice) counts once and a mistyped-then-correct code is one success. Rows with no attempt id stand as their own attempt, so older rows still count once each. breakdown is one flat row per combination; consumers pivot.

Decisions worth knowing

  • Platform families, not form factors, for device class. iCloud Keychain, Google Password Manager, Windows Hello and a Linux desktop with no platform authenticator are four different passkey experiences; mobile-vs-desktop would hide the two that matter most. bot keeps scanners out of a published rate. A server runtime such as node is unknown, not bot, because that is what an adapter that does not forward the user agent looks like and those are real people.
  • Provider names, never domains. A domain is an address for anyone on a family or small business domain, and these rows are meant to be published. Custom domains on Google Workspace or Microsoft 365 land in other; telling them apart takes an MX lookup on the send path.
  • The address is never stored. subjectEmail is an input to two classifiers and nothing else. It defaults from req.user only when the principal is the subject, never for a decoy or an administrator acting on another account, so those rows carry null.
  • Publishability and retention are written down in docs/telemetry.md: aggregates are what leave a deployment, never rows; cells of one are suppressed; the dimensions follow whatever auth_events retention becomes (Audit events have no retention policy and no bulk export #173). The fleet-wide form of the query, with COUNT(DISTINCT deployment_id) and owner = false, is there too.

Verification

  • The migration chain was run up and down against a scratch Postgres, then the real getSignInMetrics and a real AuthEventService.log write (a 442 character user agent, owner derivation, every dimension) against it. A hand-seeded fixture covering a failed-then-successful OTP inside one attempt, a double-logged success, an owner passkey failure, an abandoned attempt and legacy rows with no attempt id produced the counts computed by hand.
  • typecheck, lint, format:check, build, check:changeset-bump clean
  • test:run: 118 files, 1464 passing (new unit tests for both classifiers and the metrics service, extended tests for the event service, token, bearer middleware, trusted-context middleware, OTP and registration controllers, login and the new route)
  • /security-review found nothing meeting its bar. It did catch that an address at constructor.io reached Object.prototype through an object-literal lookup and would have dropped that subject's audit rows; the tables are Maps now, with a regression test.
  • openapi.json and src/generated/api.ts regenerated; minor changeset

Downstream

Contract change: a new route, a new request header, a new event type, a new token claim (additive), and new columns on the auth event row. AuthEventSchema in @seamless-auth/types is a plain z.object, so the five columns are stripped from GET /admin/auth-events until the shared schema carries them. The adapter does not forward the user agent yet, so every adapter-fronted deployment records device_class = 'unknown' until it does. Ripple issues are linked from the comments.

…and attempt on every auth event

None of these can be backfilled: the mail provider is taken from an address the
row does not store, the device class from a user agent that only means
something once the adapter forwards the browser's, and the attempt id from a
token claim that did not exist. Each is recorded at write time for that reason,
before the Seamless Idea fleet starts producing sign-ins.

deployment_id is APP_ID, the same value as sessions.infraId. device_class folds
the user agent into a platform family, since platform is where passkeys differ.
mail_provider folds the subject's address into a consumer provider or "other",
never the domain, so the rows can be published. owner records whether the
subject is in OWNER_EMAIL, which is what makes "did somebody other than the
owner sign in" a query. attempt_id is the ephemeral token's new jti: /login and
/registration/register mint it, write it on the row that starts the attempt,
and every step taken on the token carries it. user_agent widens to text at the
same time, since a real browser user agent could exceed the old column and fail
the whole audit write.

Every OTP send is now one otp_success row with metadata.channel, including the
send registration makes, which used to write none. A send the provider refuses
is a new otp_failed event, distinguished from a server fault by a typed
DeliveryError.

The adapter can forward the browser's user agent as
x-seamless-client-user-agent, honoured under the same service-token rule as
x-seamless-client-ip. Without it every row records the adapter's own user agent
and the device class breakdown reads "unknown".

GET /internal/metrics/sign-ins answers how many attempts succeeded and failed
per method, device class, mail provider and owner flag, plus where attempts
stop. It counts a method presented within an attempt rather than an event, so a
completed OTP sign-in, which logs verify_otp_success twice, counts once.

docs/telemetry.md records how each dimension is derived, the fleet-wide form of
the query, and the publishability and retention decision.

Closes #154
@Bccorb

Bccorb commented Sep 12, 2026

Copy link
Copy Markdown
Contributor Author

Ripple, in merge order:

  1. this (the API: columns, header, route)
  2. feat(authEvent): carry the telemetry dimensions recorded on every auth event seamless-auth-types#66 (AuthEventSchema carries the five columns; additive, bump it here afterwards so GET /admin/auth-events stops stripping them)
  3. feat(proxy): forward the browser user agent and pass the sign-in metrics route through seamless-auth-server#161 (forwards x-seamless-client-user-agent on every proxied call and passes /internal/metrics/sign-ins through)
  4. feat(overview): show sign-in outcomes by method, device and mail provider seamless-auth-admin-dashboard#259 (the Overview "Sign-in Outcomes" section, which needs the route and a published adapter)

Until 3 is deployed in front of a deployment, every row it writes has device_class = 'unknown'.

Bccorb added a commit to fells-code/seamless-auth-types that referenced this pull request Sep 12, 2026
…h event (#66)

seamless-auth-api now writes deployment_id, device_class, mail_provider, owner
and attempt_id on every auth_events row. AuthEventSchema is a plain object
schema, so it stripped them from GET /admin/auth-events and a dashboard could
not show which device or attempt an event came from.

All five are optional and nullable, so an event written before they existed
still parses. The two classifications stay plain strings rather than enums for
the same reason type does: a newer server may add a class an older consumer has
not heard of.

Pairs with fells-code/seamless-auth-api#306.

Co-authored-by: Brandon Corbett <Bccorb@users.noreply.github.com>
@Bccorb
Bccorb merged commit c7902d4 into main Sep 12, 2026
5 checks passed
@Bccorb
Bccorb deleted the feat/auth-path-instrumentation branch September 12, 2026 14:19
Bccorb added a commit to fells-code/seamless-auth-server that referenced this pull request Sep 12, 2026
…ics route through

The auth API now records a device class on every audit row, folded from the
request user agent, and reports sign-in outcomes by it. The adapter is the only
client it sees, so every row carried the adapter's own user agent and the
breakdown read "unknown" for every sign-in. Both adapters now send the browser's
User-Agent as x-seamless-client-user-agent beside x-seamless-client-ip on every
proxied call, trimmed and capped at 512 characters. The API honours it under the
same service-token rule as the address.

GET /internal/metrics/sign-ins is proxied on an access session the way the
funnel route is, with the window forwarded and pinned by the query forwarding
tables and the parity cases.

Depends on fells-code/seamless-auth-api#306.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Instrument the auth path: time to first login, deliverability, drop-off, device class

1 participant