docs: open source and self-host plan, Phase O (D52, D53)

docs/decisions.md

@@ -53,3 +53,5 @@ Format: ID, date, decision, why. Add new entries at the bottom; do not rewrite h
 - **D49 (2026-10-21) OPEN: payment provider for worldwide sales (needed only for S-9, built last).** Owner asked whether the gateways accept international payments. Findings (web research 2026-10-21, vendor claims unverified): **Razorpay** accepts foreign cards in about 135 currencies but activation is on request, fees reported at up to ~3% + 18% GST, and whether an Indian merchant can settle in USD must be confirmed with them; **Stripe India** supports international payments (IEC required, INR payouts, 3D Secure) but the India program is invite-only; **merchant of record (Paddle, Dodo Payments)** becomes the legal seller and handles foreign VAT/sales tax, at roughly 5% + $0.50 per transaction (Paddle), no figures published for Dodo. Recommendation: a merchant of record for the first worldwide customers (no foreign tax registration, no foreign entity), Razorpay if mostly Indian customers or lower fees matter. Decide when starting S-9; questions for the owner: company or individual, customer mix, willingness to pay ~5%. Sources are in the PR description.
 - **D50 (2026-10-21) Launch plan and order of work for a public web launch with the first customer expected in 1 to 3 months and customers from anywhere.** Facts from the owner: no registered organization yet, product advertised to developers, not focusing on India. Decisions: (1) *Phase L (launch readiness) is added and takes priority over Phase 5 polish*; the full interactive site stays last, but a launch-lite site (getting started, API reference, sample code, pricing, status) is needed before advertising. (2) *Early customers are onboarded by hand through the admin API* (it already works) and pay through a payment link; self-service signup and automated checkout (S-9, S-10) follow once there are customers, which keeps the first release small. (3) *Updates D49:* because there is no company yet and customers are worldwide, a merchant of record is the most practical route, but the owner should first register a business (sole proprietorship, LLP or private limited, ask an accountant) because providers, AWS billing and customer contracts all prefer one; check that the chosen provider accepts a new sole proprietor before relying on it. (4) *Legal pages and a DPA are launch blockers* (worldwide customers, GDPR in practice). (5) The owner-side items (registration, bank account, AWS account, domain) have the longest lead time, so they start first while the agent builds the infrastructure code and the staging load test.
 - **D51 (2026-10-21) Validate demand before production spend; the product category is real but the independent market is small, so we need a wedge.** Owner asked whether this can be a business (web research 2026-10-21). Findings: the category exists with paying customers (Svix lists Brex, Lob, Lightspeed and Clerk; Hookdeck has a public pricing ladder from a free 10,000 events to $39 and $499 plans), but the independent vendors are small: Svix has raised roughly $2.6M to $13M depending on the source with an estimated 5 to 11 staff and revenue estimates around $1M, Hookdeck about $2.7M raised and 1 to 10 staff (all from low-quality directories, no company-reported revenue). No credible independent market size exists (one report-selling site claims $1.8B growing to $10.2B, not corroborated). The space has free tiers (Hookdeck), open-source and self-hosted options (Svix, Hookdeck Outpost, Hook0), and many companies simply build it themselves or use AWS services. Conclusion: viable as a small, profitable niche business if cost and focus are right, unlikely to be a large market, and not proven for us until a real customer pays. Decisions: (1) Phase L gets demand-validation items L-L12 and L-L13; (2) keep spending near zero (local stack and a staging environment only) until at least one design partner commits, because the production floor is about $200-250 per month (cost-model.md, unverified); (3) pick a wedge from customer conversations; our structural advantage is cost per event, not features. Sources are in the PR description.
+- **D52 (2026-10-21) The project becomes open source (Apache-2.0) and self-hostable with Docker Compose; the hosted paid service is postponed.** Owner changed course after the market review (D51): publish the repo so anyone can use it, goal adoption and reputation, public after a clean-up and a working quick start. Why Apache-2.0: it is the license companies are most comfortable adopting (permissive, patent grant), which matters more for adoption than protecting against cloud vendors; the trade-off is that someone else may sell it as a service, which the owner accepted by choosing adoption over a protected hosted business (AGPL was the alternative). Consequences: Phase O is now the main track; Phase L (company registration, AWS account, payments, legal pages for a paid service) and S-9/S-10 (checkout, self-service signup) are postponed; the developer docs and sample code (S-1..S-5, S-7) move into the public launch; costs drop to zero for the owner. Honest note recorded: open source lowers the barrier to try the product but does not remove the need for a reason to choose it, so the cost per event and one-command simplicity are the wedge (D51).
+- **D53 (2026-10-21) Self-host mode uses PostgreSQL for queues, delivery state, rollups and the event archive; Valkey is optional; AWS adapters stay as an alternative backend.** Why: a self-hoster wants one `docker compose up` with as few moving parts as possible. Today's local stack uses Moto, an AWS test emulator that is not durable or meant for traffic, so it is demo-only. Postgres is already required for configuration (D21), supports a durable queue with `FOR UPDATE SKIP LOCKED` at a few thousand messages per second on one instance, and gives backup/restore for everything in one place. Valkey stays optional for shared rate limits (without it Ingest uses in-process limits, which is exact for one Ingest instance). The event archive defaults to Postgres (compressed batches with a retention job) and can use any S3-compatible store; MinIO is not bundled because of its license and maintenance status. KMS is replaced by a master key from an environment variable or file (envelope encryption, rotation documented); KMS stays available. Trade-offs accepted: a Postgres queue will not reach the 30,000 events/s Stage 2 design (that remains the AWS backend's job, `architecture.md` section 10), and the self-host guide states the supported throughput honestly. Port interfaces in `src/Abstractions` (D37) make this an additional adapter project (`src/Postgres` grows the queue/state/archive adapters), not a rewrite.

docs/roadmap.md

@@ -55,7 +55,29 @@ Status: **v1 (2026-10-20)**, derived from the approved architecture v4 (D20-D28)
 | P4-1 | Orchestrator (`webhookgateway-orchestrator`), per `orchestrator-spec.md`; owner approval of that spec needed first | L+ | risky | P0-3 |
 | P4-2 | Stage 2 swaps from `architecture.md` section 10, each when its trigger fires | per item | risky | P3-4 |
 
-## Phase L: launch readiness (added 2026-10-21; target first paying customer within 1 to 3 months)
+## Phase O: open source and self-host (added 2026-10-21, now the main track; supersedes the AWS-first launch order of Phase L)
+
+Owner decision: make the repo public (Apache-2.0), goal adoption and reputation, run on Docker Compose on a local machine or any server. The AWS adapters stay as an optional backend and a "deploy on AWS" guide can follow; Phase L items about registering a company, payment and production AWS are postponed until there is demand (D52, D53).
+
+| ID | Work item | Size | Notes | Depends |
+|---|---|---|---|---|
+| O-1 | **Public-repo hygiene:** `LICENSE` (Apache-2.0), `NOTICE`, scan the full git history for secrets/personal data, check dependency licenses, `README.md` (what it is, features, architecture picture, quick start, status), `CONTRIBUTING.md`, `CODE_OF_CONDUCT.md`, `SECURITY.md` (how to report), issue and PR templates, repo description and topics | M | do first; blocks going public | |
+| O-2 | **PostgreSQL queue adapters** (`Backend:Mode=Postgres`): durable queue table with visibility timeout, delay, receive count and dead-letter, `SELECT ... FOR UPDATE SKIP LOCKED`; implements the event and delivery publishers/sources and the retry publisher | L | replaces SQS for self-host; target a few thousand messages per second on one Postgres | |
+| O-3 | **PostgreSQL state adapters:** delivery state + due index, rollups (upsert add), idempotency/quota when Valkey is absent | M | replaces DynamoDB | O-2 |
+| O-4 | **Archive adapter without S3:** Postgres (compressed batches) by default with a retention job, S3-compatible storage (any endpoint) optional | M | replaces S3 for self-host; the existing S3 adapter works with any S3-compatible service | |
+| O-5 | **Secrets without KMS:** a production-grade key provider from an environment variable or mounted file (envelope encryption with a master key), documented key rotation; KMS stays optional | S | D45 keeps KMS for AWS | |
+| O-6 | **One-command stack:** `docker compose up` brings up migrator, Ingest, Fan-out, Dispatcher, ControlPlane, PostgreSQL (+ optional Valkey) with generated secrets and healthchecks; prebuilt images published to GitHub Container Registry from CI | M | | O-2..O-5 |
+| O-7 | **Quick start that works in 5 minutes:** create a tenant and key, register an endpoint, send an event, see it arrive (a bundled demo receiver container that verifies the signature and prints events), plus a smoke test script run in CI | M | | O-6 |
+| O-8 | **Self-host docs:** configuration, upgrading and migrations, backups and restore, sizing (events per second per CPU, Postgres tuning), reverse proxy and TLS example (Caddy), monitoring (logs, metrics), security hardening, troubleshooting | M | | O-6 |
+| O-9 | **Metrics endpoint** (Prometheus format) and a sample Grafana dashboard: accepted, delivered, failed, retries, queue depth and age, latency | M | | O-6 |
+| O-10 | **Minimal built-in UI** (read-only admin page served by the ControlPlane: tenants, endpoints, recent failures, replay) so a self-hoster is not blind without the portal | L | the full portal (P2-2..P2-5) stays optional | O-6 |
+| O-11 | **Client SDKs / sample code** in C#, Java, PHP, Python, Node.js, Go, Ruby, cURL, tested in CI (this is S-5 from Phase 5, now part of the public launch) and the docs site (S-1..S-4, S-7) | L | | O-7 |
+| O-12 | **Release process:** semantic versions, changelog, signed images, GitHub Releases, upgrade notes, CI on pull requests from forks | S | | O-6 |
+| O-13 | **Launch the project:** announcement post, Show HN / Reddit / dev.to, comparison page honest about Svix/Hookdeck/others, "good first issue" labels, a discussion board | S | after O-7 and O-8 | O-8 |
+
+Going public (flip the repo to public) after O-1 and O-7. The load-test plan (LT items) now runs on a single Docker host and on AWS as an optional benchmark.
+
+## Phase L: launch readiness (added 2026-10-21; POSTPONED by D52: only needed if a hosted paid service is started later)
 
 Context from the owner: no registered organization yet, product launched on the web, advertised to developers, customers expected from anywhere in the world. "Owner" items need the owner (legal, money, accounts); "agent" items are mine. Not legal or tax advice: confirm the owner items with an accountant.
 

docs/worklog.md

@@ -36,6 +36,12 @@ Resume point for agents after a sudden stop, crash, or new session. **Read this
 - **Outcome:** `architecture.md` v4 approved; decisions D20-D28 logged; `contracts.md` v2; `database.md` rewritten config-only (still **in review**); `docs/roadmap.md` lists the work items; plan numbers still provisional.
 - **Resume from:** see the next entry (roadmap execution).
 
+### 2026-10-21 Open source and self-host (Phase O) - current main track
+- **Status:** in progress
+- **Why:** owner decided to publish the repo (Apache-2.0) for adoption and reputation, running on Docker Compose; hosted business postponed (D52, D53). Owner answers: goal adoption, Apache-2.0, real self-host backends, public after clean-up + quick start.
+- **Plan:** O-1 hygiene and license first, then O-2..O-5 Postgres/self-host adapters, O-6 compose stack, O-7 quick start, then docs/SDKs/UI as in Phase O. Go public after O-1 and O-7.
+- **Resume from:** O-1 (LICENSE, README, history secret scan), then O-2 Postgres queue.
+
 ### 2026-10-21 Market reality check (D51)
 - **Status:** done (answer given to the owner); follow-ups L-L12, L-L13 on the roadmap
 - **Why:** owner asked whether this can be a business and how big the market is.