Skip to content

fix(services): give managed PostgreSQL a larger shared memory ceiling - #220

Merged
vishr merged 2 commits into
mainfrom
fix/postgres-shared-memory
Oct 17, 2026
Merged

vishr merged 2 commits into
mainfrom
fix/postgres-shared-memory

Conversation

@vishr

@vishr vishr commented Oct 17, 2026

Copy link
Copy Markdown
Member

What this changes

Managed PostgreSQL ran with Docker's default 64 MiB /dev/shm. PostgreSQL's parallel queries put their dynamic shared memory segments there (dynamic_shared_memory_type = posix), so a large parallel hash join or sort fails with could not resize shared memory segment.

The generated service document for every managed PostgreSQL service (the Onebox-owned 18 image and the older official images) now sets shm_size:

  • 1 GiB with no resources.memory limit, or with a zero limit (which Docker treats as unlimited).
  • With a positive limit: a quarter of it, clamped to 256 MiB–1 GiB, and never more than half the limit.
  • A limit Compose cannot decode, or one that overflows when decoded, keeps the 1 GiB default; mem_limit is still emitted verbatim for the existing validation path.

The tmpfs allocates nothing up front, but pages PostgreSQL writes there count against the container's memory cgroup, which is why the ceiling is a fraction of the limit. The half-limit bound keeps small limits (128 MB, 256 MB) from getting a /dev/shm that could fill the whole cgroup and OOM-kill a backend, where the old 64 MiB default would have failed only the query.

No other driver changes. No project-file field is added.

The add-a-database guide documents the ceiling and corrects two statements this change made visible as wrong: that ob deploy never creates or replaces a service container, and that no deploy can stop a service. Deploy converges PostgreSQL services with declared extensions with docker compose up -d, so it recreates them when their rendered configuration changes.

No issue was filed first.

Closes #

Why this is correct

  • TestPostgresSharedMemory renders the service document for: no resources, empty resources, CPU-only, 1B, 128MB, 256MB, 512MB, 0B, 1GB (floor boundary), 2GB, 1.5GB, 4GB (cap boundary), 8GB, an overflowing 999999999999999999999999TB, and the official version 17 image. It asserts the unmarshalled shm_size and that mem_limit is exactly the authored value.
  • TestOtherServicesHaveNoSharedMemoryOverride asserts every other managed driver renders no shm_size, with and without a memory limit.
  • Parsing uses compose-go's types.UnitBytes, the same parser Compose applies to mem_limit. docker/go-units converts the parsed float to int64 without an overflow check; on amd64 an out-of-range value becomes MinInt64 (arm64 saturates to MaxInt64). A decoded limit of zero or below therefore falls back to 1 GiB, so the result is positive on both architectures.
  • internal/app/testdata/contract-verdicts.json: only the nine PostgreSQL service hashes changed; application hashes, other service hashes, verdicts and error codes are unchanged.
  • Ran: just check (fmt, vet, full go test ./..., generated docs, site build). Not run: the Docker e2e suite, and the tests on amd64 (the amd64 overflow path was reasoned from source; CI is its first execution).

Effect on the safety envelope

Managed PostgreSQL containers get a larger /dev/shm ceiling. Running services keep their current one until recreated: by ob service apply, or, for PostgreSQL services with declared extensions, by the first ob deploy after upgrading, because deploy converges those services and their rendered configuration changed. That recreation restarts the database. No /status/capabilities change.

Checklist

  • just check passes locally.
  • Tests cover the new behaviour, including the failure paths.
  • Generated documentation is current (just check verifies this).
  • I have accepted the CLA, or will when the bot asks on my first pull request.

https://claude.ai/code/session_019ifKpdgURZKS36s78SCmx5

Docker's default 64 MiB /dev/shm is too small for PostgreSQL's parallel
query segments, which fail with "could not resize shared memory segment".
Managed PostgreSQL now gets 1 GiB, or a quarter of a positive memory
limit clamped to 256 MiB-1 GiB and never more than half the limit. A
zero or unparseable limit keeps the 1 GiB default. The tmpfs allocates
nothing up front; used pages count against mem_limit, hence the fraction.

Existing services pick it up on the next ob service apply. PostgreSQL
services with declared extensions are recreated by the first ob deploy
after upgrading, because deploy converges them and shm_size changed.

Claude-Session: https://claude.ai/code/session_019ifKpdgURZKS36s78SCmx5
@vishr vishr mentioned this pull request Oct 17, 2026
4 tasks done
@vishr
vishr merged commit bd51e70 into main Oct 17, 2026
5 checks passed
@vishr
vishr deleted the fix/postgres-shared-memory branch October 17, 2026 17:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant