Skip to content

- valkey pubsub rework#191

Open
g3co wants to merge 14 commits into
masterfrom
dev/valkey-integration
Open

- valkey pubsub rework#191
g3co wants to merge 14 commits into
masterfrom
dev/valkey-integration

Conversation

@g3co

@g3co g3co commented Jan 19, 2026

Copy link
Copy Markdown
Contributor

No description provided.

@github-actions

github-actions Bot commented Jan 19, 2026

Copy link
Copy Markdown

📊 Performance Metrics

Performance Metrics (memory storage)

  • CPU: 0.84s (4 cores) • Goroutines: 8 • Threads: 8
  • Memory: 15.9MB heap • 49.3MB RAM • 39.6MB total • 274258 allocs
  • GC: 12 cycles (0.40ms avg)
  • FDs: 8/65536 (0.0%)

@github-actions

github-actions Bot commented Jan 19, 2026

Copy link
Copy Markdown

📊 Performance Metrics

Performance Metrics (postgres storage)

  • CPU: 0.94s (4 cores) • Goroutines: 10 • Threads: 7
  • Memory: 19.9MB heap • 45.9MB RAM • 40.9MB total • 294127 allocs
  • GC: 12 cycles (0.61ms avg)
  • FDs: 34/65536 (0.1%)

@github-actions

github-actions Bot commented Jan 19, 2026

Copy link
Copy Markdown

📊 Performance Metrics

Performance Metrics (cluster-valkey storage)

  • CPU: 0.86s (4 cores) • Goroutines: 14 • Threads: 10
  • Memory: 31.8MB heap • 72.7MB RAM • 49.6MB total • 345554 allocs
  • GC: 7 cycles (0.59ms avg)
  • FDs: 73/65536 (0.1%)

@github-actions

github-actions Bot commented Jan 19, 2026

Copy link
Copy Markdown

📊 Performance Metrics

Performance Metrics (nginx storage)

  • CPU: 0.14s (4 cores) • Goroutines: 6 • Threads: 6
  • Memory: 2.7MB heap • 16.3MB RAM • 3.4MB total • 13749 allocs
  • GC: 1 cycles (0
    5000.00ms avg)
  • FDs: 15/65536 (0.0%)

@github-actions

github-actions Bot commented Jan 19, 2026

Copy link
Copy Markdown

📊 Performance Metrics

Performance Metrics (dnsmasq storage)

  • CPU: 0.52s (4 cores) • Goroutines: 14 • Threads: 12
  • Memory: 18.5MB heap • 40.6MB RAM • 24.6MB total • 160088 allocs
  • GC: 5 cycles (1.09ms avg)
  • FDs: 73/65536 (0.1%)

@g3co

g3co commented Jan 20, 2026

Copy link
Copy Markdown
Contributor Author
Screenshot 2026-01-20 at 12 15 13 PM

All 3 tests have been executed sequentially, with the same test parameters and in the same environment. All storage data has been flushed before each test.

  1. V2 Valkey PubSub lib, current PR
  2. V1 Valkey PubSub lib, Use sharded pub/sub #163
  3. Old Redis PubSub implementation, master branch

@g3co

g3co commented Jan 20, 2026

Copy link
Copy Markdown
Contributor Author
Metric PubSub V2 PubSub V1 Redis PubSub
Delivery latency avg 0.438 ms 0.513 ms 0.437 ms
Delivery latency p95 1 ms 1 ms 1 ms
Delivery latency max 32 ms 29 ms 33 ms
HTTP req avg 118.5 us 141.5 us 137.4 us
HTTP req p95 190 us 218 us 207 us
Iteration avg 153.6 us 185.4 us 176.6 us
SSE throughput 885.71 885.70 885.79
HTTP failed 0% 0% 0%

@g3co

g3co commented Jun 17, 2026

Copy link
Copy Markdown
Contributor Author

Why these changes

The branch switches the bridge3 Valkey pub/sub from regular PUBLISH/SUBSCRIBE to sharded pub/sub (SPUBLISH/SSUBSCRIBE) on the native valkey-go client. Regular PUBLISH broadcasts every message to all cluster
nodes, so per-shard load doesn't scale with shard count. Sharded pub/sub routes each message only to the shard owning the channel — cutting per-shard CPU ~2× — and the per-channel receiver design (replacing the
single consumer goroutine) raises the fan-out ceiling ~20%. A pub/sub deadlock was also fixed along the way.

Result: same delivery semantics and throughput as the baseline (hardware-bound), but noticeably lower CPU/memory usage. Feature-complete and shipping-ready.

Why not merged

Only one validation is outstanding: a production-environment regression run to confirm there is no degradation versus the current prod version. Local and staging tests pass and the code review found no
regressions, but we couldn't reproduce/verify prod conditions — so the merge is held pending that check.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant