Repository navigation
Create BatchSpanProcessorBurstBenchmark.java - #8911
adhamahmad wants to merge 2 commits into
Conversation
Pull request dashboard statusWaiting on the author · refreshed 2026-10-27 22:10 UTC Respond to 3 review items (e.g. link a commit, explain why not, ask a follow-up): Status above doesn't look right?
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #8911 +/- ##
============================================
+ Coverage 90.31% 90.34% +0.03%
- Complexity 10981 10984 +3
============================================
Files 1019 1019
Lines 30072 30072
Branches 3927 3927
============================================
+ Hits 27160 27170 +10
+ Misses 1952 1940 -12
- Partials 960 962 +2 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Hey @jack-berg , Appreciate any feedback on this. |
jack-berg
left a comment
jack-berg
left a comment
There was a problem hiding this comment.
Are you interested in following this up with work to try to improve things around this benchmark? If so, what is your plan?
Its not useful to land a benchmark unless we're going to use it as a way to measure optimization progress.
| /** Per-iteration totals for produced, exported, and dropped spans. */ | ||
| @AuxCounters(AuxCounters.Type.EVENTS) | ||
| @State(Scope.Thread) | ||
| public static class BurstCounters { |
There was a problem hiding this comment.
Here's the output on my machine:
Benchmark (burstSize) (exporterLatencyMs) (maxExportBatchSize) (maxQueueSize) Mode Cnt Score Error Units
BatchSpanProcessorBurstBenchmark.burst 6000 50 512 2048 avgt 5 268.292 ± 7.797 ms/op
BatchSpanProcessorBurstBenchmark.burst:droppedSpans 6000 50 512 2048 avgt 5 327312.000 #
BatchSpanProcessorBurstBenchmark.burst:exportedSpans 6000 50 512 2048 avgt 5 242688.000 #
BatchSpanProcessorBurstBenchmark.burst:gc.alloc.rate 6000 50 512 2048 avgt 5 9.000 ± 0.261 MB/sec
BatchSpanProcessorBurstBenchmark.burst:gc.alloc.rate.norm 6000 50 512 2048 avgt 5 2532199.579 ± 8299.326 B/op
BatchSpanProcessorBurstBenchmark.burst:gc.count 6000 50 512 2048 avgt 5 ≈ 0 counts
BatchSpanProcessorBurstBenchmark.burst:producedSpans 6000 50 512 2048 avgt 5 570000.000 #
BatchSpanProcessorBurstBenchmark.burst 6000 50 512 8192 avgt 5 641.490 ± 5.290 ms/op
BatchSpanProcessorBurstBenchmark.burst:droppedSpans 6000 50 512 8192 avgt 5 ≈ 0 #
BatchSpanProcessorBurstBenchmark.burst:exportedSpans 6000 50 512 8192 avgt 5 240000.000 #
BatchSpanProcessorBurstBenchmark.burst:gc.alloc.rate 6000 50 512 8192 avgt 5 4.088 ± 0.017 MB/sec
BatchSpanProcessorBurstBenchmark.burst:gc.alloc.rate.norm 6000 50 512 8192 avgt 5 2750498.600 ± 17012.829 B/op
BatchSpanProcessorBurstBenchmark.burst:gc.count 6000 50 512 8192 avgt 5 1.000 counts
BatchSpanProcessorBurstBenchmark.burst:gc.time 6000 50 512 8192 avgt 5 1.000 ms
BatchSpanProcessorBurstBenchmark.burst:producedSpans 6000 50 512 8192 avgt 5 240000.000 #
Presumably, the goal is to characterize the fraction of spans dropped in different load scenarios. And to use that as a reference for future work tweaking and tuning to reduce this fraction. If so, then can we configure jmh / this benchmark to output the fraction of spans dropped rather than the count?
| } | ||
|
|
||
| @Benchmark | ||
| public void burst(BurstCounters counters) { |
There was a problem hiding this comment.
Can we organize this around steady and bursty load scenarios, then compare drop percentage at different offered loads, queue sizes, and exporter latencies? The bursty scenario could use the issue’s approximately 6,000 spans over 30 ms. Currently, producing spans as fast as possible and flushing after every burst leaves the arrival rate undefined and prevents backlog from carrying forward. Two defined load shapes would give us a useful baseline without parameterizing every timing detail.
|
Hi @adhamahmad — just a friendly reminder that this pull request is waiting on you. The dashboard status comment has the open items and is kept current.
|
Add JMH benchmark for BatchSpanProcessor burst absorption
Description
Part of #7508.
Follows the approach agreed in the issue: "Add a benchmark/test scenario that reproduces bursty high-throughput conditions and reports span drop behavior consistently" (agreed by @jack-berg).
Benchmark only: no production code or defaults change. It is a baseline for comparing opt-in alternatives later.
Changes
BatchSpanProcessorBurstBenchmark. Each operation produces a burst (6,000 spans) and callsforceFlush(); to ensure the accepted spans are fully processed before the next burst.maxQueueSize,maxExportBatchSizeand exporter latency.producedSpans,exportedSpansanddroppedSpansas aux counters, updated during the measured operation. Drops come from the processor's internal telemetry (BatchSpanProcessorMetrics) and are checked against what the exporter completed; the benchmark fails if they disagree.Testing
Per 6,000-span burst (aux counters are per-iteration totals, divided by bursts):
With the default queue, 57% of the burst is dropped. Increasing the queue to 8192 eliminates the drops for this workload, but requires draining all 6,000 spans rather than the 2,560 spans accepted with the default queue.
Environment: Windows, Temurin JDK 21.0.10. Absolute times are platform-dependent.